Development
Skill Creator
Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill.
Create new skills, modify and improve existing skills, and measure skill performance. Use when users want to create a skill from scratch, edit, or optimize an existing skill, run evals to test a skill.
A skill for creating new skills and iteratively improving them.
Start by understanding the user's intent:
skill-name/
├── SKILL.md (required)
│ ├── YAML frontmatter (name, description required)
│ └── Markdown instructions
└── Bundled Resources (optional)
├── scripts/ - Executable code
├── references/ - Docs loaded into context
└── assets/ - Files used in output
Skills use a three-level loading system:
After writing the skill draft, come up with 2-3 realistic test prompts. Save test cases to evals/evals.json.
{
"skill_name": "example-skill",
"evals": [
{
"id": 1,
"prompt": "User's task prompt",
"expected_output": "Description of expected result",
"files": []
}
]
}
For each test case, spawn two subagents:
Draft quantitative assertions for each test case.
{
"total_tokens": 84852,
"duration_ms": 23332,
"total_duration_seconds": 23.3
}
Focus improvements on test cases with specific complaints.
The description field determines whether Claude invokes a skill. After creating or improving a skill, optimize the description for better triggering accuracy.
Create 20 eval queries (mix of should-trigger and should-not-trigger).
python -m scripts.run_loop \
--eval-set <path-to-trigger-eval.json> \
--skill-path <path-to-skill> \
--model <model-id> \
--max-iterations 5
Take best_description from the JSON output and update the skill's SKILL.md frontmatter.