Probe: skill scenarios (writing-skills test of the replication-materials skill)
probes/2026-10-19-skill-scenarios/README.md · Status: Done
# Probe: skill scenarios (writing-skills test of the replication-materials skill) | Item | Value | |---|---| | Status | Done | | Date | 2026-10-19 | | Written by | Claude Code (Opus 5.5) | This probe tested the skill text with the test-first method of the superpowers writing-skills skill. Fresh subagents acted as Claude Code in scratch projects. Each scenario ran twice: first without the skill (baseline), then with the skill. | File | What it measures | |---|---| | `make-scenarios.mjs` | Builds the four scratch projects from the templates: set-up with no remote, a decision, the end of a milestone, a paid-call request. | | `baseline.md` | What four agents did without the skill. Two failed: set-up invented its own folder, and the milestone close broke the build. | | `with-skill.md` | The same four scenarios with the skill. All four passed. The controller checked each folder with `git status` and the build. | ## Run ```bash node make-scenarios.mjs <path to claude-skills> <output folder> <tag> ``` The scenario prompts are in the session transcript `b36ad167` (Task 9 of the execution ledger). ## Versions and calls - Node 22.14.0. Subagents: Sonnet 5.5. Skill commit `189d07a`. - No paid API call. No network call. No GitHub change.