prompt evolve

Stop guessing whether a prompt change actually worked. This skill replaces trial-and-error manual editing with a measured optimization loop that scores outputs against a test set, identifies specific failures, and evolves the prompt version-by-version. It treats prompt engineering like software engineering, persisting every iteration as a diffable file.

What it does

• Systematic optimization runs prompts against a defined test set to measure real performance gains.
• Version tracking saves every iteration in a generations/ directory for diffing and rollbacks.
• Failure analysis identifies specific test cases where the prompt fails to generate actionable improvement data.
• Multi-modal scoring supports deterministic checks like regex and JSON schema alongside LLM-based rubric grading.
• Progress reporting generates a report.md tracking the score curve and highlighting the best-performing version.

How it works

1. Initialize the workspace with a prompt.tmpl containing an {input} placeholder and a testset.jsonl file.
2. Execute iterations where you render the prompt, capture outputs, and score them using deterministic or interactive rubric scorers.
3. Persist results using the pevolve.py score command to save the current version and its performance metrics.
4. Refine the prompt based on reported failures and repeat until the target score is met or the loop plateaus.

Frameworks & tools

This skill uses a Python-based CLI (pevolve.py) for local file management and scoring logic. It integrates with critique-gate for rubric-based evaluation and is designed to work within Claude Code or Cursor environments.

Why this beats prompting it yourself

Manual tweaking is subjective and prone to regression where fixing one case breaks three others. This skill enforces a data-driven approach that ensures the "best" prompt is chosen based on evidence, not intuition, while maintaining a full audit trail of every change.

Use cases

• Optimizing an extraction prompt to ensure it strictly follows a specific JSON schema.
• Tuning a creative writing prompt against a multi-point quality rubric.
• Debugging edge-case failures in a complex classification task.
• Benchmarking prompt performance across different model versions.

Known limitations

The prompt template only supports a literal {input} placeholder. Deterministic JSON schema scoring is limited to checking for the presence of specific keys rather than full validation.