- Home
- Skills
- Prompt & Skill Engineering
- llm-eval-mutation-harness
Works with the AI Agents you already use
Llm Eval Mutation Harness
Prevent silent LLM prompt and RAG regressions using 30-case golden eval sets, mutation-checked deterministic graders, and token cost/latency release gates.
$19
llm-eval-mutation-harness
Example session with this skill installed
We want to rewrite our extraction system prompt in src/prompts/extractor.ts and switch from temperature 0.2 to 0.0. Build a golden eval harness and verify there are no regressions.
- Read your context and instructions
- Compiled the llm-eval-mutation-harness
LLM Pipeline Evaluation & Mutation Receipt
| Metric | Baseline (v1.2, temp=0.2) | Candidate (v1.3, temp=0.0) | Delta | Gate Verdict |
|---|---|---|---|---|
| Golden Pass Rate (32 cases) | 29/32 (90.6%) | 32/32 (100.0%) | +9.4% | PASS |
| Critical Schema Regressions | 0 | 0 | 0 | PASS |
| Avg Prompt + Completion Tokens | 1,840 | 1,390 | -24.5% | PASS |
| p95 Latency | 1,420 ms | 1,080 ms | -340 ms | PASS |
| Mutation Check (Guard Removed) | n/a | Failed 6/32 (case-04..09) | Grader Verified | PASS |
Connects securely to your tools. The creator never sees your data.
About this skill
Changing a system prompt, switching models, tweaking temperature, or refactoring a tool schema without running a quantitative evaluation harness is the #1 cause of silent production regressions in AI applications.
llm-eval-mutation-harness equips Claude Code, Cursor, Windsurf, and Copilot with a strict, reproducible evaluation engineering workflow:
- 30-Case Golden Eval Dataset Standard: Automatically scaffolds stratified test suites covering happy paths, adversarial prompt injections, malformed JSON boundaries, Unicode/empty inputs, and multi-step tool calls.
- Mutation-Checked Graders: Proves your evaluation assertions actually catch regressions by mutating the prompt guard or schema validator and verifying the eval suite fails before trusting a green pass rate.
- Single-Variable A/B Delta Reporting: Enforces changing one variable at a time (prompt, model, temperature, or parser) and generates a per-case regression matrix alongside p50/p95 latency and token-cost deltas.
- Zero-Regression Ship Gate: Blocks deployment if any critical safety/schema case regresses or if token cost per task increases by >15% without a matching accuracy gain.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Prompt & Skill Engineering.
Requires initial manual baseline recording. Performance gates for cost and latency require a minimum of 30 cases for statistical significance.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 10 days ago
Permissions
No special permissions declared or detected