Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIDeepSeekDeepSeekMuseMuse+22 more

    Verified Best-of-N: Test-Gated Multi-Patch Selection

    1

    Run 3-5 independent fix attempts, drop any that fail your tests, pick the winner by blind A/B review, apply only on approval.

    $19

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    Verified Best-of-N: Test-Gated Multi-Patch Selection

    Verified Best-of-N: Test-Gated Multi-Patch Selection

    Example session with this skill installed

    Use verified-best-of-n with 3 candidates. The retry helper in src/retry.js runs one attempt too few when maxAttempts is set. Fix it, keep the public API, don't weaken tests. Validate with npm test. Don't apply anything until I've seen the report.

    • Read your context and instructions
    • Compiled the verified best-of-n

    Verified Best-of-3 result: candidate-2 selected. 2 of 3 candidates passed npm test; the baseline on the unmodified code failed 2 of 4 tests, confirming the tests reproduce the bug.

    CandidateTestsScoreNote
    candidate-2passed1.00Minimal loop-bound fix; adds a regression test for maxAttempts: 1
    candidate-3passed0.00Also raised the default budget from 3 to 5 and loosened assert.equal(calls, 3) to assert.ok(calls >= 3)
    candidate-1failed—Still ran only 2 of 3 attempts; 2 tests failed, so it never reached judging

    Why it won: two independent judges compared the survivors blind, once in each A/B order, and both preferred candidate-2 on all three criteria (spec, output, error handling). It changes only the loop bound and adds coverage, while candidate-3 changed a default the task never mentioned and weakened an existing assertion to get green. AB/BA agreement: 100%.

    Winner patch: src/retry.js (+1/-3), test/retry.test.js (+14/-0), SHA-256 81b97548878c…

    -  // Intentional demo defect: maxAttempts should include the final allowed attempt.
    -  const attemptsToRun = Math.max(1, maxAttempts - 1);
    -  for (let attempt = 1; attempt <= attemptsToRun; attempt += 1) {
    +  for (let attempt = 1; attempt <= maxAttempts; attempt += 1) {
    

    Report: .git/verified-best-of/runs/20261024-163828-c6b7a8/report.md

    Apply this patch to your working tree? Nothing is committed either way.


    After you reply "apply it": the helper re-checked the repository, HEAD, a clean tree and the patch SHA-256, applied the patch (not staged, not committed) and re-ran npm test: 5 of 5 tests passed. Your original checkout was never touched before that point.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    One coding-agent attempt can look right and still be the worse fix. Verified Best-of-N makes your agent solve the same task several times in isolation, lets your own test suite reject the broken attempts, compares the survivors blind, and only then offers one patch for you to apply.

    How it works

    1. Plan gate - your agent restates the task, candidate count (3 or 5), test command and setup command, and waits for your OK.
    2. Isolated candidates - each attempt runs in its own detached git worktree. Agents with subagents (such as Claude Code) run them in parallel; others run them one by one.
    3. Test gate - every patch is captured with a SHA-256 hash and run against your test command. Failing candidates never reach review.
    4. Blind A/B judging - survivors are compared as anonymous "Patch A / Patch B" on three criteria: specification adherence, output match, and error-signal handling. Each pair is judged in both orders to cancel position bias, and consistency is reported.
    5. Report - ranking, scores, the judges' reasons, excluded candidates with logs, and warnings (for example, a candidate that loosened or deleted tests).
    6. Apply gate - nothing touches your checkout until you approve. Before applying, the helper re-checks the repository, HEAD, a clean tree and the patch hash, then re-runs your tests. It never commits.

    What you get

    • SKILL.md - a step-by-step workflow any SKILL.md-compatible agent can follow
    • scripts/bestof.py - a dependency-free Python CLI (init, validate, blind judging files, rank, integrity-checked apply, cleanup)
    • references/judging-rubric.md - the three criteria with evidence rules and a worked example
    • references/prompts.md - candidate and judge subagent prompts, plus a single-agent fallback
    • examples/sample-report.md - a real Best-of-3 report

    Safety by design

    • No network access, no package installs, no API keys. The helper is pure Python standard library.
    • No commits, pushes, stashes or resets. Worktrees are removed with git worktree remove only.
    • All run state lives in .git/verified-best-of/, invisible to git status.
    • Flags candidates that touch, loosen or delete tests, so a "green" result cannot hide a weakened suite.
    • Validation runs your own test command on your machine, so use it on repositories you trust.

    Good fits

    Off-by-one and boundary bugs, flaky logic, tricky edge cases, refactors that must not change behaviour, and any change where being right matters more than being fast.

    Auto-detects npm / pnpm / yarn / bun test, pytest (uv, poetry or plain), cargo test, go test ./... and make test, or takes any command you give it.

    Built by the author of dsh-llm-verifier, the open-source DeepSeek Harness plugin. This skill is a standalone, agent-agnostic rewrite of that workflow: no Harness, no Node build and no DeepSeek API key required.

    Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Code Quality & Review.

    Requires a clean git working tree with at least one commit and a runnable automated test command. Cost scales with the candidate count (roughly N times one attempt plus judging). Validation runs your repository's own code on your machine, so use it only on repositories you trust. In agents without subagents, candidates are produced one after another and judging is self-judged rather than blind; the report says so. Needs permission to run git commands that write under .git (git worktree). Tested on macOS and Linux, not on native Windows.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before buying a skill.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 23 days ago

    Software
    Git 2.17+Python 3.8+

    Permissions

    Terminal / Shell
    Read Files
    Write Files

    File Scopes

    ./**
    .git/verified-best-of/**

    The helper script makes no network calls, installs nothing and reads no credentials. It passes your existing shell environment through to the test command you approve (adding CI=1 so runners stay non-interactive). Validation runs inside isolated git worktrees, then once more in your checkout only after you approve the winner.

    Listed23 days ago

    What's inside

    Frequently Asked Questions

    No. Candidates and reviews come from the agent you already use. The helper script is offline Python with no dependencies, and it reads no credentials.

    No. There are two approval gates: you approve the plan before any work starts, and you approve the winning patch before it is applied. The winner is applied to your working tree only, never staged or committed, and only after integrity checks on the repository, HEAD and the patch hash.

    Nothing is applied. The report lists each failed candidate with its exit code and log so you can refine the task or the test command. A baseline run on the unmodified code also tells you up front whether your tests reproduce the problem.

    Roughly N times a single attempt, plus review. Use 3 candidates and single-order judging for a cheaper run, and 5 candidates for high-stakes changes.

    Any project with a test command: it auto-detects JavaScript/TypeScript (npm, pnpm, yarn, bun), Python (pytest via uv, poetry or plain), Rust, Go and Makefile projects, or takes any command you give it. It works in any agent that can run shell commands; agents with subagents, such as Claude Code, run candidates and blind judges in parallel.

    New in Code Quality & Review