Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIDeepSeekDeepSeekMuseMuse+22 more

    AI Feature Eval Writer

    1

    AI Feature Eval Writer does the design and writes the artifacts.

    $14

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    3 usesSecurity scanned
    AI Feature Eval Writer

    AI Feature Eval Writer

    Example session with this skill installed

    I'm building a RAG bot for our internal HR docs. I need to make sure it doesn't hallucinate benefits info and always cites the right PDF. Give me an eval suite design.

    • Read your context and instructions
    • Compiled the ai feature eval

    Here is the eval suite design for your HR RAG bot, written to three files (eval-design-worksheet.md, llm-judge-prompt-template.md, sample-eval-suite.md):

    Metrics (each mapped to a failure mode)

    • Faithfulness (blocking): LLM-as-judge rubric that flags any benefits claim not grounded in the retrieved source.
    • Citation accuracy (blocking): programmatic check that every cited PDF exists and matches the answer's source.
    • Correct refusal (warning): checks the bot declines when the docs don't cover the question.

    Golden dataset: 10 starter cases spanning common questions, edge cases, an adversarial "invent a policy" prompt, and a correct-refusal case.

    Rubrics: anchored 1-to-5 scales with observable descriptions per level.

    Judge prompt: mitigates position, length, self-preference, and halo bias, with a human calibration step before you rely on it.

    CI gate: fail the build if faithfulness drops below your threshold on any prompt or model change.

    Note: this designs and writes the artifacts — you plug them into your own test framework or CI to run them. It does not execute evals or call models.

    HR RAG bot, written to three files (eval-design-worksheet.md

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Teams ship LLM features with unit tests for the plumbing and vibes for the model — then a prompt tweak or model upgrade quietly breaks quality and nobody notices until users do. Evals are the missing test suite, and writing them is a craft: bad rubrics measure fluency instead of correctness, and naive judge prompts have known biases. AI Feature Eval Writer does the design and writes the artifacts. Describe your feature, what good looks like, and the failure modes you fear, and it produces the eval plan — each failure mode becomes its own metric with the cheapest grader that works (programmatic checks first, LLM-as-judge only where genuinely needed) and a blocking or warning threshold; a golden dataset design with 8 to 12 concrete starter cases including adversarial and correct-refusal cases; anchored 1-to-5 rubrics with observable level descriptions; ready-to-use judge prompts that mitigate position, length, self-preference, and halo biases, with a human calibration step before you trust them; and the CI regression gate that runs on every prompt or model change. The download includes three reference files: the eval-design worksheet, the LLM-as-judge prompt template with bias guards, and a complete worked sample suite. It designs and writes the artifacts; it does not execute evals or call models. Works with Claude Code, Cursor, Codex CLI, Gemini CLI, and any SKILL.md agent.

    Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for LLM Ops.

    This skill designs and writes the eval artifacts; it does not execute evals, run test suites, call or benchmark models, or connect to any service. You run the resulting evals in your own test framework or CI provider. It generates a starter golden dataset (about 8 to 12 cases) that you should expand with your real production examples, and its LLM-as-judge prompts require a one-time human calibration pass before you trust the scores. Output is Markdown text and prompt templates, not runnable code or a hosted dashboard, and it does not include automatic, ongoing, or lifetime updates. The download contains three reference files: eval-design-worksheet.md, llm-judge-prompt-template.md, and sample-eval-suite.md.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before buying a skill.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    3 uses

    Times used to date

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 4 months ago

    Permissions

    Read Files
    Write Files

    The skill only reads the context you provide and writes Markdown artifacts (the eval plan, rubrics, judge-prompt templates, and worked sample suite) to your project. It needs Read Files to review any feature notes or spec you point it at and Write Files to save the generated artifacts. It does not use a terminal, make network requests, run evals, call or benchmark models, or read environment variables.

    Listed4 months ago
    Updated16 days ago

    What's inside

    Frequently Asked Questions

    This skill automates the creation of professional-grade evaluation suites, replacing 'vibes-based' testing with structured golden datasets, anchored rubrics, and LLM-as-judge prompts designed to catch regressions before they reach production.

    The skill is compatible with any AI agent that supports SKILL.md files, including Claude Code, Cursor, Codex CLI, and Gemini CLI, allowing you to generate eval artifacts directly within your existing development environment.

    You will receive the core skill file for your agent, an eval-design worksheet, a bias-guarded LLM-as-judge prompt template, and a complete worked sample suite to use as a reference for your own implementation.

    The skill focuses on the 'design and write' phase; while it provides the logic, prompts, and gates needed for CI/CD integration, it does not execute the evaluations or make model calls itself.

    The prompts generated by this skill include specific guards to mitigate common LLM-as-judge issues like position bias, length bias, and self-preference, ensuring your metrics measure actual quality rather than model artifacts.

    New in LLM Ops