More screenshots

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIDeepSeekDeepSeekMuseMuse+22 more

    Research Evaluation Designer

    1

    Design an evidence-backed evaluation rubric for LLM, RAG, research, or agent outputs.

    $19.99

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    Research Evaluation Designer

    Research Evaluation Designer

    Example session with this skill installed

    Design a HYBRID evaluation rubric for a clinical-QA RAG system in the medical domain. Include the RAGAS core, domain-specific clinical safety dimensions, anchored human review, a HIPAA-related evaluation overlay, and a 20–30 example pilot. Do not present any threshold as production-validated without pilot evidence.

    • Read your context and instructions
    • Compiled the research evaluation designer

    Clinical-QA RAG Evaluation Blueprint

    System: clinical-QA RAG pipeline | Mode: HYBRID | Domain: medical

    DimensionMethodInitial thresholdEvidence status
    FaithfulnessClaim extraction plus NLI against retrieved context0.80Pilot target, not production validated
    Answer relevanceReverse-question generation plus cosine similarity0.80Pilot target
    Context precisionUseful chunks divided by total retrieved chunks0.70Pilot target
    Context recallGround-truth statements covered divided by total statements0.80Pilot target
    Clinical accuracyClaims checked against a curated medical-guidelines set0.85Domain-specific proposed metric
    Contraindication detectionDangerous recommendations flagged across seeded cases0.90Requires explicit pilot evidence before adoption

    Human anchors

    Coherence: 1 = contradictory or illogical; 3 = mostly coherent with minor gaps; 5 = fully coherent clinical reasoning.

    Completeness: 1 = misses most sub-questions; 3 = addresses the main question only; 5 = addresses every stated sub-question.

    Compliance overlay

    For the supplied HIPAA requirement, verify that the evaluation set contains no patient identifiers and retain provenance for clinical claims. This mapping supports evaluation design; it does not establish legal compliance.

    Pilot

    Use 20–30 stratified examples including retrieval misses, irrelevant context, contraindication traps, and complete answers. Double-rate a subset, examine disagreement, revise anchors, and recalibrate thresholds before any production gate.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Design a domain-specific evaluation rubric for LLM, RAG, research, or multi-agent outputs. Research Evaluation Designer selects evidence-backed dimensions, combines automated metrics with anchored human scales, adds compliance mappings only when supplied, rejects unsupported thresholds, and includes a calibration plan without pretending the rubric is already validated.

    You receive a dimension table, measurement methods, source register, human-rating anchors, compliance overlay, threshold status, and a pilot plan. The package includes the full SKILL.md workflow, RAGAS references, calibration guidance, compliance matrix, JSON rubric schema, deterministic checker, worked sample, tests, and a six-page PDF buyer guide.

    It designs rubrics; it does not run evaluations, generate a dataset, establish regulatory compliance, or claim production validity without a pilot.

    Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for LLM Ops.

    Designs evaluation rubrics only. It does not run evaluations, generate datasets, establish regulatory compliance, or claim production validity. Thresholds remain pilot targets until supported by calibration evidence.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before buying a skill.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 2 months ago

    Permissions

    No special permissions declared or detected

    Listed2 months ago

    What's inside

    Frequently Asked Questions

    This skill solves the problem of subjective or inconsistent AI testing by generating evidence-backed evaluation rubrics that combine automated metrics (like RAGAS) with anchored human-rating scales.

    The package includes a comprehensive workflow featuring the full SKILL.md, JSON rubric schemas, a deterministic checker, a calibration plan, and a six-page PDF buyer guide to assist with implementation.

    No, this skill is designed strictly for the architectural phase; it builds the framework and dimensions for evaluation but does not execute the actual tests or generate the datasets themselves.

    It is compatible with any LLM, RAG, or multi-agent system, providing a domain-specific framework that can be integrated into your existing quality assurance or AI governance pipelines.

    The Designer requires you to provide a dimension table and measurement methods; it will then output a structured rubric, calibration guidance, and a pilot plan to help you move toward production validity.

    New in LLM Ops