Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIDeepSeekDeepSeekMuseMuse+22 more

    AI Agent User Experience Designer

    1

    Systematic RAG evaluation tool to detect hallucinations, verify citations, and audit retrieval accuracy.

    $23.99

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    AI Agent User Experience Designer

    AI Agent User Experience Designer

    Example session with this skill installed

    I have a RAG agent answer and the retrieved passages. The agent says our refund policy is 30 days, but the document says 14. Can you evaluate this for grounding and citation quality?

    • Read your context and instructions
    • Compiled the ai agent user

    I have evaluated the agent answer against the provided refund policy document. There is a material contradiction between the source evidence and the generated response.

    Evaluation overview

    • Agent or system: Customer Support RAG
    • Knowledge-base scope: Internal Refund Policies
    • Evidence supplied: 1 Policy PDF excerpt, 1 Agent Answer, 2 Retrieved Passages
    • Test cases evaluated: 1 (Factual Lookup)
    • Evaluation limitations: Single test case; cannot assess retrieval recall without full corpus access.

    Overall results

    DimensionScoreExplanation
    Answer faithfulness0/5Answer contradicts the 14-day policy stated in the source.
    Retrieval relevance5/5The correct policy section was successfully retrieved.
    Citation support1/5Citation points to a passage that refutes the claim made.
    Evidence coverage5/5The retrieved passage contained the complete answer.

    Test case results

    • Test ID: REF-001
    • Question: "What is the duration of the refund policy?"
    • Status: Contradicted

    Failure diagnosis: Answer adds unsupported details (30 days) that conflict with retrieved evidence (14 days). Potential model over-reliance on internal training data over RAG context.

    Priority recommendations

    • Priority: Critical
    • Observed issue: Hallucination of 30-day window despite 14-day evidence.

    Recommended action: Increase prompt weighting for provided context; implement a "check-your-work" step before output.

    • Expected improvement: Reduction in factual contradictions.

    Next steps

    1. Review the system prompt to ensure "grounding instructions" are explicit.
    2. Run a batch test using the 14-day policy to see if this is a recurring model bias.
    3. Check if any stale 30-day policy documents exist in the vector store.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    RAG systems often fail because developers cannot pinpoint if the error lies in retrieval, grounding, or knowledge coverage. This skill provides a systematic framework for auditing AI agent responses against source documents to identify hallucinations and retrieval gaps.

    What it does

    • Evaluates grounding by comparing agent answers directly against provided source passages for factual support.
    • Audits retrieval quality to determine if the system fetched the correct context or missed relevant document chunks.
    • Detects hallucinations by identifying material claims that lack evidence in the supplied knowledge base.
    • Assesses abstention logic to see if the agent correctly identifies out-of-scope questions or incorrectly refuses supported ones.
    • Generates regression tests including multi-hop, ambiguous, and out-of-scope cases for continuous performance tracking.

    How it works

    1. Supply the knowledge base excerpts, agent answers, and any available retrieval logs.
    2. The skill analyzes the evidence to separate retrieval-stage failures from generation-stage errors.
    3. It generates a quality report with scores (0-5) across dimensions like faithfulness and citation support.
    4. It provides prioritized recommendations for chunking, metadata, or threshold adjustments.

    Frameworks & tools

    This tool works with any RAG architecture, vector database, or LLM output. It is framework-agnostic and processes text-based logs and document excerpts from systems like LangChain, LlamaIndex, or custom stacks.

    Why this beats prompting it yourself

    Generic prompts often conflate writing style with factual accuracy. This skill enforces a strict separation between retrieval relevance and answer grounding, ensuring you don't approve a fluent but hallucinated response.

    Use cases

    • Pre-deployment audits to verify a new RAG pipeline meets accuracy requirements.
    • Regression testing after changing embedding models or chunking strategies.
    • Hallucination debugging to find exactly where a specific user query went wrong.
    • Citation verification to ensure agent references actually point to the supporting evidence.

    Known limitations

    Cannot connect to live databases or production telemetry. Analysis is limited to the excerpts, logs, and documents provided in the session.

    Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for LLM Ops.

    Requires manual input of document excerpts and logs. Cannot execute live queries against vector stores. Limited to text-based analysis of provided samples.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before buying a skill.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 15 days ago

    Permissions

    No special permissions declared or detected

    Listed15 days ago

    What's inside

    Frequently Asked Questions

    This skill functions as a diagnostic layer that separates retrieval errors (fetching the wrong data) from generation errors (hallucinating based on correct data), allowing you to pinpoint exactly where your pipeline is failing.

    The skill is framework-agnostic and works with any text-based logs or document excerpts from platforms like LangChain, LlamaIndex, or custom-built stacks.

    You will receive the complete evaluation framework, including specialized logic for grounding audits, retrieval assessment, and a reporting template that generates scores and improvement recommendations.

    No direct integration is required; you simply provide the text logs and document excerpts from your system, and the skill performs the audit based on the data you upload.

    Yes, it is designed to generate complex test scenarios, including multi-hop and out-of-scope queries, specifically to help you build a robust regression suite for future model or chunking updates.

    New in LLM Ops