Works with the AI Agents you already use
Claude Code
Cursor
Codex CLI
Muse
Dots+21 moreClaude CodeCursorCodex CLIMuseDots
Manus
Grok
Grok Bot+18 agentsAI Agent Hallucination & Grounding Auditor
Audit AI agent responses and RAG systems for hallucinations, weak grounding, and broken source attribution.
$34.99
AI Agent Hallucination & Grounding Auditor
Example session with this skill installed
Audit this RAG response.
Query: What is our policy on remote work?
Response: Employees can work remotely 3 days a week.
Context: [Doc 1] 'Hybrid work allows for 2 days of remote work.'
- Read your context and instructions
- Compiled the ai agent hallucination
The audit is complete. I have reviewed the response against the provided context from Doc 1. The claim regarding the number of remote days is directly contradicted by the evidence.
Audit Report: Remote Work Policy
1. Audit scope
- System: RAG-based internal policy assistant.
- Task: Informing employees about remote work limits.
- Artifacts: User query, agent response, 1 context document.
2. Executive verdict
Critical Risk: The agent provided a specific numeric limit (3 days) that directly conflicts with the source document (2 days), creating a high risk of policy misinformation.
3. Claim evidence table
| Claim | Source Locator | Status | Explanation |
|---|---|---|---|
| Remote work allowed | Doc 1 | GROUNDED | Document confirms hybrid work is allowed. |
| 3 days a week | Doc 1 | CONTRADICTED | Doc 1 explicitly states '2 days', not 3. |
4. Retrieval assessment
The retrieval was successful in finding the relevant document (Doc 1), but the generation stage failed to accurately extract the numeric value.
5. Grounding test suite
| Test ID | Category | Query | Expected Grounded Behavior |
|---|---|---|---|
| T-001 | Contradiction | Remote work policy | Response must state '2 days' as per Doc 1. |
| T-002 | Abstention | Office pet policy | Agent should abstain if no documents are retrieved. |
Next steps
- Update the system prompt to enforce strict numeric extraction from context.
- Implement a citation check that flags responses where numeric values do not appear in the source chunks.
- Add T-001 to the automated evaluation suite for this agent.
Connects securely to your tools. The creator never sees your data.
About this skill
AI agents frequently present factual errors as truth, leading to failed automations and eroded user trust. This skill provides a rigorous auditing framework for developers to identify where RAG systems and autonomous agents break down. It focuses specifically on grounding, source attribution, and evidence sufficiency rather than general "quality."
What it does
- Claim decomposition breaks agent responses into atomic, independently checkable factual statements.
- Evidence tracing maps specific claims to source IDs, chunk locators, or tool outputs to verify support.
- Retrieval validation analyzes query-to-document relevance, chunk boundaries, and coverage gaps.
- Grounding classification labels claims as grounded, unsupported, contradicted, or misattributed.
- Test generation creates reproducible test cases with specific failure conditions and pass criteria.
How it works
- Define the boundary by providing the agent's answer, the original user query, and the retrieved context or tool logs.
- Execute the audit using the specialized workflow to trace claims against the provided evidence.
- Review the audit report which includes a claim-evidence table and a retrieval assessment.
Frameworks & tools
Works with any RAG architecture, vector database (Pinecone, Weaviate, Milvus), or agent framework (LangChain, LlamaIndex, CrewAI). It analyzes the artifacts these systems produce, such as retrieval traces and model outputs.
Why this beats prompting it yourself
Standard prompts often fail to distinguish between "reasonable sounding" text and text actually supported by data. This skill enforces a strict evidence boundary and classification taxonomy (e.g., UNSUPPORTED vs. CONTRADICTED) that prevents the auditor from hallucinating its own justifications.
Use cases
- Auditing RAG pipelines to find why specific queries return incorrect or stale data.
- Verifying that autonomous agents have factual premises for high-impact decisions.
- Generating regression test suites for LLM application deployments.
Known limitations
Cannot verify external facts or live system states unless the specific artifacts or tool outputs are provided in the context. Results are limited to the evidence boundary established by the user.
Cannot verify live systems or external facts without provided artifacts. Audit quality depends entirely on the completeness of retrieved chunks and logs supplied by the user.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 8 days ago
Permissions
No special permissions declared or detected