RAG Hallucination Root-Cause Analyzer

The problem

RAG pipelines often produce fluent but factually incorrect or poorly cited answers, making them risky for production. Developers frequently struggle to pinpoint whether the failure lies in the chunking strategy, vector retrieval, reranking logic, or the prompt itself.

What it does

• Identifies the exact failure stage across the 21-point RAG lifecycle, from corpus quality to citation binding.
• Performs a design-level audit of chunking configurations, metadata schemas, and retrieval logs to find hidden bottlenecks.
• Diagnoses specific grounding issues like lost-in-the-middle context, stale index skew, and citation fabrication.
• Produces a 100-point grounding score and a severity-ranked remediation plan for engineering teams.

Frameworks & tools

Compatible with any RAG stack including Pinecone, Weaviate, Milvus, LangChain, LlamaIndex, and various embedding models.

Why this beats prompting it yourself

General LLMs often overlook subtle retrieval-stage failures like metadata over-filtering or embedding dimension mismatches. This skill enforces a structured 23-step diagnostic workflow that separates symptoms from root causes, ensuring no stage of the pipeline is ignored.

Use cases

• Pre-release safety audits to determine if a RAG workflow is ready for production.
• Debugging specific user reports of hallucinations or missing citations in search results.
• Optimizing chunk size and overlap configurations based on evidence-span requirements.
• Establishing an evaluation matrix for multi-hop questions or complex document sets.

Known limitations

This skill does not query live production systems, execute retrieval scripts, or modify vector indexes directly. It relies on provided logs, configurations, and document samples for analysis.