Claim-Grounding Auditor — Catch Unsupported Claims in AI Output

About This Skill

## What it does Claim-Grounding Auditor makes your agent treat every factual claim in a draft as a liability until it is grounded in a verifiable source. Given a draft — and the source material it was built from — the agent extracts every checkable claim, locates the supporting evidence, grades the strength of that source, and returns a structured substantiation report. It is domain-agnostic. It works on research reports, product copy, technical docs, RAG output, briefs, and summaries — any text where a confident-but-wrong sentence is a real cost. ## The classification framework Every claim is assigned exactly one status: - GROUNDED — directly supported by a strong, specific, citable source. - INFERRED — a reasonable conclusion assembled across sources or partly reasoned rather than stated. Flagged so a human can confirm the leap. - WEAK — supported only by a low-quality, outdated, circular, or non-authoritative source. Usable, at the author's risk. - UNSUPPORTED — no source found, or the source contradicts the claim. Must be fixed or removed. ## Source-strength hierarchy The agent ranks sources rather than treating all citations as equal: primary/authoritative/current first, then reputable secondary, then aggregated/tertiary, then weak signals (undated, self-referential, or marketing content). A claim supported only by weak sources cannot be graded higher than WEAK — no matter how confident the prose sounds. ## Fact vs. inference A core rule: the agent distinguishes what a source SAYS from what the draft CONCLUDES. If a sentence combines two facts into a new claim, that new claim is INFERRED, not GROUNDED — even when both inputs are solid. This is the most common way confident drafts smuggle in unverified conclusions, and the audit surfaces it explicitly. ## Sample output SUBSTANTIATION REPORT Draft: q3-market-brief.md | Sources: 4 | Claims checked: 12 [GROUNDED] "The API rate limit is 100 requests/minute." Source: docs/api-reference.md L44 (primary, current) [INFERRED] "This makes the API unsuitable for high-frequency trading." Basis: combines the rate limit with an unstated throughput assumption. No source states this. -> Confirm or qualify. [WEAK] "Competitor X has 2M users." Source: competitor's 2021 press release (outdated). -> Replace. [UNSUPPORTED] "Most teams migrate within 30 days." No supporting source found. -> Remove or supply a source. SUMMARY: 6 Grounded · 3 Inferred · 2 Weak · 1 Unsupported ## Why use this skill Standard prompting produces fluent text whose confidence is uncorrelated with its accuracy — the failure mode behind most "the agent made it up" incidents. This skill replaces tone-based trust with an explicit, auditable paper trail. It is built for anyone whose agent ships output other people act on: research and analyst agents, documentation pipelines, RAG systems, and content workflows where a single fabricated number is a real cost. The grading framework is ported from regulated scientific and technical publishing, where every claim must survive expert review with a traceable, graded source — a discipline that rarely exists in general-purpose AI tooling. ## Use cases - Gate RAG output before it reaches a user — flag anything not actually grounded in the retrieved documents. - Audit research briefs and analyst reports for unsupported or inferred claims. - Pre-flight marketing and product copy for statements that legal or comms would challenge. - Check technical documentation against the actual codebase or spec. - Run as a final quality gate in any multi-agent writing pipeline. ## Known limitations - The audit is only as good as the sources provided. With no source material, the agent flags internally unsupported or self-contradicting claims but cannot verify external facts. - It checks grounding and source strength, not domain correctness — a well-sourced claim from a flawed source passes as GROUNDED against that source. - It reads files within its declared scope and does not fetch the open web unless your agent separately provides that capability.