Claim Check: Every Result Backed by Its Data

What it does

A results section is where reviewers hold your sentences against your numbers. This skill makes your agent back every sentence about a result with its data: each number, verdict word and citation checked against the results file it cites. Point it at a results paragraph, a caption, an abstract or a claims list, and it checks each claim against its evidence (the estimate, its interval, its n) and rewrites only what fails:

• Numbers match the data. Every number in a cited sentence must sit in the cited result within rounding. A mis-rounded estimate, a value copied from the wrong entry, a dropped minus sign or a fraction turned into a percent by hand is caught and fixed.
• No verdict without the evidence for it. "Improves", "outperforms" and "significantly better" need a stored difference whose interval excludes zero. "No effect", "matches" and "no extra cost" need an equivalence test, not a wide interval. The skill writes the estimate and interval instead, or "not distinguishable from zero at this sample size".
• One fact per citation. Two arms, two conditions or a raw value and its difference each get their own sentence and their own citation.
• Scope stays where it was measured. A single-seed result is reported as one run; "generalises" needs evidence at every setting; a mechanism no experiment isolated becomes "one explanation, not tested here".
• Reread before returning. An 8-step procedure, ending with a fact-by-fact reread of every claim sentence against its entry, then the checker, resolving every error it reports.

The checker

check_claims.py (Python 3.10+, standard library, no network) resolves every [results/<file>.json: <entry>] citation in your prose and claims.md and reports errors and warnings: number mismatches within rounding, hand unit conversions, verdict words over an interval that includes zero, equivalence words with no equivalence test, verdicts on n = 1 or with no interval, comparison words with no stored difference, citations that cover several results, and broken citations. In the demo it found 6 errors and 9 warnings in a draft results section and 0 in the corrected copy.

How it differs from citation auditors

Citation auditors check that a reference supports a sentence. Claim Check checks your own results: it opens the JSON your sentence cites and verifies the numbers and verdict words against the stored value, interval and n.

What's included

• SKILL.md: the citation syntax, the 8-step procedure, a quick-reference table and worked examples
• references/verdict-words.md: nine families of verdict words (comparison, significance, equivalence, magnitude, mechanism, scope, trend, confirmation, ranking) and what evidence each needs
• references/hard-cases.md: phrasing for null results, overlapping intervals, single runs, n = 1, missing intervals, many comparisons, percent versus percentage points
• scripts/check_claims.py: the deterministic checker

Part of the Aurora Lab research pipeline

Works entirely on its own: with a research/ folder it reads research/results/*.json and updates research/claims.md; without one it works from the numbers you paste. Pairs well with Aurora Lab's Scientific Voice (how sentences read) and Test Readers (how a cold reader takes the paper), but needs neither.