- Home
- Skills
- Testing & Debugging
- Evidence-Backed Root Cause
Works with the AI Agents you already use
Evidence-Backed Root Cause
Makes your agent cite a real line for every claim in an incident report. A script confirms each quote exists in your files and limits the confidence it can report.
$9
Evidence-Backed Root Cause
Example session with this skill installed
Checkout started returning 500s this evening. The files are in ./incident. Find out why, and only tell me what you can back up.
- Read your context and instructions
- Compiled the evidence-backed root cause
- Generated the data export
This uses the sample incident in
examples/, which ships with the skill. The checker output is real output from runningverify_rca.pyon those files, so you can reproduce it. The staging test in the sample report is part of the sample data.
Summary
Likely cause: release v412 cut the payment service's database pool from 40 to 4. Status supported, confidence 0.9.
What supports it
| Handle | Source | Quote |
|---|---|---|
| E1 | deploys.log:3 | 2026-03-04T18:55:12Z payment-service v412 rollout complete |
| E2 | config.diff:6 | + pool_size: 4 |
| E3 | payments.log:3 | 2026-03-04T19:02:07Z ERROR pool timeout: waited 30000ms for connection |
| E4 | db-metrics.csv:4 | 2026-03-04T19:05:00Z,18,4 |
What else was considered
The database became slow (H2). Rejected: E4 shows query p95 at 18 ms during the incident.
Checker result
{
"status": "supported",
"hypothesis": "H1",
"stated_confidence": 0.9,
"confidence": 0.9,
"confidence_cap": 0.95,
"citation_validity": 1.0,
"independent_kinds": 4,
"reasons": [],
"dropped_citations": []
}
The same incident with a made-up story
examples/bad-report.json blames a database failover and states 0.95 confidence. The checker rejects it:
{
"status": "unsupported",
"stated_confidence": 0.95,
"confidence": 0.0,
"citation_validity": 0.0,
"reasons": [
"no valid evidence supports the conclusion",
"0 independent evidence kind(s), 3 are needed for more than 0.75",
"the cause was not verified by a test or a fix",
"cause_evidence must name a valid supporting item",
"no alternative hypothesis was considered"
],
"dropped_citations": [
"H1: E1 is invalid (quote not found in source)",
"H1: E9 does not exist"
]
}
Next steps
- Restore
pool_sizeto 40 or roll back v412. - Confirm that pool timeouts stop in
payments.log. - Add a review or alert for large pool size changes.
db-metrics.csv
CSV · data export
Example file from a real run - the skill writes it into your workspace.
Connects securely to your tools. The creator never sees your data.
About this skill
During an outage, a plausible wrong answer costs more than "I don't know yet". This skill makes your agent show its work: every claim cites a handle, every handle points to an exact line in a real file, and a script checks those lines before any confidence is reported.
What it does
- Checks every citation.
verify_rca.pylooks for each quote in the cited file. Quotes that aren't there, and handles that don't exist, are dropped and listed. - Limits confidence by fixed rules. It takes the lower of the agent's own number and a cap. The cap falls for unexplained contradictions, few independent evidence kinds, no verification, and no alternative considered. It never goes above 0.95.
- Checks that the cause came first. If the cited cause is dated after the symptom began, the report is
unsupported. - Requires a real alternative. A rival with equal support and nothing against it holds confidence at 0.5.
- Gives a clear result.
supported,tentative, orunsupported, with reasons and an exit code.
What's in the package
SKILL.md: the investigation routine your agent followsscripts/verify_rca.py: the checker, in Python standard library only, with no network calls, reading only inside the folder you namereferences/report-format.md: the report format and the exact rulesreferences/evidence-guide.md: where to look first, common traps, and causes that are easily confusedexamples/: a sample incident with one report that passes and one that fails
Good fit
- Outages, error spikes, latency regressions, and failed deployments
- Postmortems that need to hold up under questions
- Teams who want to check an agent's conclusions quickly
Known limitations
- It confirms a quote exists, not what it means. An agent can still misread a real line.
- It can't catch faked source files. Evidence must come from your real systems.
- Passing doesn't prove the cause. It means the reasoning meets fixed rules.
- The thresholds are judgement calls. They're adapted from a benchmarked system and aren't calibrated to your environment.
- You supply the data. It doesn't connect to your log or metric tools, so export what you need to a folder first.
Where it comes from
The approach comes from Sentinel, my open-source AIOps platform, where cited evidence and capped confidence are part of the investigation pipeline. The benchmark results published for Sentinel measure that system, not this skill.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Testing & Debugging.
Requires Python 3.9+ and exported incident files. Confirms that quotes exist, not what they mean, and can't detect faked source files. Confidence stops at 0.95. Thresholds are judgement calls, not calibrated to your environment.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 15 days ago
- Software
Python 3.9+
Permissions
Terminal runs scripts/verify_rca.py. Read covers the incident folder you name. Write covers the report file. The script makes no network calls and reads only inside the folder you pass to --root.