- Home
- Skills
- Testing & Debugging
- Stop Your AI Agent Saying "Done" Before It's Proven
More screenshots
Works with the AI Agents you already use
Stop Your AI Agent Saying "Done" Before It's Proven
AI agents often end with "all tests pass", "deployed" or "done" and show nothing to back it up.
$7.99
Stop Your AI Agent Saying "Done" Before It's Proven
Example session with this skill installed
Example request
Review the saved status report. Do not accept a completion claim unless the report or transcript shows a command and its result. Then show how the corrected report scores.
Commands
python scripts/claim_check.py sample/bad-report.md --transcript sample/bad-log.txt
python scripts/claim_check.py sample/fixed-report.md
- Read your context and instructions
- Compiled the stop your ai
Example result
The bad report contains four completion claims. The test claim is contradicted by the transcript. The other three claims show no successful result.
SUMMARY: 0 PROVEN | 3 UNPROVEN | 1 CONTRADICTED | 4 claims
The corrected report names a command and success result for each claim.
SUMMARY: 4 PROVEN | 0 UNPROVEN | 0 CONTRADICTED | 4 claims
Nothing outside the supplied report, transcript, and referenced local file was checked.
Connects securely to your tools. The creator never sees your data.
About this skill
AI agents often end with "all tests pass", "deployed" or "done" and show nothing to back it up. This skill checks the final report, line by line, and marks every completion claim PROVEN, UNPROVEN or CONTRADICTED before you accept it.
The proof
These are real outputs saved in the package, from the sample reports it ships with.
A bad report with 4 claims, checked against its test log (the log says "7 passed, 1 failed"):
| Line | Claim | Status | Evidence |
|---|---|---|---|
| 3 | All 8 tests pass. | CONTRADICTED | passed=7 |
| 5 | The deployment is verified. | UNPROVEN | missing success output |
| 7 | The export works. | UNPROVEN | missing success output |
| 9 | The ticket is done. | UNPROVEN | missing success output |
Result: 0 PROVEN, 3 UNPROVEN, 1 CONTRADICTED, 4 claims. Exit code 1.
The same report after the commands and results were added:
Result: 4 PROVEN, 0 UNPROVEN, 0 CONTRADICTED, 4 claims. Exit code 0.
The script has 13 pytest tests. All 13 pass.
What you get
SKILL.md: the workflow your agent follows. Works in Claude Code and OpenAI Codex.RULES.md: 12 short proof rules.scripts/claim_check.py: the checker. Standard-library Python (tested on 3.12), nothing to install.sample/: a bad report, a fixed report, a test log and the saved outputs above.tests/: 13 pytest tests.example-task.mdandexample-output.md: a worked example.
How to use
- Put your agent's final report in a Markdown file. Keep command output in one log file if it lives elsewhere.
- Run
python scripts/claim_check.py REPORT.md --transcript LOG.txt. - Read the table. Each claim gets a status and the evidence the script found.
- Fix every UNPROVEN or CONTRADICTED row. Add the command and its result, or re-run the check.
- Run it again. Exit code 0 means the script found no unsupported completion claim.
The script accepts these kinds of evidence: a command with exit code 0, a test pass count, a good HTTP status, and an EXISTS: path line for a file that exists. A failed test, a traceback or a non-zero exit marks the claim CONTRADICTED.
Known limitations
- It is pattern-based. It can miss unusual wording, and it can flag a word used in another sense.
- It shows whether evidence was given, not whether the claim is true. A command and output can be stale, wrong or unrelated.
- Read the cited evidence and re-run high-risk checks yourself.
- It looks for 8 kinds of claim: tests pass, no errors, fixed, verified, deployed, works, done, passing.
FAQ
Does it prove my agent's work is correct? No. It checks that each claim comes with a command and a visible result. You still need to read that result.
Which agents does it work with? It is a SKILL.md skill for Claude Code and OpenAI Codex. The script is plain Python, so you can also run it by hand on any report.
What do I need installed? Python 3 (tested on 3.12). The script uses only the standard library. The tests need pytest.
Does it send my reports anywhere? No. It reads the files you point it at and prints a table.
What if it flags something that is fine? It matches words, so this can happen. Rephrase the sentence or add the command and result next to it, then run it again.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Testing & Debugging.
Pattern-based detection may miss unique phrasing. Verifies presence of evidence, not the truthfulness or freshness of the provided logs. Requires manual review for high-risk claims.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 6 days ago
- Accounts
Example
- Software
Python
Permissions
Allowed Hosts