mcp-benchmark-hygiene — stop false 'model failed' grader res

The problem

LLM agents frequently hallucinate successful task completion, claiming a file was written or a server started when no such action occurred. Without a gate, these fabricated reports pollute audit logs and mislead human operators.

What it does

• Categorizes every agent claim into FACT, INFERENCE, SPECULATION, or UNVERIFIED tiers.
• Enforces a DO-NOT-REPORT gate that blocks fabricated completions lacking execution artifacts.
• Requires explicit exit codes, execution logs, or file artifacts to graduate a claim to FACT status.
• Provides standardized report templates for each tier to maintain audit trail consistency.

Frameworks & tools

This is a deterministic MCP server that works over stdio or HTTP. It uses no external LLM calls or network requests, ensuring low latency and privacy.

Why this beats prompting it yourself

Prompting an agent to be "honest" fails because the LLM remains the judge of its own success. This skill introduces a deterministic, code-based verification layer that the agent cannot bypass with clever phrasing, effectively decoupling the claim from the validation.

Use cases

• Verify deployment status by gating "success" reports behind actual CLI exit codes.
• Audit agentic research by separating primary source facts from model inferences.
• Prevent automated PR descriptions from claiming fixes that weren't actually tested.

Known limitations

This tool does not perform the execution itself; it requires the agent or orchestrator to provide the evidence artifacts for classification.