ai agent run auditor

The problem

LLM agents often hallucinate task completion, skip critical constraints, or invent data points that look plausible but lack foundation. Without a manual line-by-line review of execution logs, developers cannot trust if an agent actually performed the work or just generated a convincing summary.

What it does

• Extracts explicit and implicit completion claims from agent logs and final outputs.
• Maps setiap claim to specific evidence in the transcript, such as tool calls or log lines.
• Flags unsupported figures, skipped steps, and invented constraints not present in the original brief.
• Calculates efficiency scores based on redundant tool calls and token-heavy reasoning loops.
• Provides a prioritized recommendation for a full rerun or targeted correction.

Why this beats prompting it yourself

Basic prompts often suffer from the same "optimism bias" as the agent being audited. This skill enforces a strict verification protocol that treats any unevidenced claim as a failure, ensuring objective QA that catches subtle hallucinations a standard prompt would miss.

Use cases

• QA for autonomous coding agents to ensure all unit tests were actually executed.
• Audit of data extraction agents to verify that every number in a report has a source citation.
• Compliance checks for agents handling sensitive workflows where every action must be logged.
• Resource optimization to identify agents that are looping or using excessive tool calls.

Known limitations

Cannot audit the internal logic of hidden skill files or compiled tools. Quality of the audit is dependent on the level of detail provided in the execution transcript.