- Home
- Skills
- Testing & Debugging
- Agent Trace Auditor
Works with the AI Agents you already use
Agent Trace Auditor
Audit agent trace logs to diagnose loops, calculate token costs, and generate regression tests from failed runs.
$9
Agent Trace Auditor
Example session with this skill installed
Audit the latest Claude Code session in ~/.claude/projects/payment-service/ to find out why it looped and what it cost. Generate a regression test.
- Read your context and instructions
- Compiled the agent trace auditor
I have audited the session log session_20231027_1430.jsonl. The agent entered a retry loop after a permissions error on the sed command, consuming significant tokens without progressing.
Agent Trace Audit Report
| Metric | Value |
|---|---|
| Verdict | S1 - Same error repeated 5 times |
| Total Cost | $4.12 (Estimated) |
| Total Tokens | 245,600 |
| Input (Fresh) | 42,000 |
| Cache Reads | 198,000 |
| Output Tokens | 5,600 |
Findings
| Severity | Finding | Steps | Fix |
|---|---|---|---|
| High | Tool call loop (read_file) | 12, 14, 16, 18 | Add file existence check before read |
| High | Repeating Error (EACCES) | 22, 24, 26, 28, 30 | Update shell tool to handle permission prompts |
| Med | Context Bloat | 32-45 | Model failed to truncate large compiler output |
Regression Test Case
# Regression Case: EACCES Loop in payment-service
**Given** the agent is operating in a directory with read-only permissions on `config/`
**When** the agent attempts to modify `config/settings.json` using `sed`
**Then** the agent should recognize the `Permission denied` error after 1 attempt
**And** the agent should prompt the user for sudo access instead of retrying the command.
**Fixture:** session_20231027_1430.jsonl (Steps 22-30)
Next steps
- Review
agent-trace-report.mdfor the full timeline of the 5 repeated errors. - Apply the permission handling fix to the shell tool configuration.
- Add
regression-case.mdto your agent evaluation suite.
Connects securely to your tools. The creator never sees your data.
About this skill
Agent runs often fail in expensive, repetitive ways that are difficult to debug manually. This skill analyzes trace logs from Claude Code, Codex, or generic JSONL sessions to identify why an agent looped, stalled, or wasted tokens. It reconstructs the session timeline, calculates exact costs, and isolates specific failure modes with evidence-backed findings.
What it does
- Log Analysis parses Claude Code, Codex, and generic JSONL trace files to reconstruct agent sessions.
- Cost Auditing calculates prompt tokens, cache hits, and dollar costs based on session usage data.
- Pathology Detection identifies 14 specific failure modes including tool call loops, retry cycles, and context bloat.
- Timeline Reconstruction maps every turn and tool call to a costed sequence for step-by-step inspection.
- Regression Generation produces Markdown-based regression test cases from failed runs to prevent recurrence.
How it works
- Locate Trace identifies the session log file in standard directories like
~/.claude/projects/or~/.codex/sessions/. - Execute Audit runs the analysis script with optional flags for token budgets, step caps, or custom price tables.
- Review Report generates a structured document containing a verdict, cost breakdown, and prioritized fix list.
- Export Test writes a
regression-case.mdfile containing a Given/When/Then scenario based on the identified pathology.
Frameworks & tools
This skill requires Python 3.9 or newer. It is designed to work with Claude Code and Codex CLI session logs, as well as generic agent transcripts.
Why this beats prompting it yourself
Generic prompts often guess at agent behavior or hallucinate costs. This skill uses a specialized parser to extract hard data from logs, ensuring every finding cites specific step numbers and exact token counts.
Use cases
- Diagnose why a coding agent spent $40 in a single session without finishing a task.
- Identify the exact tool call that triggered an infinite retry loop in a repo.
- Convert a one-off agent failure into a permanent regression test for an eval suite.
- Compare two different agent runs to see which one utilized caching more effectively.
Known limitations
The auditor cannot see side effects that occurred outside the trace or determine agent "intent." Cost analysis requires usage fields to be present in the log file.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Testing & Debugging.
Cannot see side effects outside the log or agent intent. Requires usage fields in the trace for cost analysis. Pathology checks are limited to 14 pre-defined runaway behaviors.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 27 days ago
- Software
Python