runtime observability
Convert black-box agent logs into structured execution traces, token cost breakdowns, and quality scores.
Free
Works with the AI Agents you already use
runtime observability
Example session with this skill installed
Use runtime-observability to analyze this session log file and show the execution timeline, token breakdown, and any redundant tool calls.
- Read your context and instructions
- Compiled the runtime observability
I have analyzed the provided session log. The execution spanned 12 events across two model providers.
Runtime Observability Report
Status: Completed with Inefficiencies
Confidence: High (Direct log evidence)
Execution Summary
- Total Duration: 4.2s
- Total Tokens: 8,450 (6,100 input / 2,350 output)
- Estimated Cost: $0.12
- Quality Score: 72/100
Execution Timeline
| Step | Actor | Event | Detail | Duration | Status |
|---|---|---|---|---|---|
| 1 | Model | model_call | initial_research | 1.1s | success |
| 2 | Tool | tool_call | search_web | 0.8s | success |
| 3 | Tool | tool_call | search_web | 0.7s | redundant |
| 4 | Model | model_call | synthesis | 1.6s | success |
Diagnostic Findings
[Severity: Medium] Redundant Tool Call: The agent called search_web twice with identical query parameters at Step 2 and Step 3. This resulted in an avoidable 0.7s delay and 1,200 redundant tokens.
[Severity: Low] Token Spike: The final synthesis step used 40% more tokens than historical averages for this skill due to uncompressed search results in the prompt context.
Quality Scoring
- Output Compliance: Pass (JSON schema followed)
- Factual Grounding: Pass (All claims mapped to search tool results)
- Scope Discipline: Pass (Stayed within research boundary)
Next steps
- Implement a client-side cache for
search_webresults to prevent identical repeat calls. - Add a summarization step for tool outputs before injecting them into the final synthesis prompt.
- Review the branching logic in the research skill to ensure it checks previous tool results before re-invoking.
Connects securely to your tools. The creator never sees your data.
About this skill
Developers often struggle to understand why an agentic workflow failed, ran up a high bill, or took too long to complete. This skill provides a post-run analysis layer that converts opaque execution logs into structured diagnostic reports. It eliminates the guesswork involved in debugging black-box agent sessions by providing a clear view of the internal decision-making process.
What it does
- Execution tracing reconstructs the exact sequence of tool and model invocations to visualize the agent's path.
- Cost attribution breaks down token usage and associated costs by model, tool, or execution stage.
- Anomaly detection identifies redundant tool calls, infinite loops, and sudden token spikes that indicate inefficiencies.
- Quality scoring evaluates final outputs against a rubric for compliance, factual grounding, and scope discipline.
- Latency analysis isolates bottlenecks by calculating durations for each span and identifying expensive wait states.
How it works
- Acquire logs by resolving a session ID through a log provider or reading a provided JSON event stream.
- Normalize events into a standard timeline schema including timestamps, actor types, and tool statuses.
- Build the trace to map out branches, retries, and explicit decision points found in the telemetry.
- Generate diagnostics based on a weighted scoring rubric to identify waste and quality issues.
- Export reports in Markdown format, complete with execution summaries and improvement actions.
Frameworks & tools
The skill utilizes Python scripts for log normalization and report generation. It integrates with session log providers via CLI or environment variables and can resolve local JSONL artifacts from the Trae environment.
Why this beats prompting it yourself
Parsing raw event streams manually is error-prone and misses subtle patterns like token-heavy prompt repetitions or branch logic failures. This skill applies a consistent weighted rubric and normalization logic that ensures every diagnostic claim is backed by log evidence rather than model hallucination.
Use cases
- Debugging expensive agent runs to find specific steps causing token inflation.
- Auditing agent decision-making to understand why a specific tool path was chosen.
- Post-mortem analysis of failed sessions to identify missing output validation.
- Generating compliance reports for automated workflows to verify factual grounding.
Known limitations
Does not support live telemetry ingestion or real-time APM monitoring. Analysis quality depends entirely on the completeness of the input event stream or log file provided.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for LLM Ops.
Requires post-run logs or session IDs. Cannot perform live tracing or APM setup. Quality of insights is strictly tied to the granularity of provided telemetry.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Free to use
No account required to browse
Trust & safety
Security scan
Security scan passed 14 days ago
- Free with an Agensi account