runtime observability

    1

    Convert black-box agent logs into structured execution traces, token cost breakdowns, and quality scores.

    Free

    Security scanned

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIDeepSeekDeepSeekMuseMuse+22 more

    runtime observability

    Example session with this skill installed

    Use runtime-observability to analyze this session log file and show the execution timeline, token breakdown, and any redundant tool calls.

    • Read your context and instructions
    • Compiled the runtime observability

    I have analyzed the provided session log. The execution spanned 12 events across two model providers.

    Runtime Observability Report

    Status: Completed with Inefficiencies
    Confidence: High (Direct log evidence)

    Execution Summary

    • Total Duration: 4.2s
    • Total Tokens: 8,450 (6,100 input / 2,350 output)
    • Estimated Cost: $0.12
    • Quality Score: 72/100

    Execution Timeline

    StepActorEventDetailDurationStatus
    1Modelmodel_callinitial_research1.1ssuccess
    2Tooltool_callsearch_web0.8ssuccess
    3Tooltool_callsearch_web0.7sredundant
    4Modelmodel_callsynthesis1.6ssuccess

    Diagnostic Findings

    [Severity: Medium] Redundant Tool Call: The agent called search_web twice with identical query parameters at Step 2 and Step 3. This resulted in an avoidable 0.7s delay and 1,200 redundant tokens.

    [Severity: Low] Token Spike: The final synthesis step used 40% more tokens than historical averages for this skill due to uncompressed search results in the prompt context.

    Quality Scoring

    • Output Compliance: Pass (JSON schema followed)
    • Factual Grounding: Pass (All claims mapped to search tool results)
    • Scope Discipline: Pass (Stayed within research boundary)

    Next steps

    1. Implement a client-side cache for search_web results to prevent identical repeat calls.
    2. Add a summarization step for tool outputs before injecting them into the final synthesis prompt.
    3. Review the branching logic in the research skill to ensure it checks previous tool results before re-invoking.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Developers often struggle to understand why an agentic workflow failed, ran up a high bill, or took too long to complete. This skill provides a post-run analysis layer that converts opaque execution logs into structured diagnostic reports. It eliminates the guesswork involved in debugging black-box agent sessions by providing a clear view of the internal decision-making process.

    What it does

    • Execution tracing reconstructs the exact sequence of tool and model invocations to visualize the agent's path.
    • Cost attribution breaks down token usage and associated costs by model, tool, or execution stage.
    • Anomaly detection identifies redundant tool calls, infinite loops, and sudden token spikes that indicate inefficiencies.
    • Quality scoring evaluates final outputs against a rubric for compliance, factual grounding, and scope discipline.
    • Latency analysis isolates bottlenecks by calculating durations for each span and identifying expensive wait states.

    How it works

    1. Acquire logs by resolving a session ID through a log provider or reading a provided JSON event stream.
    2. Normalize events into a standard timeline schema including timestamps, actor types, and tool statuses.
    3. Build the trace to map out branches, retries, and explicit decision points found in the telemetry.
    4. Generate diagnostics based on a weighted scoring rubric to identify waste and quality issues.
    5. Export reports in Markdown format, complete with execution summaries and improvement actions.

    Frameworks & tools

    The skill utilizes Python scripts for log normalization and report generation. It integrates with session log providers via CLI or environment variables and can resolve local JSONL artifacts from the Trae environment.

    Why this beats prompting it yourself

    Parsing raw event streams manually is error-prone and misses subtle patterns like token-heavy prompt repetitions or branch logic failures. This skill applies a consistent weighted rubric and normalization logic that ensures every diagnostic claim is backed by log evidence rather than model hallucination.

    Use cases

    • Debugging expensive agent runs to find specific steps causing token inflation.
    • Auditing agent decision-making to understand why a specific tool path was chosen.
    • Post-mortem analysis of failed sessions to identify missing output validation.
    • Generating compliance reports for automated workflows to verify factual grounding.

    Known limitations

    Does not support live telemetry ingestion or real-time APM monitoring. Analysis quality depends entirely on the completeness of the input event stream or log file provided.

    Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for LLM Ops.

    Requires post-run logs or session IDs. Cannot perform live tracing or APM setup. Quality of insights is strictly tied to the granularity of provided telemetry.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before buying a skill.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Free to use

    No account required to browse

    Trust & safety

    Security scan

    Security scan passed 14 days ago

    • Free with an Agensi account

    Permissions

    Terminal / Shell
    Read Files
    Write Files
    Environment Variables
    Listed14 days ago

    What's inside

    Frequently Asked Questions

    This skill converts opaque, black-box agent logs into structured execution traces, allowing you to pinpoint exactly where an agentic workflow failed, wasted tokens, or experienced latency bottlenecks.

    The skill is framework-agnostic and works by normalizing standard JSON event streams or session logs; if your agent can export execution data to a JSONL format, this skill can analyze it.

    No, this is a post-run analysis tool that processes session IDs or local log artifacts after a run is complete, ensuring it doesn't add any latency to your live agent workflows.

    You will receive the core Python-based normalization scripts, the weighted diagnostic rubric for quality scoring, and the report generation templates for Markdown exports.

    The skill generates comprehensive reports that include a visual execution timeline, a granular breakdown of token costs by tool, and a quality score based on factual grounding and scope discipline.

    New in LLM Ops

    Free