More screenshots

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIDeepSeekDeepSeekMuseMuse+22 more

    LLMs in Production: Quality, Cost and Security

    2

    Measure quality, cut token costs, route models, observe and protect responses. 10 ordered skills, starting with Prompt Caching and Token Cost Optimization, LLM Response Streaming and UX, LLM Evaluation (Evals).

    Free

    Security scanned
    LLMs in Production: Quality, Cost and Security

    LLMs in Production: Quality, Cost and Security

    Example session with this skill installed

    Fictional company "Northwind Desk" runs a support copilot on a Claude model. Each request carries a long system prompt, 14 tool schemas and a pasted policy document, then the user turn. The team reports a high bill and a cache hit rate near zero.

    • Read your context and instructions
    • Compiled the llms in production

    Token Cost Review: support-copilot

    Provider/model: Anthropic mid-tier. Date: 2026-11-08. Sample size: 25 requests (fictional)

    Baseline

    | Segment | Tokens (avg) | Share | Static? |
    | System | 600 | 6% | yes (after fix) |
    | Tool schemas | 3,100 | 33% | yes |
    | Policy document | 5,200 | 55% | yes |
    | History + user | 500 | 5% | no |

    Findings

    1. Timestamp at prompt start invalidated the prefix on every call. Fix: move to end.
    2. Unsorted tool schema keys produced different bytes per call. Fix: deterministic serialization.
    3. Policy text resent uncached. Fix: breakpoint after policy block.

    Verification

    | Metric | Before | After | Source field |
    | Cached token share | 0% | 88% (second request onward) | cache_read_input_tokens / total input |
    | Eval score | 54/60 | 54/60 | eval run |

    Risks: sparse overnight traffic will miss the short lifetime; re-measure per hour of day.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    The problem

    Teams with an AI app that must be made reliable and cheap often need a repeatable process, not a one-off prompt. This pack targets: Measure quality, cut token costs, route models, observe and protect responses.

    What's inside (10 skills, suggested learning order)

    1. Prompt Caching and Token Cost Optimization — Reduce cost and latency of LLM calls.
    2. LLM Response Streaming and UX — Implement incremental responses and reactive interfaces.
    3. LLM Evaluation (Evals) — Measure output quality with test sets and metrics.
    4. LLM-as-a-Judge — Use models to automatically grade other outputs.
    5. Prompt Regression and Robustness Testing — Ensure prompt changes do not degrade results.
    6. Hallucination Mitigation — Reduce invented answers with grounding and verification.
    7. Model Routing — Dynamically pick the best model per task and cost.
    8. LLM Gateways — Centralize keys, quotas and observability for models.
    9. LLM Observability — Trace prompts, costs and latency in production.
    10. AI Guardrails — Apply input/output filters and policies to LLMs.

    Why a skill pack instead of a one-off prompt

    Each skill gives the agent a defined trigger, step-by-step procedure and output format, so results are consistent across sessions and easier to review.

    How to use

    1. Install the pack in your SKILL.md-compatible agent.
    2. Ask for the task in plain language (for example: "reduce cost and latency of llm calls").
    3. Review the output against the skill's quality checks before using it.

    Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for LLM Ops.

    Outputs are drafts that need human review; they are not professional, legal or financial advice. Tools, APIs and prices change; check current vendor documentation before relying on version-specific details. Quality depends on the context you give the agent (data, requirements, constraints). Instruction-only skills: they produce documents and plans, not running software.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before buying a skill.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Free to use

    No account required to browse

    Trust & safety

    Security scan

    Security scan passed 8 days ago

    • Free with an Agensi account

    Permissions

    No special permissions declared or detected

    Listed8 days ago

    What's inside

    Frequently Asked Questions

    10 skills, each with SKILL.md and references/sources.md: Prompt Caching and Token Cost Optimization; LLM Response Streaming and UX; LLM Evaluation (Evals); LLM-as-a-Judge; Prompt Regression and Robustness Testing; Hallucination Mitigation; Model Routing; LLM Gateways; LLM Observability; AI Guardrails.

    Teams with an AI app that must be made reliable and cheap.

    Agents that read the SKILL.md format. See the Compatibility note for what was actually tested.

    New in LLM Ops

    Free