Metric Denominator Audit

    1

    Check whether experiment rates use comparable denominators. Surface missing cases, recompute supported metrics, and specify which comparisons can be reported.

    $1

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIDeepSeekDeepSeekMuseMuse+22 more

    Metric Denominator Audit

    Example session with this skill installed

    Audit these evaluations: baseline 72 correct out of 100 assigned; candidate 75 correct out of 90 completed, with 10 timeouts. The table reports 72% versus 83.3%.

    • Read your context and instructions
    • Compiled the metric denominator audit

    Illustrative output

    On an assigned-case basis the baseline is 72/100 = 72% and candidate is 75/100 = 75%, a 3-point gain if timeouts count as failures. Completed-only candidate accuracy is 75/90 = 83.3%, with 90% coverage. With only aggregate counts, matched-case accuracy and significance are unknown. Replace the mixed-denominator table with a common denominator and a coverage column.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Extract the assigned, attempted, completed, scored and correct counts by configuration. Keep timeouts, skips, invalid records and missing labels separate. Reconcile totals from case IDs where possible. Note how the input defines the metric and whether higher or lower is better.

    Compare only the same unit and population. Show a common assigned-case rate when the task treats failures as zero, plus completed-case rate and coverage if available. Report absolute percentage-point change for rates and label relative change separately. Use a matched-case subset only when IDs and denominators can be checked. For non-rate metrics compare aggregation weights, missingness and units before calculating deltas.

    Return a metric ledger with numerator, denominator, exclusions and source for each row. Then list the comparisons supported by the evidence and any missing inputs. State when the result changes under a different treatment of failures. Avoid assigning missing labels or inferring per-case agreement from aggregates.

    Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Research & Analysis.

    Requires counts for timeouts/skips to calculate coverage. Cannot infer per-case agreement from aggregates. Analysis is limited to provided denominators and IDs.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before buying a skill.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed today

    Permissions

    No special permissions declared or detected

    Listedtoday

    What's inside

    Frequently Asked Questions

    This skill identifies 'denominator drift,' which occurs when one model appears more accurate simply because it failed to produce a response for harder cases. It re-aligns your data so you compare models based on the total assigned tasks, not just the successful completions.

    The skill is framework-agnostic and works with any platform that exports CSV, JSON, or log files, including LangSmith, Weights & Biases, or custom evaluation scripts. If your data contains raw counts of attempts and successes, this skill can audit it.

    You receive the core auditing logic, a pre-configured workflow for denominator reconciliation, and template reports for presenting your audited results to stakeholders. It is ready to use immediately upon integration with your data source.

    Yes, the skill generates a detailed 'metric ledger' for every row, allowing you to see exactly which records were excluded and why. This ensures your final audit report is fully transparent and auditable by other team members.

    The skill requires explicit counts or logs for timeouts, skips, or errors to function correctly. It cannot accurately recompute rates if your input data only provides a single 'success' percentage without the underlying raw numbers.

    New in Research & Analysis