Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIDeepSeekDeepSeekMuseMuse+22 more

    ab test significance calculator

    1

    Deterministic A/B test calculator for statistical significance, z-scores, and conversion uplift with audit logging.

    $5.99

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    ab test significance calculator

    ab test significance calculator

    Example session with this skill installed

    Calculate A/B test significance: Control has 12,450 visitors and 410 conversions. Variant has 12,800 visitors and 495 conversions. Use a 0.05 alpha level.

    • Read your context and instructions
    • Compiled the ab test significance

    I have calculated the statistical significance and conversion uplift for your experiment. The results are deterministically verified.

    A/B Test Results Analysis

    MetricValue
    Control Conversion Rate3.29%
    Variant Conversion Rate3.87%
    Relative Uplift17.51%
    Z-Score2.58
    P-Value0.0098
    Statistically SignificantYES
    Alpha Threshold0.05
    Audit Hash7a9e5b2c8f1d4e3a6c5b9d2f1e4a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4g

    Next steps
    1.

    Implement Variant: Since the result is statistically significant with a high uplift, consider full deployment of the variant.
    2.

    Review Sample Size: Ensure the test duration was sufficient to avoid "peaking" bias before concluding the experiment.
    3. Archive Results: Store the audit hash in your experiment log for future data integrity checks.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Running A/B tests without statistical rigor leads to false positives and wasted resources. This skill provides a deterministic engine for calculating significance, z-scores, and conversion uplift. It replaces manual spreadsheets and unreliable LLM-based arithmetic with verified statistical formulas.

    What it does

    • Statistical significance calculates p-values and z-scores to determine if experiment results are valid.
    • Conversion uplift computes the relative percentage increase between control and variant groups.
    • Hypothesis testing supports two-tailed and one-tailed tests based on pre-specified thresholds.
    • Data integrity generates a SHA-256 audit hash to ensure the calculation has not been tampered with.
    • Traffic analysis evaluates pooled proportions and standard error across large sample sizes.

    How it works

    1. Input the total visitors and conversions for both the control and variant groups.
    2. Specify optional parameters like alpha significance level, traffic split, or continuity correction.
    3. The skill executes the pooled proportion formula and normal CDF calculations.
    4. Review the verified output, including conversion rates and the deterministic significance status.

    Frameworks & tools

    This skill uses a JSON-RPC execution engine for mathematical precision. It is designed for integration with data science workflows, product experimentation platforms, and marketing automation stacks.

    Why this beats prompting it yourself

    LLMs frequently hallucinate during complex multi-step arithmetic, especially with small p-values and square root operations. This skill uses native deterministic code to ensure 100% mathematical accuracy every time.

    Use cases

    • Product experimentation to decide if a new feature rollout actually improved user sign-ups.
    • Marketing optimization to validate email subject line performance or landing page changes.
    • UI/UX testing to confirm if a button color change had a statistically valid impact on CTR.

    Known limitations

    Requires specific numeric inputs for visitors and conversions. It does not provide qualitative business advice or replace professional regulatory legal warranties.

    Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for LLM Ops.

    Requires non-null integer inputs for visitors and conversions. Does not perform external financial transactions or provide legal warranties.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before buying a skill.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 17 days ago

    Permissions

    Read Files
    Browser
    Listed17 days ago

    What's inside

    Frequently Asked Questions

    Unlike standard LLM prompts that often struggle with the complex square root and division operations required for p-values, this skill uses a deterministic mathematical engine to ensure 100% calculation accuracy without hallucinations.

    You will need the total number of visitors (sessions) and the total number of successful conversions for both your control group and your variant group.

    The skill is designed as a portable JSON-RPC tool, making it compatible with any AI agent or platform that supports external function calling and tool integration.

    Every calculation includes a SHA-256 audit hash, providing a verifiable digital fingerprint that ensures the results haven't been modified or tampered with since the computation.

    Yes, the skill allows you to specify optional parameters such as the alpha significance level (e.g., 0.05 or 0.01) and choose between one-tailed or two-tailed hypothesis tests.

    New in LLM Ops