Works with the AI Agents you already use
ab test significance calculator
Deterministic A/B test calculator for statistical significance, z-scores, and conversion uplift with audit logging.
$5.99
ab test significance calculator
Example session with this skill installed
Calculate A/B test significance: Control has 12,450 visitors and 410 conversions. Variant has 12,800 visitors and 495 conversions. Use a 0.05 alpha level.
- Read your context and instructions
- Compiled the ab test significance
I have calculated the statistical significance and conversion uplift for your experiment. The results are deterministically verified.
A/B Test Results Analysis
| Metric | Value |
|---|---|
| Control Conversion Rate | 3.29% |
| Variant Conversion Rate | 3.87% |
| Relative Uplift | 17.51% |
| Z-Score | 2.58 |
| P-Value | 0.0098 |
| Statistically Significant | YES |
| Alpha Threshold | 0.05 |
| Audit Hash | 7a9e5b2c8f1d4e3a6c5b9d2f1e4a7b8c9d0e1f2a3b4c5d6e7f8a9b0c1d2e3f4g |
Next steps
1.
Implement Variant: Since the result is statistically significant with a high uplift, consider full deployment of the variant.
2.
Review Sample Size: Ensure the test duration was sufficient to avoid "peaking" bias before concluding the experiment.
3. Archive Results: Store the audit hash in your experiment log for future data integrity checks.
Connects securely to your tools. The creator never sees your data.
About this skill
Running A/B tests without statistical rigor leads to false positives and wasted resources. This skill provides a deterministic engine for calculating significance, z-scores, and conversion uplift. It replaces manual spreadsheets and unreliable LLM-based arithmetic with verified statistical formulas.
What it does
- Statistical significance calculates p-values and z-scores to determine if experiment results are valid.
- Conversion uplift computes the relative percentage increase between control and variant groups.
- Hypothesis testing supports two-tailed and one-tailed tests based on pre-specified thresholds.
- Data integrity generates a SHA-256 audit hash to ensure the calculation has not been tampered with.
- Traffic analysis evaluates pooled proportions and standard error across large sample sizes.
How it works
- Input the total visitors and conversions for both the control and variant groups.
- Specify optional parameters like alpha significance level, traffic split, or continuity correction.
- The skill executes the pooled proportion formula and normal CDF calculations.
- Review the verified output, including conversion rates and the deterministic significance status.
Frameworks & tools
This skill uses a JSON-RPC execution engine for mathematical precision. It is designed for integration with data science workflows, product experimentation platforms, and marketing automation stacks.
Why this beats prompting it yourself
LLMs frequently hallucinate during complex multi-step arithmetic, especially with small p-values and square root operations. This skill uses native deterministic code to ensure 100% mathematical accuracy every time.
Use cases
- Product experimentation to decide if a new feature rollout actually improved user sign-ups.
- Marketing optimization to validate email subject line performance or landing page changes.
- UI/UX testing to confirm if a button color change had a statistically valid impact on CTR.
Known limitations
Requires specific numeric inputs for visitors and conversions. It does not provide qualitative business advice or replace professional regulatory legal warranties.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for LLM Ops.
Requires non-null integer inputs for visitors and conversions. Does not perform external financial transactions or provide legal warranties.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 17 days ago