Backtest-Validator
by Ifásola
Stop trusting overfit backtests. One honest verdict: PASS / FAIL / INCONCLUSIVE.
- Identify if a strategy's performance is statistically indistinguishable from luck.
- Calculate the maximum trade cost a strategy can handle before becoming unprofitable.
- Correct for cherry-picking bias when multiple strategy variants were tested.
$5
· or 25 creditsSecure checkout via Stripe
Included in download
- Identify if a strategy's performance is statistically indistinguishable from luck.
- Calculate the maximum trade cost a strategy can handle before becoming unprofitable.
- Ready for Cursor
Sample input
Analyze the trade logs in strategy_v4_backtest.csv. I tested about 12 different parameter sets before picking this one, and my broker charges 1.5 bps per round-trip.
Sample output
Backtest Verdict: FAIL
- Reason: Failed Multiple-Testing Correction (Sidak).
- Edge Probability: 62% (below 95% threshold).
- Break-even Cost: 0.8 bps.
- Analysis: Your real cost (1.5 bps) exceeds the break-even. The 'best' variant you found is likely a result of luck among 12 trials.
Backtest-Validator
by Ifásola
Stop trusting overfit backtests. One honest verdict: PASS / FAIL / INCONCLUSIVE.
$5
· or 25 creditsSecure checkout via Stripe
Included in download
- Identify if a strategy's performance is statistically indistinguishable from luck.
- Calculate the maximum trade cost a strategy can handle before becoming unprofitable.
- Ready for Cursor
- Instant install
Sample input
Analyze the trade logs in strategy_v4_backtest.csv. I tested about 12 different parameter sets before picking this one, and my broker charges 1.5 bps per round-trip.
Sample output
Backtest Verdict: FAIL
- Reason: Failed Multiple-Testing Correction (Sidak).
- Edge Probability: 62% (below 95% threshold).
- Break-even Cost: 0.8 bps.
- Analysis: Your real cost (1.5 bps) exceeds the break-even. The 'best' variant you found is likely a result of luck among 12 trials.
About This Skill
Validates whether a trading strategy has a real, tradeable edge by running the four checks most backtests skip: sample-size/power, realistic costs with break-even, luck (10k bootstrap), and multiple-testing (Sidak + PSR). Feed it a trades CSV; get a verdict plus the break-even cost, the level where the edge dies. For traders tired of strategies that look great and lose live.
Use Cases
- Identify if a strategy's performance is statistically indistinguishable from luck.
- Calculate the maximum trade cost a strategy can handle before becoming unprofitable.
- Correct for cherry-picking bias when multiple strategy variants were tested.
- Determine if a trade log has a sufficient sample size for reliable deployment.
Known Limitations
- Does not re-run strategy logic: requires trade logs.
- Assumes trades are independent.
- Historical analysis only: does not predict future regime shifts.
How to Install
mkdir -p ~/.claude/skills && curl -sL https://www.agensi.io/api/install/backtest-validator -o /tmp/backtest-validator.zip && unzip -o /tmp/backtest-validator.zip -d ~/.claude/skills && rm /tmp/backtest-validator.zipFree skills install directly. Paid skills require purchase - use the download button above after buying.
Reviews
No reviews yet - be the first to share your experience.
Only users who have downloaded or purchased this skill can leave a review.
Early access skill
Be the first to review this skill.
Only users who have downloaded or purchased this skill can leave a review.
Security Scanned
Passed automated security review
Permissions
No special permissions declared or detected
Requires Python 3.8+ with numpy and pandas. Works with any SKILL.md-compatible agent (Claude Code, Cursor, Gemini CLI).
Creator
Frequently Asked Questions
Learn More About AI Agent Skills
More Premium Skills
designing-hybrid-context-layers
Architects the right retrieval strategy for every query — teaching your agent when to use RAG, a knowledge graph, or a temporal index instead of defaulting to vector search for everything.
ai-automation-qa-pack
Professional QA & UAT documentation generator for AI automation agencies and complex agent deployments.
Bounty Security Pattern Master Library — 399 Vulnerability Patterns
A premium library of 399 vulnerability patterns and DeFi attack vectors for AI-driven bug hunting and security audits.
World-Class Site & App Design
Every AI-built UI looks generic and templated. This skill teaches your agent to actually match the design to the project, the industry, and the audience.