- Home
- Skills
- Trading & Investing
- out of sample edge reality checker
out of sample edge reality checker
Validates trading strategy persistence by comparing in-sample backtests against unseen out-of-sample data.
$5
Works with the AI Agents you already use
out of sample edge reality checker
Example session with this skill installed
Analyze this strategy. IS (2020-2022): Sharpe 2.1, MaxDD 12%. OOS (2023): Sharpe 0.8, MaxDD 25%. Benchmark OOS Sharpe is 0.5. Here is the trade list.
- Read your context and instructions
- Compiled the out of sample
Verdict: FRAGILE EDGE
The strategy shows a 62% degradation in Sharpe ratio and a doubling of Max Drawdown in the OOS period. While it outperforms the benchmark, the performance decay suggests significant overfitting to the 2020-2022 volatility regime. Recommendation: Do not scale capital.
Connects securely to your tools. The creator never sees your data.
About this skill
The problem
Backtested trading strategies often look profitable on paper but fail immediately in live markets because they were overfit to historical noise. Identifying whether a strategy has a genuine edge or just a statistical fluke requires rigorous validation against unseen data.
What it does
- Identifies the in-sample (fitting) and out-of-sample (validation) data periods to ensure true isolation.
- Calculates degradation metrics for Sharpe ratio, CAGR, and maximum drawdown across both periods.
- Performs statistical t-tests and distribution overlap checks to confirm performance stability.
- Flags overfitting signatures like outlier dependency and parameter sensitivity.
- Assigns a final verdict of Real Edge, Fragile Edge, or Overfit to guide capital allocation.
Why this beats prompting it yourself
General prompts often accept backtest results at face value or miss subtle signs of data leakage. This skill enforces strict degradation thresholds and statistical checks that act as a professional-grade filter, preventing you from deploying strategies that only work in the past.
Use cases
- Validating a quantitative strategy before moving from paper trading to live capital.
- Auditing a third-party strategy report to verify claimed performance persistence.
- Re-evaluating a modified strategy after adding new indicators or parameters.
- Assessing if a live strategy's recent performance dip is expected or a sign of edge decay.
Known limitations
Small sample sizes in the out-of-sample period can lead to an 'Insufficient Data' verdict. It cannot account for hidden data leakage if the developer already viewed the validation data during the design phase.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Trading & Investing.
Requires sufficient trade counts in both periods for statistical significance. Cannot detect "manual" overfitting where the developer repeatedly tweaked the strategy after seeing OOS results.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 3 months ago
Permissions
No special permissions declared or detected