- Home
- Skills
- Trading & Investing
- leakage aware backtest auditor
Works with the AI Agents you already use
leakage aware backtest auditor
Audit backtests for data leakage, verify point-in-time feature availability, and calculate cost-adjusted returns.
$18
leakage aware backtest auditor
Example session with this skill installed
Audit this backtest JSON for the period 2020-2024. Check for leakage in the sentiment feature and apply a 5bps cost deduction per rebalance. Data: input_2024_strat.json
- Read your context and instructions
- Compiled the leakage aware backtest
Audit complete for strategy ID: STRAT-2024-001. I have processed the return series and verified feature availability timestamps against the provided data snapshot. Leakage was detected in the 'Sentiment_Score' feature during the 2021-Q2 window due to revision vintages appearing before publication dates.
Backtest Audit Report
| Metric | Value | Status | Notes |
|---|---|---|---|
| Audit Window | 2020-01-01 to 2024-01-01 | PASS | Chronological separation maintained. |
| Leakage Check | Sentiment_Score (2021-Q2) | FAIL | Point-in-time timestamp mismatch. |
| Gross Return | 42.15% | - | Cumulative across period. |
| Net Return | 36.82% | - | Adjusted for 5bps cost/rebalance. |
| Max Drawdown | -12.4% | - | Net compounded basis. |
| Survivorship Bias | Indeterminate | WARN | Missing delisting history in universe. |
Analysis
The strategy shows high sensitivity to rebalance frequency. The 5bps cost deduction reduced the Sharpe ratio from 1.8 to 1.4. The leakage in the sentiment feature (v2.1) suggests the backtest used revised data not available until 48 hours post-event.
Next steps
- Re-run the backtest using the
Sentiment_Score_Laggedfield to resolve the 2021-Q2 leakage. - Provide universe membership history to clear the survivorship bias warning.
- Invoke the helper with the
--output report.jsonflag to save the scored series for further walk-forward analysis.
Connects securely to your tools. The creator never sees your data.
About this skill
The problem
Standard backtests often suffer from look-ahead bias and unrealistic cost assumptions. Most tools fail to account for the actual publication time of data, leading to inflated performance that vanishes in live trading.
What it does
- Audits feature availability by mapping every data point to its earliest possible publication timestamp.
- Enforces strict chronological separation between training and test sets to prevent data leakage.
- Calculates cost-adjusted returns by applying fractional deductions to period-specific performance.
- Generates sensitivity tables across independent walk-forward windows and parameter variants.
- Identifies missing evidence of survivorship bias and corporate-action handling.
Frameworks & tools
Python 3 for execution, JSON for data contracts, and local CLI scripts for analysis.
Why this beats prompting it yourself
General LLMs struggle with the rigorous, multi-step verification required to prove train/test isolation. This skill follows a specific audit protocol that treats source references as metadata rather than proof, forcing an evidence-based evaluation of strategy robustness.
Use cases
- Auditing a new quantitative strategy before committing capital.
- Identifying look-ahead bias in third-party backtest reports.
- Standardizing performance reporting across multiple walk-forward windows.
- Stress-testing return series against varying slippage and commission tiers.
Known limitations
Does not execute raw strategy code or fetch market data. Requires pre-computed return series and data availability timestamps as input.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Trading & Investing.
Requires pre-formatted JSON input following a specific contract. Cannot independently verify data provenance or execute live trades. Limited to auditing supplied return series.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 1 month ago
- Software
Python