lookahead bias proof tester

The problem

Backtests often produce "too good to be true" results because of hidden information leakage. Developers unknowingly use future data in feature scaling, signal generation, or universe selection, leading to catastrophic failure in live trading.

What it does

• Audits trading scripts and execution models for point-in-time data integrity.
• Identifies common leakage paths like centered moving averages, full-sample normalization, and shift(-k) errors.
• Detects survivorship bias in universe selection and restated financial data.
• Runs leakage probes including lag tests, label shuffles, and rolling retrain tests to stress-test strategy validity.
• Provides a Lookahead Bias Audit Report with code-level fixes and corrected performance metrics.

Frameworks & tools

Python, Pandas, NumPy, Scikit-Learn, and various backtesting engines.

Why this beats prompting it yourself

Generic LLMs often miss subtle alignment errors in time-series data or fail to account for serial correlation between train/test splits. This skill enforces a strict non-causal audit workflow that assumes leakage exists until proven otherwise, specifically looking for pandas index mismatches and lookahead execution artifacts that manual reviews overlook.

Use cases

• Verifying a newly developed quantitative strategy before deploying capital.
• Auditing third-party trading algorithms or academic research claims.
• Debugging unexpected discrepancies between backtest and paper trading results.
• Ensuring ML models for time-series use proper purging and embargo techniques.

Known limitations

Requires access to the strategy source code and data handling logic. Cannot detect leakage hidden within proprietary, pre-processed black-box datasets where the original collection methodology is unknown.