Walk-Forward Signal Validator

Most "my backtest shows 65% win rate" claims don't survive contact with a proper out-of-sample test — and the reason is almost always one of a few repeatable mistakes, not bad luck. This skill runs any rules-based trading or crypto signal through four checks before it's trusted with capital:

• Benchmark everything. A strategy's raw return means nothing without comparing it to just holding the asset — a "profitable" strategy that's really riding the market's own trend (beta capture) looks identical to a winning one until you compute the difference.
• Expanding-window walk-forward folds, not a single train/test split — one backtest window is one lucky simulation, not a validated result.
• Full-history testing, including a real drawdown. A short, favorable test window can completely invert the verdict — this skill documents an actual case where a 2-year window showed +2.85% "passing" alpha and the same signal, tested across full history including a crash, showed −1.16% and failed.
• A real classification bar (FAIL / WEAK / STRONG), not just "mean alpha > 0" — and explicit discipline around testing multiple signal variants against the same data, so a single borderline pass out of many trials doesn't get mistaken for a discovery.

Includes a dependency-free, tested reference implementation (scripts/walk_forward.py, Python standard library only) for the fold construction and classification math, plus a references/pitfalls.md walking through five real diagnosed failure modes with concrete before/after numbers. Built from lessons learned validating live trading-bot signals — not textbook theory.