- Home
- Skills
- Trading & Investing
- Honest Strategy Validation for Trading Bots
More screenshots
Works with the AI Agents you already use
Honest Strategy Validation for Trading Bots
The expensive way to learn that is with real money after weeks of tuning.
Free
Honest Strategy Validation for Trading Bots
Example session with this skill installed
I want to build a scalping bot for BTCUSD on my broker (spread is around $25 at BTC ~95k). Idea: EMA9 crosses EMA21 on the 1-minute chart, hold 5 minutes. I attached two weeks of 1m candles. Can you check if this is worth building before I spend time on it?
- Read your context and instructions
- Compiled the honest strategy validation
- Generated the data export
Validation: EMA9/EMA21 cross, 5-minute hold, BTCUSD 1m, 2025-01-15 to 2025-01-28
Verdict
Don't build it as specified. The signal doesn't predict direction even before costs (gross -0.73 bps per trade, 95% CI -1.9 to +0.3). After the $25 spread you lose about 3.2 bps per trade, with every bootstrap resample negative. This is a cheap "no".
What I tested
- Data: your file, 20,160 one-minute bars (14 days, no gaps). BTC traded at about $96.5k–$109k, median ~$103.7k.
- Rule: EMA9 and EMA21 on 1m closes. When EMA9 crosses above EMA21, go long. When it crosses below, go short. Entry is at the
open of the next bar, so there is no look-ahead. Exit is at the close 5 minutes later. No stop and no target, since you didn't specify any.
- Cost: $25 round-trip spread, which is about 2.4 bps at these prices (2.6 bps at $95k). I added
no commission or slippage, so real results would be worse.
- The rule fired 966 times, about 69 trades per day.
Gates
| Gate | Result | Evidence |
|---|---|---|
| 0 Accounting | n/a | No live P&L yet. This is a pre-build check on bars. |
| 1 Cost vs move | 0.28–0.30× at 5 min (0.69× at 1 min) | Median absolute 5-minute move is 8.7 bps against 2.4–2.6 bps cost. The horizon is playable only with a strong signal: the spread eats about 30% of a typical move. At 1 min it eats 69%, and up to 0.95× around 06 UTC. |
| 2 Random baseline | random side, 5-min hold: gross -0.01 bps, net -2.44 bps, win rate 42.5% | A coin flip with these exits loses the spread every trade. That is the bar to beat. |
| 3 Gross before costs | -0.73 bps (day-bootstrap 95% CI -1.90 .. +0.26, PF 0.89, win rate 48.4%) | No directional information. Even the top of the CI (+0.26) is about 10× smaller than the 2.4 bps cost. Investigation stops here. |
| Net (for reference) | -3.16 bps/trade (CI -4.31 .. -2.16, p≈1.00, PF 0.59, win rate 40.8%) | Slightly worse than random (-2.44). |
| 4 Pre-registration | not done | Not needed: the idea failed the cheaper gates. |
| 5 Holdout | not opened | Same reason. |
In money terms: trading 1 BTC per signal would have averaged
-$32 per trade, about -$31k over the two weeks. At 0.01 BTC that is about -$0.32 per trade, or about -$22 per day at ~69 trades/day. These are backtest numbers on 14 days, not a forecast. They only show that the costs win.
Diagnostics (not used to decide)
- Gross by horizon, same signals: 1m -0.09, 3m -0.44, 5m -0.73, 10m -0.41, 15m -0.87, 30m +0.24, 60m -0.43 bps. Every horizon is inside noise. No hold time shows an edge, so changing the hold time won't rescue the idea.
- By side (5 min, gross): long -1.30 bps, short -0.16 bps. Both are within noise.
- Caveat: 14 days is one regime and a short sample (14 independent days for the bootstrap). That cuts both ways, but a gross upper bound of +0.26 bps against a 2.4 bps cost is a very wide gap to close.
What would have to change
For a 5-minute hold to work at this spread, the signal would need to average
more than +2.5 bps gross per trade, consistently. The best this one can plausibly do is about +0.3. Alternatively, the round-trip cost would need to drop well below ~0.3 bps (raw-spread/ECN pricing), and even then this signal shows nothing to harvest.
What not to do next
Don't invert it ("it loses, so trade the opposite"). The gross mean is indistinguishable from zero, so the inverse also has no edge and still pays the spread. An inverted rule would also be a new hypothesis that needs new data.
- Don't grid-search EMA lengths, hold times, or add RSI/MACD filters on these same two weeks. Those indicators come from the same OHLCV. You'd likely find a combination that looks good in-sample by chance, and it would be fitted noise.
Don't add a tight stop / wide target to "fix R:R". Exits redistribute the outcomes but cannot create direction (see Gate 2).
Cheapest honest next step
If you still want to build something on BTCUSD with this broker, move to horizons where the spread is small relative to the move. At 1 hour, cost/move is 0.09×; at 4 hours it is 0.05×; daily is about 0.01×. Then bring a hypothesis based on information that isn't just a 1m price crossover. Before testing it, write the rule down and lock it (pre-registration), split train/holdout by time, and run it through the same gates.
Files the agent wrote: ema_cross_gross.py (signal + random baseline), gate1_cost_vs_move.txt, gate2_3_ema_cross.txt, gate6_day_bootstrap.txt, trades.csv.
trades.csv
CSV · data export
Example file from a real run - the skill writes it into your workspace.
Connects securely to your tools. The creator never sees your data.
About this skill
Most trading strategies have no edge. The expensive way to learn that is with real money after weeks of tuning. This skill makes your agent find out first, with the cheapest test that can kill the idea, and only moves to slower tests when the idea survives.
It turns your agent into a skeptical quant colleague, not a cheerleader. It reports what the numbers say, including "this has no edge", and never presents a backtest as proof of future profit.
What it checks, cheapest first
| Gate | Question |
|---|---|
| 0 Accounting | Is the P&L real? Broker net vs bot log, including the MT5 trap where closing deals carry magic 0 |
| 1 Cost vs move | Can this holding period pay its round-trip cost at all? |
| 2 Random baseline | What do your exact stop, target and time exit produce with a coin-flip entry? |
| 3 Gross before costs | Does the signal know the direction, before any cost? |
| 4 Pre-registration | Rule, costs and pass bar written and SHA-256 locked before the deciding test |
| 5 Holdout | Train, then one look at unseen data, and only once |
| 6 Statistics | Bootstrap by calendar day, so same-day trades don't fake confidence |
Problems it catches
- "My bot wins 72% but loses money" (a wide stop and tight target win 72% on random entries)
- "5:1 reward/risk means I can be wrong 80% of the time" (with no entry edge you will be, and break even at best)
- Adding indicator votes built from the same candles
- Inverting a losing rule, picking the best horizon after the fact, re-opening a holdout
- Survivorship (today's symbol list applied to past years)
- Sizing with an LLM's stated probability
- A real effect smaller than your spread
What's inside
SKILL.md: the gated workflow and a fixed report formatreferences/pitfalls.md: 10 traps with numbers from a real retail project that tested strategy after strategy on demo accountsreferences/accounting_traps.md: MetaTrader 5 and exchange P&L trapsreferences/prereg_template.md: pre-registration template- 4 scripts (Python + numpy, any CSV of OHLC bars):
cost_vs_move.py,random_baseline.py,day_bootstrap.py,prereg_lock.py
Tested
Evaluated against the same agent without the skill on 8 realistic cases. With the skill it passed every check (100%). Without it, it missed measuring the rule before costs when asked to invert a losing strategy, and on another case it grid-searched 20 parameter sets on the test data. With the skill the agent also pre-registered and locked the rule, kept a holdout unopened, and traced a "55% win rate" to the market trend rather than the rule.
It will not hand you a profitable strategy. It tells you, quickly and with evidence, whether the one you have is worth your money.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Trading & Investing.
It does not generate strategies and does not promise profits; most strategies it checks fail, which is the point. Results are only as good as the data and cost assumptions you give it: spot data standing in for a CFD, or a spread that widens in fast markets, can still mislead. The scripts work on bar data, not tick or order-book data. It is analysis, not financial advice.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
12 uses
Times used to date
Free to use
No account required to browse
Trust & safety
Security scan
Security scan passed 19 days ago
- Free with an Agensi account
- Software
Python 3.9+numpy
Permissions
File Scopes
Runs local Python scripts on CSV price files you provide, and writes a .sha256 lock file next to pre-registration files. No network access, no broker connection, no credentials.