- Home
- Skills
- Testing & Debugging
- Flaky Test Finder: Catch the Test That Breaks Others
Works with the AI Agents you already use
Flaky Test Finder: Catch the Test That Breaks Others
Turn a flaky test into a named culprit: measures its failure rate and bisects test order to find the polluting test in a handful of runs.
$12
About this skill
"It fails in CI and passes on my machine." Your agent reruns it, it goes green, and the bug that leaked state between tests ships.
Measured: on more-itertools (766 tests) with a seeded state leak, it named the one polluting test out of 363 candidates in 15 runs (165 s), and the two-test reproduce command failed 3 out of 3 times.
Flaky Test Bisector gives your agent a tool that finds the cause instead of hiding it:
- rate: reruns one test N times on its own and reports the failure rate with a 95% confidence interval (Wilson), plus the failure messages grouped by count. 0/20 is reported honestly as "below 16%", with the number of runs a 1% flake would need (299).
- bisect: for a test that passes alone but fails in the suite, it bisects the tests that ran before it and names the polluter in about log2(n) runs. It also finds the reverse case, a state-setter the test silently depends on. Every probe is printed, the pair is re-run to confirm it, and you get a two-test reproduce command.
- If no single test explains the failure, it falls back to delta debugging and returns the minimal set.
- Cause hints: points at the culprit's line that leaks state (env var writes, patches never undone, module-level dicts, setters on shared modules, caches, time, ports, files).
- pytest, jest and vitest built in; go test, mocha and others through a command template.
--order-filereplays the exact order of a failing CI run.
| Suite | Tests before the victim | Runs | Time | Pair reproduces |
|---|---|---|---|---|
| more-itertools, seeded leak | 363 | 15 | 165 s | 3/3 |
| 300-test demo, seeded cache leak | 295 | 14 | 56-91 s | 3/3 |
Times are on a busy Windows laptop; runner start-up dominates them. The demo ships in the skill with one command to reproduce it.
What it does not do: fix the test for you (the agent does, with the proof in hand), bisect across files under jest or vitest (they isolate each file), or prove a cause hint (hints are leads; the reproduce command is the proof).
Works in Claude Code, Cursor, Codex CLI, Copilot and any SKILL.md agent.
Privacy: one Python file, standard library only. It runs your own tests locally, writes only to the system temp folder, and makes no network calls and no telemetry.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Testing & Debugging.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 27 days ago
- Software
- mocha
Node.jsjestvitest
Git