Test Flake Hunter

What it does

Test Flake Hunter is a diagnostic tool designed to identify non-deterministic test failures. It automatically detects your testing framework—supporting pytest, Jest, and Go test—and executes your suite or specific test files multiple times to uncover "flaky" behavior. By comparing pass/fail patterns across runs, it calculates flake ratios and provides structured reports on test reliability.

Why use this skill

Manually re-running tests to catch intermittent failures is tedious and prone to human error. This skill automates the repetition, normalizes output from different runners, and applies severity scoring to help you prioritize fixes. It goes beyond simple "pass/fail" by analyzing error messages and execution patterns to suggest likely root causes, such as network timeouts or race conditions.

Supported Tools

• Python: pytest, py.test
• JavaScript/TypeScript: Jest
• Go: go test
• Generic: Any runner with standard exit codes via Makefile or custom commands

Output format

The skill produces two primary artifacts: a human-readable flake-report.md summary for quick review, and a structured flake-report.json for integration into CI/CD pipelines or further data analysis. Reports include flake ratios, failure message snippets, and suggested remediation steps.