Flaky Test Detector

What it does

The Flaky Test Detector is a specialized diagnostic skill designed to stabilize your CI/CD pipeline by identifying and resolving non-deterministic test failures. It systematically hunts for the root causes of "intermittent" failures—tests that pass or fail without any code changes—across any language or framework including Python (pytest), JavaScript (Jest), Go, and Java (JUnit).

Why use this skill

Manually debugging flaky tests is a massive time sink. This skill automates the investigative heavy lifting: it runs stress tests with rerun loops, analyzes CI logs for environmental patterns, and scans your codebase for anti-patterns like hardcoded sleeps, shared mutable state, and unmocked external dependencies. It doesn't just find the flakiness; it provides standardized fixes and quarantine strategies to unblock your team immediately.

Supported Features

• Detection Loops: Automated execution of suspected tests (50+ runs) to calculate failure rates.
• Isolation Testing: Detects shared state by running tests in isolation vs. parallel execution.
• Pattern Scanning: Greps for problematic syntax like Math.random(), sleep(), or global fixtures.
• Automatic Quarantining: Generates standard @skip or @mark.flaky wrappers and tracking registries.
• CI Integration: Provides YAML configurations for GitHub Actions to detect and report flakes automatically.