Test Coverage Auditor Pro

100% line coverage means nothing if your tests don't verify behavior. Get mutation testing, gap analysis, and CI threshold enforcement in one toolkit.

Free vs Pro

The free auditor maps untested functions and coverage gaps into a prioritized report. Pro adds the generation layer: framework-aware test scaffolding (pytest, Jest, Vitest, Go testing), edge-case matrices derived from function signatures, mutation-testing integration to verify your tests can actually fail, and CI trending that flags coverage regressions per module. Free shows the map — Pro builds the roads.

Upgrade Path

Free to see the gaps; Pro when you want them closed with generated tests rather than by hand.

The Problem

Your team celebrates hitting 90% code coverage — then a critical bug ships because every test was verifying the happy path. The coverage report said 90% because the code was exercised, not because the behavior was validated. You have no way to distinguish between tests that catch real bugs and tests that just run code. Meanwhile, untested error handling paths and edge cases silently rot, and nobody knows which functions have genuinely useful tests versus hollow assertions.

What You Get

• Coverage quality analysis — measures line, branch, and function coverage plus assertion density and test-to-code ratio, distinguishing exercised code from verified behavior
• Mutation testing engine — introduces small code mutations (mutants) and checks if your tests catch them; surviving mutants indicate tests that don't actually verify correctness
• Coverage gap analysis — identifies functions with zero coverage, untested conditional branches, error handling paths without tests, and edge cases missing from test suites
• Threshold enforcement scripts — coverage_auditor.py enforce --threshold 80 --fail-under 70 with exit codes that integrate directly into CI/CD pipelines to block PRs below minimum coverage
• Comprehensive report generation — produces HTML and Markdown reports with per-file coverage breakdowns, gap summaries, and mutation survival rates, suitable for stakeholder review
• CI integration templates — GitHub Actions workflow steps that run coverage analysis, enforce thresholds, and annotate PRs with coverage deltas on every push

Why This Beats Prompting It Yourself

"Write tests to improve coverage" produces tests that exercise code paths without validating behavior — exactly the problem you're trying to solve. The mutation testing component is the differentiator: it systematically breaks your code and asks "did your tests notice?" No prompt-only approach can simulate the 200+ code mutations that catch tautological, assertion-free, or happy-path-only tests. Coverage percentage is vanity; mutation score is sanity.

Example Session

You: Audit the test coverage of src/payments/ and tell me which tests are real.

Agent: Ran coverage analysis on src/payments/ (14 files, 2,847 LOC). Line coverage: 87% (2,476/2,847). Branch coverage: 61% (1,102/1,807). Assertion density: 0.8 per test (below recommended 2.0+). Mutation testing: 200 mutants introduced, 89 survived (55% mutation score). Top surviving mutants: payment.py:112 (error path not tested), retry.py:34 (backoff boundary not tested), webhook.py:67 (timeout default not validated). Gap analysis: 12 functions with zero coverage (all in webhook.py error handlers). 8 conditional branches in payment.py with only one side tested. Recommendation: Focus test-writing effort on webhook.py error paths and payment.py boundary conditions. Current 87% line coverage is misleading — real effective coverage is ~55%.

Use Cases

• Setting minimum coverage thresholds for CI that enforce real quality, not just exercise metrics
• Identifying which test files are genuinely validating behavior vs. just running code
• Planning a testing sprint by prioritizing the highest-impact coverage gaps
• Reporting coverage quality to stakeholders with mutation scores, not just percentages
• Validating that a coverage increase from a PR actually improves test quality

Known Limitations

Mutation testing is computationally expensive on large codebases (10,000+ LOC) and may take 10+ minutes per run. Use targeted module-level mutation testing rather than full-codebase scans during active development. The full scan is best reserved for pre-release validation.

Tags: testing coverage mutation-testing qa ci-cd code-quality

Version: 1.0.0