Pytest Bug Fix Proof

The problem

Developers often struggle to verify that a bug fix actually works without manual trial and error. Relying on "it looks right" or unchecked test runs leads to regressions and unverified patches in production.

What it does

• Reproduces specific pytest failures and records verifiable pre-fix execution data.
• Enforces a minimal-fix policy by tracking exactly which files changed and why.
• Verifies repairs by rerunning the original failing tests under identical environment constraints.
• Runs mandatory regression tests to ensure the fix did not break existing functionality.
• Generates a Pytest Evidence Pack JSON containing execution receipts, diff summaries, and root cause analysis.

Frameworks & tools

Python, pytest, and a specialized pytest_evidence.py collector script.

Why this beats prompting it yourself

Generic prompts often hallucinate test passes or skip the critical step of reproducing the failure first. This skill uses a deterministic collector to prove execution, preventing the agent from claiming a fix is successful without real process exit codes and node-ID verification.

Use cases

• Repairing a specific failing test case in a legacy Python codebase.
• Verifying a security patch doesn't break dependent logic via regression testing.
• Creating an audit trail for bug fixes in regulated or high-stakes environments.
• Validating PR contributions by requiring structured evidence of a passing test suite.

Known limitations

Not for greenfield projects or large refactors. Requires an existing, runnable pytest environment and explicit user trust of the project code and plugins.