ai Code Review Strategist

Your AI agent writes code that looks correct but breaks in production. This skill catches the 5 failure modes humans miss.

The Problem

Your team ships 3x more PRs per week since adopting AI coding agents. The code is syntactically clean, well-structured, and passes the tests — but 43% of it needs debugging in production anyway. The agent wrote tests alongside the code, so they match the agent's wrong assumptions. The standard code review question "does this code look right?" is the wrong question for AI output. The right question is "how could this code be wrong despite looking right?" — and you need a systematic framework to answer it, not just intuition.

What You Get

• Blast radius analysis — maps every changed file's downstream consumers and cross-boundary impacts before reading a single line of code, flagging changes that touch shared modules or public APIs for deep review
• 5 AI failure mode heuristics — targeted detection for hallucinated API contracts (calling functions that don't exist), silent error swallowing (try/catch with no propagation), missing edge cases (happy-path-only logic), over-optimized style (clever patterns hiding wrong logic), and test contamination (tests that assert what the code does, not what it should do)
• Fitness function verification — checks every changed function against 5 production invariants: type safety, resource cleanup, idempotency, error propagation, and state consistency
• Structured review output — produces a standardized review with blast radius, categorized findings with severity levels (Critical/Warning/Suggestion), verification results, and an explicit APPROVED/CHANGES REQUESTED/REJECTED verdict
• Targeted fix instructions — instead of rewriting code, tells the original agent exactly what constraint was violated and what the correct behavior should be, preserving the reviewer-as-gate role
• Test contamination detection — identifies tautological tests (tests that pass because the code does what it does, not because it's correct) by checking if tests would pass against a known-broken variant

Why This Beats Prompting It Yourself

When you ask an AI to "review this PR," it typically performs a surface-level diff review — checking naming conventions, suggesting minor refactors, and missing the real problem: the AI-generated code may have hallucinated an API, swallowed an error, or assumed inputs that won't exist in production. This skill's 5 heuristics are specifically tuned to LLM failure modes that human-review-style prompts completely miss. It doesn't replace human judgment — it gives the reviewer the structured evidence to exercise judgment efficiently.

Example Session

You: Review this PR. Most of the code was generated by Claude Code for a payment retry service.

Agent: Blast Radius: 4 files changed, 12 downstream consumers, touches payment-critical path → Deep Review.

Findings:

• [HALLUCINATED_API] payment.py:12 — stripe.Charge.create(amount=amount) uses v3 kwargs; installed stripe is v2. Will crash at runtime. CRITICAL.
• [SILENT_ERROR] payment.py:47 — except Exception: logger.error("failed") catches all exceptions without propagating. Payment failures are invisible to callers. CRITICAL.
• [MISSING_EDGE_CASE] retry.py:23 — calculate_backoff(attempt) returns 0 for negative inputs. No bounds check on attempt parameter. Warning.
• [TEST_CONTAMINATION] test_retry.py:8 — Test asserts result.status == "retry" which is exactly what the code sets. Tautological. Warning.

Verdict: CHANGES REQUESTED — 2 critical findings must be resolved. Fix instructions attached.

Use Cases

• Reviewing AI-generated PRs before merging to a production branch
• Establishing a team-wide review framework for AI-assisted development
• Prioritizing review effort on high-blast-radius changes (auth, payments, data layer)
• Catching hallucinated dependencies before they crash in production
• Identifying test suites that provide false confidence due to tautological assertions

Known Limitations

The skill is optimized for AI-generated code (Claude Code, Cursor, Codex CLI output) and may flag legitimate patterns in human-written code that deliberately uses broad exception handling or liberal .get() usage. Use the blast radius analysis first to determine review depth — not every PR needs the full 5-heuristic scan.

Tags: code-review ai-generated-code pr-quality quality-gate production-safety

Version: 1.0.0