- Home
- Skills
- Research & Analysis
- Performance Regression Detector: Prove It Got Slower
More screenshots
Works with the AI Agents you already use
Performance Regression Detector: Prove It Got Slower
Turn 'feels slower' into a verdict: runs old and new code alternately, then reports the slowdown with a confidence interval, or says the noise is too high.
$15
About this skill
Your agent says "now 30% faster" after one run on a busy laptop. One run proves nothing: on the machine we tested on, the same unchanged program varied by up to 50% between runs.
In our runs, Perf Regression Gate never raised a false alarm: 0 REGRESSION or IMPROVEMENT verdicts in 9 comparisons of unchanged code (A/A runs), and none for two seeded changes with no real effect, one of which looked 20% slower by raw medians.
What the tool does (one Python script, standard library only, any language's benchmark command):
- Checks out the base in a temporary git worktree and compares it with your working tree, uncommitted edits included. Nothing in your repo is modified; the worktree is removed afterwards.
- Interleaves the runs (AB, BA, AB...) after warm-ups, so background load and thermal drift hit both sides equally.
- Pairs each round and computes the ratio with a bootstrap 95% confidence interval, plus a Mann-Whitney U p-value and Cliff's delta effect size.
- Gives one verdict: REGRESSION, IMPROVEMENT, NO CHANGE (the whole interval inside your threshold) or NOISY, where it refuses to judge and tells you how many runs it would need.
- Measures what you choose: wall time, CPU time of the whole process tree (much steadier on a busy machine), or a number your benchmark prints.
- Prints a one-line receipt the agent must quote before claiming any speed-up. Exit code 1 fails CI; a GitHub Actions example is included.
Measured on a loaded 12-thread Windows laptop (other jobs running):
| Case | Result |
|---|---|
| Demo: stop words changed from a set to a list (tests still pass) | REGRESSION 1.237x, CI [1.158, 1.324], p=3e-06; at a busier moment, NOISY (no verdict) |
| mistune (public repo), escape() rewritten as one regex pass, 915 tests pass | n=20: NOISY, asked for 54 runs; n=54: REGRESSION 1.068x, CI [1.029, 1.147], p=0.017 |
| tabulate (public repo), a fast path removed: raw medians said +20% | not flagged; CPU-time ratio 0.962x [0.918, 1.062] |
| 9 A/A runs of unchanged code | 0 false alarms (1 NO CHANGE, 8 NOISY) |
Exact commands, commit SHAs and every run are in the skill's references/how-it-works.md.
What it does not do: it measures the command you give it, so a benchmark that misses the changed code says NO CHANGE. Changes under about 5% need a quiet machine or many runs; on a busy one it says NOISY rather than guessing. It does not profile or fix the slowdown.
Works in Claude Code, Cursor, Codex CLI, Copilot and any SKILL.md agent.
Privacy: runs locally, no network, no telemetry. It runs your own benchmark command, like your tests.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Research & Analysis.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 27 days ago
- Software
Python
Git