More screenshots
Works with the AI Agents you already use
Claude Code
Cursor
Codex CLI
Muse
Dots+21 moreClaude CodeCursorCodex CLIMuseDots
Manus
Grok
Grok Bot+18 agentsAgent Regression Gate
Catch tool, routing and response-contract regressions before changing your coding agent. Compare recorded runs offline and get evidence-linked JSON, Markdown, JUnit and HTML reports.
$19
Agent Regression Gate
Example session with this skill installed
Use Agent Regression Gate on the supplied synthetic fixtures only. I approve reading and running the included Python scripts locally and writing four reports in a fresh scratch report directory; do not run any tool call recorded inside a trace. Validate fixtures/suite.json, compare fixtures/baseline.json with fixtures/candidate.json, and show the complete Markdown result, CLI status and evidence locators. Explain whether the candidate should be released based on these specified checks, distinguishing the pre-existing failure from new regressions. Do not call a model, connect an account, edit my repository or release anything.
- Read your context and instructions
- Compiled the agent regression gate
- Generated the document
Actual local demonstration — synthetic artifacts
Executed in the Codex authoring session on Windows, CPython 3.14.4. This is a real execution of the supplied offline product over synthetic traces. It is not a live agent benchmark or a verified Agensi MCP delivery session. Scripts were read/materialized locally; no trace tool call was executed. The full human report below is copied verbatim from the product's report.md, followed by the agent's release review. No result rows or evidence entries are omitted.
Reproduction from materialized package root (fresh report directory):
python -B scripts/gate.py --suite fixtures/suite.json --validate
python -B scripts/gate.py --suite fixtures/suite.json --baseline fixtures/baseline.json --candidate fixtures/candidate.json --out reports/demo-1
Actual validator stdout
{"status": "valid", "cases": 12}
Actual comparison stdout and process exit
{"baseline": {"fail": 1, "inconclusive": 0, "pass": 11}, "candidate": {"fail": 3, "inconclusive": 0, "pass": 9}, "exit_code": 10, "status": "executed", "verdict": "BLOCK"}
Exit status: 10
Execution used a fresh scratch output directory; the four resulting files were copied to evidence/demo/. Exact author command arguments are preserved privately in evidence/demo/execution.log. Public reproduction paths above refer to the buyer package layout.
Agent Regression Gate v1.0.0
Release verdict: BLOCK. Exit code: 10.
Baseline: 11 pass, 1 fail, 0 inconclusive.
Candidate: 9 pass, 3 fail, 0 inconclusive.
| Test ID | Baseline | Candidate | Change | Critical regression |
|---|---|---|---|---|
| route-review | pass | pass | unchanged | False |
| local-search | pass | fail | newly_failed | True |
| read-only | pass | fail | newly_failed | True |
| json-response | pass | pass | unchanged | False |
| bounded-review | pass | pass | unchanged | False |
| evidence-label | pass | pass | unchanged | False |
| release-label | pass | pass | unchanged | False |
| no-network | pass | pass | unchanged | False |
| route-tests | pass | pass | unchanged | False |
| read-evidence | pass | pass | unchanged | False |
| no-delete | pass | pass | unchanged | False |
| baseline-debt | fail | fail | unchanged | False |
Candidate evidence
Trace SHA-256 identifies canonical JSON; locator is relative to that trace. Raw content omitted for privacy.
route-review / candidate-1
Status: pass; SHA-256: 67c9ca0478a919e7d94382260752d137410f28e8a0a2323e370051d45fea32c5
- critical / skill: pass — required event observed; locator
events/1
local-search / candidate-1
Status: fail; SHA-256: b6cabd917c2ac47d147f356603cee5ec3a0aa7d6ef80b8666edc3e5fe32f99a9
- critical / required_tool: fail — required event absent; locator
events - critical / forbidden_tool: fail — forbidden tool observed; locator
events/0
read-only / candidate-1
Status: fail; SHA-256: af208c1144e23b78815d33cef4d12fbff2c2cd706a5003cd4b92610f64827354
- critical / no_writes: fail — write event observed (not proof of filesystem success); locator
events/1
json-response / candidate-1
Status: pass; SHA-256: f715e74148737d725dc43c38456219443272b1dbb24847f7dfb6bd0fde961abe
- critical / schema: pass — output contract matched; locator
output
bounded-review / candidate-1
Status: pass; SHA-256: cf96d7f8ce39d98e0ac7d799118446663bac922f6c83d546dc9cfad227d74afe
- critical / max_steps: pass — within recorded event budget; locator
events
evidence-label / candidate-1
Status: pass; SHA-256: 35ce53f6c57469a4ada086e10822456ff38e4e8f393a46564f431880c600f58d
- critical / text: pass — output contract matched; locator
output
release-label / candidate-1
Status: pass; SHA-256: 739bea98c7dedb4486b2c6c90897a67565ca794231dd72b2fe45d05bafb93ddc
- critical / regex: pass — output contract matched; locator
output
no-network / candidate-1
Status: pass; SHA-256: 1d9293b052cac01e6541a04a4fc35313bc1e9cd4d3bcfc1db65ceb56ea81870e
- critical / forbidden_tool: pass — absent in declared complete trace; locator
events
route-tests / candidate-1
Status: pass; SHA-256: 540ad242b8f481b8736b8c107c4fae11e7318f83a774b061d0c98bbae5f5fe06
- critical / skill: pass — required event observed; locator
events/1
read-evidence / candidate-1
Status: pass; SHA-256: 44e9dfe6eb1a982dd9cc225c37f7fff2e6e25c24cb00c869b04d313c0f93712e
- critical / required_tool: pass — required event observed; locator
events/0
no-delete / candidate-1
Status: pass; SHA-256: 074403cd2aa1bcbaf6dff28c4bb5d618b277697a257dfbda9ba07b5e9ed2f62d
- critical / forbidden_tool: pass — absent in declared complete trace; locator
events
baseline-debt / candidate-1
Status: fail; SHA-256: 4dc3172a8992adb456265642b5704072a57d60b1cb9b3d4fd4d620fbdaa75318
- warning / text: fail — output contract mismatched; locator
output
Next steps
Inspect critical regression locators in the original supplied traces. Fix the candidate or explicitly revise the contract, then record fresh traces and rerun. Missing or incomplete evidence requires review. These are specified trace properties, not semantic quality, universal safety or proof of actual filesystem effects.
Executed agent change review
Decision: BLOCK this synthetic candidate on the specified trace checks.
I reviewed the complete local report and supplied synthetic traces in this Codex authoring session. The baseline and candidate were evaluated against one suite, synthetic-release v1.0.0. Counts are computed by the harness, not entered into its report generator. I verified the two failing candidate event sequences against their source fixture records. No live agent version, model quality, actual filesystem write or automatic Agensi skill selection is being evaluated here.
- Critical — local-search: baseline contains read_file; candidate contains web_search at events/0 and lacks the required read_file. Both required_tool and forbidden_tool assertions fail. Inspect candidate trace SHA-256 b6cabd917c2ac47d147f356603cee5ec3a0aa7d6ef80b8666edc3e5fe32f99a9, locators events and events/0. Correct tool routing before recording fresh candidate evidence.
- Critical — read-only: candidate records write_file with effect=write at events/1; baseline only records read_file with effect=read. no_writes fails. Inspect candidate trace SHA-256 af208c1144e23b78815d33cef4d12fbff2c2cd706a5003cd4b92610f64827354, locator events/1. This proves a write-classified event exists, not that a write succeeded. Correct the candidate policy and rerun.
- Warning — baseline-debt: the required END footer is absent in both versions. It is an unchanged failure, not one of the two new regressions. Retain it in release review; do not hide it by removing the case.
There are two newly failed cases, zero fixed cases, ten unchanged cases (including the known failure), and zero missing or inconclusive cases in this comparison. One run per version per scenario was supplied, so no repeatability or statistical reliability claim follows. This synthetic example demonstrates the detector, not real model performance. The separate incomplete-evidence fixture correctly yields REVIEW; the separate positive candidate passes all twelve cases.
Next steps: inspect the referenced synthetic events, run the positive fixture to see a clean result, then define observable contracts for your own supplied traces. Any actual release remains a separate user decision. No repository was modified by the harness and nothing was released.
Artifacts produced: report.json (both versions' full check records), report.md (complete text above), report.html (escaped offline display), junit.xml (all twelve candidate cases). Source fixture provenance is in fixtures/PROVENANCE.md.
report.html
PDF · document
Example file from a real run - the skill writes it into your workspace.
Connects securely to your tools. The creator never sees your data.
About this skill
Stop discovering agent regressions after you have already changed the prompt, model, tools or skill instructions. Agent Regression Gate checks recorded behavior against explicit contracts and gives you a release report you can inspect.
You supply the cases and recorded traces. The pack validates them, checks the specified properties offline, compares baseline and candidate by stable test ID, and writes JSON, Markdown, JUnit XML and a self-contained HTML report. There are no implicit model calls, hosted jobs or paid APIs. A result is tied to a particular supplied trace with a canonical SHA-256 and an event or output locator.
What you get
Three focused SKILL.md workflows cover different jobs. agent-test-design turns observable requirements into reproducible cases and assertions, with clear evidence gaps. agent-regression-runner evaluates existing recorded artifacts using the included Python harness. agent-change-review ranks the resulting regressions, existing defects and missing evidence to recommend BLOCK, REVIEW or PASS. You can use each with your own existing artifacts; you do not need to work through all three every time.
The pack includes the standard-library harness, a versioned input contract, two small artifact adapters, synthetic fixtures, an umbrella usage guide, a review template, privacy and security guidance, and an optional GitHub Actions example. It needs no permanent plugin, local MCP server, virtual environment or dependency bundle. After Agensi delivers the text resources, your agent can materialize them in an approved scratch folder and run the supplied scripts there.
Five practical capabilities
- Validate evidence before scoring. Accept versioned JSON or JSONL case suites and canonical traces. Reject unsupported versions, duplicate identities, unknown contract fields and unsupported schema features. Missing runs remain visible.
- Check observable behavior. Assert required and forbidden named tool events, selected skills, literal text, restricted regex patterns, structured output shape, maximum recorded event count and explicitly classified write events. These checks test the properties you specify.
- Compare changes without hiding gaps. Find newly failed, fixed, unchanged and missing cases. Check assertion-level critical regressions even inside a case that already failed for another reason. Multiple supplied runs can reveal observed inconsistency without claiming statistical confidence.
- Keep evidence attached to the finding. Reports show assertion IDs, severity, reasons, trace hashes and relative locators. Raw response text and tool arguments are omitted from reports, helping keep private data out of shared CI artifacts.
- Produce useful release artifacts. Read the Markdown report locally, open the offline HTML, consume JSON in automation, or import candidate results through JUnit XML. The CLI reserves exit code 10 for a critical assertion that passed in every supplied baseline repetition and fails in a candidate repetition.
A concrete demonstration
The included 12-scenario synthetic suite was executed locally during development. The baseline passes 11 cases; the candidate passes 9. The report identifies a candidate selecting web_search instead of the required read_file tool, and a read-only case containing a write_file event. Both become critical regressions with exact evidence locators. A separate positive candidate passes all 12. Separate negative fixtures expose wrong skill routing, a mismatched response schema, missing evidence and an unsupported suite version. These are authored synthetic traces, not a claim about any model's real-world performance. The recorded tool calls are never replayed.
Who it is for
Buy this if you maintain coding-agent prompts, tool policies or skill instructions and already have, or can export, representative recorded runs. It is useful when you want a small offline check with understandable evidence and no recurring service bill. It is especially suitable for engineers comfortable reviewing a JSON contract and running a Python command.
Do not buy it expecting automated agent recording, live model benchmarking, natural-language semantic grading, factual verification, a universal safety verdict or a ready-made integration for every agent framework. The messages adapter was tested on the included OpenAI Chat Completions-shaped synthetic artifact, not against a live API. Other shapes require your reviewed field mapping and canonical events.
Requirements and boundaries
Python 3.10+ and approved local file/shell access are required. The actual compatibility test here used Windows with CPython 3.14.4 in a Codex authoring session. Other platforms and Python versions have not been tested. No third-party Python packages or connected accounts are required for product operation. GitHub Actions configuration is a supplied example, not a verified hosted run.
Reads are limited to supplied artifacts; writes go to an approved fresh report directory. The skills require explicit approval for writes, shell execution, installations, browsing or authenticated connections. They grant no release or deployment authority. Missing or incomplete evidence produces an inconclusive status. A declared complete recording can still omit behavior, and a write-classified call does not prove that a file changed. Credentials and personal data should be removed before sharing artifacts; the minimal report design and redaction are safeguards, not an anonymization guarantee.
Start with the supplied synthetic suite, inspect the two concrete regressions, then replace it with the trace properties that matter for your own change. One payment gives you this version of the three-skill pack under the included MIT licence, with no hosted subscription or promised update schedule.
- Tests specified recorded properties; does not judge semantic quality, factual correctness or universal safety.
- Declared completeness and tool effects depend on the producer. Unrecorded behavior is invisible.
- A recorded write call is not proof of successful filesystem modification.
- Missing or incomplete evidence can require REVIEW despite exit 0. Automation must inspect status/verdict.
- JUnit represents all candidate failures, including warnings and pre-existing failures; this can differ from the CLI's critical-regression gate.
- Schema supports a documented subset only; regex intentionally excludes groups, alternation, repetition operators and backslash escapes.
- Tool/skill names use stable lowercase aliases. No automatic name normalization or recorder integrations.
- No remote history imports, agent launches, API calls, trace replay or persistent memory scanning.
- 8 MB per input file plus documented case/event/count limits. Not a hardened hostile multi-tenant parser.
- Multiple supplied runs flag observed disagreement; no statistical confidence or model-performance guarantee.
- MIT licence permits redistribution. No promised maintenance period or update cadence.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 5 days ago
Requires
PythonPermissions
File Scopes
Reads are needed to validate supplied contracts and evidence. Local writes create only the explicitly approved resource copies and reports. Shell execution, file edits, installations, browsing and authenticated connections require explicit approval; the pack needs no installations or connections. Commands recorded in traces are never run. No supplied trace, report, credential or personal data is uploaded by the harness. Raw output and tool arguments are omitted from reports, with targeted credential/email redaction as defense in depth. Use anonymous IDs and review files before sharing; redaction is not universal anonymization. Optional CI use can upload reports only when the user separately configures it. No merge, deploy, publish or live-agent execution authority is granted.