- Home
- Skills
- Code Quality & Review
- Guard Integrity: Tests That Can Actually Fail
More screenshots
Works with the AI Agents you already use
Guard Integrity: Tests That Can Actually Fail
Makes your AI agent prove every new test, lint rule and CI gate can fail: reintroduce a plausible form of the bug, watch it go red, record the probe in the guard, then commit with a report of real exit codes.
$10
Guard Integrity: Tests That Can Actually Fail
Example session with this skill installed
Last week someone called the Stripe SDK straight from src/orders/refund.ts and skipped our payment gateway, so the refund went out without an idempotency key and a retry refunded the customer twice. The code is fixed now. Add a regression test so nobody can reach the raw Stripe client outside src/payments/gateway/ again, then commit.
- Read your context and instructions
- Compiled the guard integrity
Summary
Guard added: only src/payments/gateway/ may reach Stripe. 6 plausible-form probes: 5 red, 1 green (recorded blind spot). Types can't fence an npm import (next steps). Committed 4f2c9e1.
The guard
// src/payments/raw-client-boundary.test.ts
/**
* Guard: only src/payments/gateway/ reaches Stripe (SDK or HTTP API).
* Protects: refunds/charges skipping the gateway's idempotency key.
* Sees: value import/re-export/import()/require of stripe(/*);
* "api.stripe.com" in strings (AST).
* Allowed: type-only imports; `// raw-stripe-ok: <reason>` on the statement.
* Blind to: computed module names; `import x = require()`; vendor wrappers.
*
* @probe break: refund.ts:2 `import Stripe from "stripe"`
* observed: src/orders/refund.ts:2 loads stripe
* @probe break: refund.ts:2 aliased default import
* observed: src/orders/refund.ts:2 loads stripe
* @probe break: refund.ts:6 `await import("stripe")`
* observed: src/orders/refund.ts:6 loads stripe
* @probe break: new shared/vendors.ts re-export
* observed: src/shared/vendors.ts:1 loads stripe
* @probe break: refund.ts:6 fetch to api.stripe.com
* observed: src/orders/refund.ts:6 raw HTTP to api.stripe.com
* @probe break: refund.ts:6 import(["str", "ipe"].join(""))
* observed: GREEN, see "Blind to"
*/
import { readdirSync, readFileSync } from "node:fs";
import { join, relative } from "node:path";
import { fileURLToPath } from "node:url";
import ts from "typescript";
import { describe, expect, it } from "vitest";
const SRC = fileURLToPath(new URL("..", import.meta.url));
const MARKER = /\/\/[ \t]*raw-stripe-ok:[ \t]*\S/;
const isStripe = (s: string) => s === "stripe" || s.startsWith("stripe/");
function loaded(n: ts.Node): ts.Node | undefined {
if (ts.isImportDeclaration(n)) {
const c = n.importClause, b = c?.namedBindings;
const typeOnly = !!c && (c.isTypeOnly || (!c.name && !!b && ts.isNamedImports(b) &&
b.elements.length > 0 && b.elements.every((e) => e.isTypeOnly)));
return typeOnly ? undefined : n.moduleSpecifier;
}
if (ts.isExportDeclaration(n)) return n.isTypeOnly ? undefined : n.moduleSpecifier;
if (ts.isCallExpression(n) && (n.expression.kind === ts.SyntaxKind.ImportKeyword ||
(ts.isIdentifier(n.expression) && n.expression.text === "require"))) return n.arguments[0];
return undefined;
}
function rawStripeAccess(name: string, src: string): string[] {
const sf = ts.createSourceFile(name, src, ts.ScriptTarget.Latest, true);
const out: string[] = [];
const visit = (n: ts.Node): void => {
const mod = loaded(n);
const kind = mod && ts.isStringLiteralLike(mod) && isStripe(mod.text) ? `loads ${mod.text}`
: (ts.isStringLiteralLike(n) || ts.isTemplateLiteralToken(n)) && n.text.includes("api.stripe.com")
? "raw HTTP to api.stripe.com" : null;
if (kind) {
let stmt = n; // per-site marker
while (!ts.isSourceFile(stmt.parent) && !ts.isBlock(stmt.parent)) stmt = stmt.parent;
if (!MARKER.test(src.slice(stmt.getFullStart(), stmt.getStart(sf))))
out.push(`${sf.getLineAndCharacterOfPosition(n.getStart(sf)).line + 1} ${kind}`);
}
ts.forEachChild(n, visit);
};
visit(sf);
return out;
}
const walk = (dir: string): string[] => readdirSync(dir, { withFileTypes: true }).flatMap((e) =>
e.isDirectory() ? walk(join(dir, e.name)) : /\.[cm]?[jt]sx?$/.test(e.name) ? [join(dir, e.name)] : []);
describe("raw Stripe boundary", () => {
it("only the gateway reaches Stripe", () => {
const found = walk(SRC) // derived from the tree
.map((f) => relative(SRC, f).replaceAll("\\", "/"))
.filter((rel) => !rel.startsWith("payments/gateway/"))
.flatMap((rel) => rawStripeAccess(rel, readFileSync(join(SRC, rel), "utf8")).map((h) => `src/${rel}:${h}`));
expect(found, "Raw Stripe access outside the gateway").toEqual([]);
});
// fixed fixtures
it.each([`import S from "stripe";`, `import { default as P } from "stripe";`, `export * from "stripe";`,
`await import("stripe");`, `require("stripe/lib/x");`, `import S from "stripe"; // TODO`,
"fetch(`https://api.stripe.com/v1/${p}`);", `// raw-stripe-ok:\nimport S from "stripe";`,
])("detects %j", (s) => expect(rawStripeAccess("f.ts", s)).toHaveLength(1));
it.each([`import type S from "stripe";`, `import { type S } from "stripe";`, `// import S from "stripe";`,
`// raw-stripe-ok: read-only report\nimport S from "stripe";`,
])("ignores %j", (s) => expect(rawStripeAccess("f.ts", s)).toEqual([]));
});
Probe log
First run: RED on src/admin/reconcile-report.ts:2 loads stripe (read-only listing). Marker with a reason on that statement, not the file. Green.
- P1–P6: one site, one axis each, on a stashed copy. P1:
FAIL src/payments/raw-client-boundary.test.ts > raw Stripe boundary > only the gateway reaches Stripe
AssertionError: Raw Stripe access outside the gateway: expected [ …(1) ] to deeply equal []
+ "src/orders/refund.ts:2 loads stripe",
Tests 1 failed | 12 passed (13) exit 1
- Restored: 13/13 pass; diff holds only intended files.
Commit report
Changes
- new guard · marker in
reconcile-report.ts· row indocs/INVARIANTS.md
Guards & probes
- P1–P5 RED (lines in
@probe). P6 GREEN, recorded blind spot.
Checks
| Command | Exit |
|---|---|
npm run typecheck | 0 |
npm run lint | 0 |
npx vitest run src/payments/raw-client-boundary.test.ts | 0 (13 passed) |
npm test | 0 |
Context / docs updates (applied)
docs/INVARIANTS.mdrow "Stripe only via the gateway"; verified bygrep -n(line 41).
Problems found
reconcile-report.tsreads Stripe directly. Move it behind the gateway?
Next steps
- Make the gateway the only workspace package depending on
stripe; a strict package manager blocks the import elsewhere. Then demote this test. - Optional: mirror as
no-restricted-importsaterror.
Connects securely to your tools. The creator never sees your data.
About this skill
The problem
An agent fixes a bug, adds a regression test, the suite goes green and the commit lands. Nobody asks whether that test could ever fail, and often it can't. It checks the shape of the code instead of what the code does. It compares against a hand-kept list that quietly goes stale. Or it is a lint rule at warn, so the lint command exits 0 however often it fires. In review, a guard that cannot go red looks exactly like one that works.
This skill was distilled from a long-lived codebase where that happened four times in a row. A timeout test stayed 4/4 green with a 59-second timeout, because it compared a config value to a bound instead of checking what a user sees when the server hangs. A lint rule added after three regressions could not stop the fourth: it was a warning. Six existing guards were later re-probed with a realistic version of the bug instead of the version their authors had tried, and all six turned out weaker than claimed.
What it does
It makes "a guard without a probe does not exist" a working rule for your agent. For every new test, lint rule, CI gate, inventory check, or type meant to forbid a pattern, the agent:
- first asks whether the violation can be made uncompilable, so it never needs detecting;
- writes the guard to check behaviour, derives what it covers from the code tree instead of a list, and allows exemptions only per statement, with a reason;
- reintroduces a plausible form of the defect, one axis at a time (the form the next developer would actually write, not the one just fixed), watches the guard go red, reverts, and confirms green;
- records each probe and the exact failure text in a
@probeblock inside the guard, including the forms it is known to miss; - ends with a commit report listing changes, guards and probes, real commands with exit codes, documentation updates already on disk and checked by grep, and any problems found.
What's inside
SKILL.md: the core rule, a decision order, ten rules, an 8-step probe procedure, a@probetemplate, BAD/GOOD examples, the commit workflow and report template.references/probe-catalog.md: plausible-form axes for each kind of guard, and A/B probing.references/uncompilable-violations.md: required parameters,Omit, branded types and builders;@ts-expect-errorprobes; when to delete or demote the old detector.references/source-scanning.md: AST-first scanning, plus a comment-safe scanner with fixture tests for when a regex is unavoidable.references/commit-workflow.md: the before/during/after checklist, gate health, and reporting and documentation rules.
Who it's for
Teams that let Claude Code or another agent write tests and CI gates. Maintainers whose invariants rely on custom tests. Reviewers who want evidence in an agent's commit, not the words "added a test".
What it is NOT
- Not a mutation-testing tool. It installs nothing and runs nothing by itself; the agent uses your existing test runner, linter and type checker.
- Not a coverage generator. You get fewer, stronger guards.
- Not tied to one stack. Examples are in TypeScript and Vitest, with notes for Jest, pytest, Go and ESLint.
- Not a promise that every guard ends up airtight. When a plausible probe stays green, the agent records it as a blind spot instead of hiding it.
Why it's different
Most testing advice asks for more tests. This skill asks one question of each guard: which broken input would still pass this assertion? The answer stays in the code, so the next reader can rerun the probe. It costs a few extra test runs per guard. In return, you don't find out after an incident that the test was green all along.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Code Quality & Review.
Requires local command execution to verify exit codes. Not suitable for non-deterministic tests. Type-level probes require a compiler-aware environment.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 20 days ago
- Software
Gitvitestjest
Node.js
Permissions
File Scopes
- Probes deliberately break code. The skill tells the agent to copy the file or `git stash` before probing, never to run `git checkout --`, `git reset --hard` or `git clean -f` over uncommitted work, and to confirm `git diff` shows only intended changes before committing. - The agent commits only when you ask for a commit, and pushes only when you ask.