Works with the AI Agents you already use
ADK-EvalGate: Agent CI Regression & Provenance Harness
Block silent prompt, model and tool-schema regressions in CI with deterministic Google ADK trajectory checks, injection probes and SHA-256 provenance.
$49
ADK-EvalGate: Agent CI Regression & Provenance Harness
Example session with this skill installed
Gate candidate ADK trace run_pr184.jsonl against baseline run_v2_4.jsonl and golden_suite.json with --strict-exit for our pull request.
- Read your context and instructions
- Compiled the adk-evalgate
Exit code 1 (BLOCK): 3 of 5 golden trajectories regressed—catching refund_order(amount_cents='true') JSON Schema type failure, a 4x identical search_kb retry loop (+68% token spike), and leaked injection marker CANARY_7F9A—with SHA-256 provenance scorecard and PR comment Markdown.
Connects securely to your tools. The creator never sees your data.
About this skill
A minor prompt tweak or model version bump can silently break tool-call JSON types, spin an agent into a 4x retry loop, or let an untrusted document exfiltrate a canary secret—long before manual spot-checks notice.
What your AI assistant does with it
- ADK & JSONL trajectory replay: Evaluates Google ADK session event dumps and flat traces against golden tool sequences, forbidden tools and step/cost caps.
- Strict tool-arg schema gate: Validates every function_call payload against JSON Schema—catching bool-as-int confusion, missing fields and bad enums.
- Untrusted-input canary audit: Verifies adversarial web or email tool outputs never leak canary markers into outbound tool arguments or logs.
- Tamper-evident CI provenance: Scrubs leaked API keys from traces, exits non-zero on regressions and emits a SHA-256 signed scorecard for PRs.
How a run works
- Define your golden cases, tool JSON Schemas and regression policy thresholds
- Capture baseline and candidate Google ADK or agent trajectory JSON/JSONL logs
- Run adk_eval_gate.py --strict-exit locally or inside GitHub Actions
- Review the PR regression table, schema errors and hashed provenance scorecard
Built for: Heads of AI & Platform leads, Staff ML & Agent engineers, Google ADK & MCP builders.
Included: Skill instructions with baseline-promotion STOP AND VERIFY rules, adk_eval_gate.py: ADK trajectory validator, JSON-Schema checker & canary gate, PR Markdown diff, SpreadsheetML workbook & offline HTML CI dashboard, Google ADK 2.x event-trace & GitHub Actions CI integration reference, 5-case regression evaluation suite.
Part of the B2B Operations & FinOps suite by Production AI Agents. Every calculation runs locally in standard-library Python (Decimal math, zero hidden network calls) and pauses at a human STOP AND VERIFY gate before any external write or sign-off.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for LLM Ops.
Deterministic trajectory, schema and canary regression harness; use agent-evals for Wilson/bootstrap statistical intervals on open-ended prose. Always review generated calculations, drafts and SQL/API payloads before executing external actions.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed today
- Software
Python 3.10+
- Optional
GitHub CLI
Bundled Python scripts use standard library only (zero pip installs) and run locally.
Permissions
File Scopes
adk_eval_gate.py runs 100% deterministically offline with zero LLM API calls and scrubs leaked API keys/bearer tokens before writing reports.