Works with the AI Agents you already use
AI Feature Safety Review
A pedantic, breadth-first safety review of the **AI surface** of a web application - chat, copilots, RAG pipelines, tool-using agents, MCP servers and model-generated content.
$6
AI Feature Safety Review
Example session with this skill installed
Run an AI feature safety review on this repo. It's a Next.js app with a support chat built on the OpenAI SDK, RAG search over our documentation via pgvector, and an agent tool that can create Linear tickets from chat. Ingested docs come from user uploads. Tell me what an attacker could do and what to fix before launch.
- Read your context and instructions
- Compiled the ai feature safety
- Generated the data export
Summary
Reviewed the AI surface of your Next.js app: 4 model call sites, 1 RAG pipeline, 3 tools, 2 agent loops. Sweep covered 86 checks - 61 PASS, 14 FAIL, 8 N/A, 3 UNKNOWN (see coverage.csv).
Top findings
| Sev | ID | Finding | Location |
|---|---|---|---|
| Critical | TL-04 | Agent createTicket executes with a service-role API key; no per-user authorization - attacker-controlled doc content can steer ticket creation into any project | src/agent/tools.ts:41 |
| High | INJ-02 | RAG chunks from user uploads enter the prompt unlabeled; forged instructions in a PDF reach the chat answer and the Linear tool | src/rag/ingest.ts:88 |
| High | OUT-01 | Model answers rendered as raw markdown without sanitization - stored XSS via an ingested doc | components/ChatMessage.tsx:23 |
| Medium | COST-03 | No per-user token/spend cap on the chat endpoint | api/chat/route.ts:57 |
| Medium | RAG-05 | Vector store namespace not scoped by tenant; similarity search crosses tenants on shared terms | src/rag/store.ts:112 |
Autonomy tiers found
- Chat answer: T1 - Doc search: T2 - Linear ticket creation: T3 (reversible write, no approval, no idempotency)
Next steps
- Execute createTicket with the requesting user's identity and a server-side authorization check per call (threat-model.md, blast-radius matrix)
- Label retrieved chunks as untrusted; sanitize markdown at the sink
- Add a spend cap and an eval set covering benign injection cases (eval-gaps.md)
Full report: ai-safety/REPORT.md - coverage ledger: ai-safety/coverage.csv
coverage.csv
CSV · data export
Example file from a real run - the skill writes it into your workspace.
Connects securely to your tools. The creator never sees your data.
About this skill
A pedantic, breadth-first safety review of the AI surface of a web application - chat, copilots, RAG pipelines, tool-using agents, MCP servers and model-generated content. It finds the paths by which untrusted content becomes privileged action or data exposure, and the missing engineering around the model (limits, evals, observability, rollback). Not a jailbreak contest, not a red-team exercise.
What you get
ai-safety/REPORT.md- findings with blast-radius severity, exactfile:linelocations, guardrail fixes and confidencethreat-model.md- AI surface inventory, data-flow traces and a tool ledger with autonomy tiers (T0-T4)coverage.csv+findings.json- every check marked PASS / FAIL / N/A / UNKNOWN, machine-readableeval-gaps.md- what your regression evals cover, what they cannot catch, what to add before the next prompt or model change- A 20-line chat summary: counts by severity, top 5 attack paths, autonomy tiers found
How it works
- Inventory - every call site, prompt channel, tool, agent loop, retriever, memory store and output sink, labeled trusted/untrusted, with an autonomy tier per flow
- Blast-radius matrix - per reachable tool: whose identity executes it, where authorization is enforced, argument validation, reversibility, worst case if arguments are attacker-controlled
- Checklist sweep - mapped to OWASP LLM Top 10 (2026) and the OWASP Agentic Top 10: prompt injection via untrusted content, excessive agency, tool authorization and confused-deputy access, cross-tenant retrieval and memory leaks, sensitive data sent to providers, hidden-context exposure, unsafe output handling, unbounded cost, missing evals and kill switches
- Per-call-site cards - 12 questions per call site: prompt channels, attacker-writable inputs, validation at sinks, bounds, caching, logging, fail-open vs fail-closed, eval coverage
- Report - false-positive guards applied (a finding requires a path: untrusted channel -> capability -> impact), severity rubric, coverage ledger
Guardrails
Static-first: default mode needs no model calls and no API keys. Canary tests run only in a test environment with synthetic data and an explicit call/spend budget you approve. No jailbreaks against production, no data extraction, no mutations of vector stores, memory or real systems. Secrets and PII are redacted, never pasted wholesale. Your code is not changed - patches are proposed, applied only on request.
Requirements
- Repo access to a web app with AI features
ripgrepfor the bundled static inventory script (recommended; the agent can also grep manually)- Nothing to install otherwise - works with any SKILL.md-compatible coding agent
Known limitations
- Dynamic checks need a test environment you approve; production black-box mode has severe coverage limits
- Findings require a traced path - "prompt injection is possible" alone is not a finding
- Non-AI authorization defects (plain IDOR etc.) are deduplicated and cross-referenced to a web hygiene audit
- The review proposes fixes but does not change your code unless asked
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Security.
Static-first: dynamic canary tests need a test environment with synthetic data and your explicit approval - production red-teaming is out of scope by design. Black-box mode has severe coverage limits. A finding requires a traced path; prompt injection being possible alone is not reported as a vulnerability. Non-AI authorization defects are cross-referenced to a web hygiene audit rather than duplicated. The review proposes fixes but does not change your code unless asked.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed today
- Software
- ripgrep
- Optional
- Test environment with synthetic data
Repo access to a web app with AI features.
Permissions
File Scopes
Reads your repo and writes findings to ./ai-safety/. The bundled inventory script needs ripgrep and runs locally with no network access. Optional grey-box canary checks run only in a test environment you approve, within a call and spend budget you set. No model calls, no API keys required.