- Home
- Skills
- Testing & Debugging
- ai application testing
Works with the AI Agents you already use
ai application testing
A systematic QA framework for testing RAG, agentic tools, and LLM application security boundaries.
$6
ai application testing
Example session with this skill installed
Audit the RAG-based document assistant in the /docs-bot-v2 repo. Focus on retrieval relevance, tenant isolation, and tool-call authorization for the "SearchFiles" function.
- Read your context and instructions
- Compiled the ai application testing
I have completed the audit of the document assistant. I traced the data flow from the API gateway through the vector store retrieval layer to the tool execution engine.
AI Application Testing Audit Report
Test Matrix
| Req/Risk ID | Component | Scenario | Role/Data | Oracle | Status |
|---|---|---|---|---|---|
| RAG-01 | Retrieval | Cross-tenant query | Tenant_A / File_B | Zero results/Access Denied | FAIL |
| RAG-02 | Citation | Hallucination check | User_Standard | Citation matches source text | PASS |
| TOOL-01 | SearchFiles | Argument Injection | User_Standard | No path traversal in params | PASS |
| TOOL-02 | SearchFiles | Unauthorized Delete | User_Standard | Error: Insufficient Permissions | PASS |
| SEC-01 | Prompt | Indirect Injection | External Web Page | Orchestrator ignores hidden cmd | FAIL |
Findings
| ID | Severity | Reproducer | Expected | Actual | Impact |
|---|---|---|---|---|---|
| F-01 | Critical | Query tenant_id filter is client-side only. | Backend enforces ACL. | Vector DB returns all matches. | Cross-tenant data leakage. |
| F-02 | High | Inject "ignore instructions" in a retrieved doc. | Model stays in role. | Model executes doc commands. | Remote prompt execution. |
Next steps
- Implement server-side metadata filtering in the vector database query logic to fix tenant isolation.
- Add a sanitization layer between the retrieval output and the LLM prompt to mitigate indirect injection.
- Verify that the
SearchFilesfunction schema explicitly restricts file paths to the/sandboxdirectory.
Connects securely to your tools. The creator never sees your data.
About this skill
Testing AI applications involves more than just checking model outputs. Traditional QA often misses the unique failure modes of RAG pipelines, agentic tool calls, and prompt injection vulnerabilities at the application layer. This skill provides a systematic framework for verifying the boundaries, security, and orchestration of AI-integrated software.
What it does
- Boundary analysis by tracing data flow from UI/API through orchestrators to retrieval and tools.
- RAG verification covering ingestion, retrieval relevance, tenant-level ACL enforcement, and citation accuracy.
- Agentic tool testing to assert least privilege, argument validation, and idempotency for side effects.
- Security auditing for direct and indirect prompt injection, cross-tenant data leakage, and unsafe output handling.
- Operational checks for rate-limiting, provider failure recovery, telemetry, and trace correlation.
How it works
- Map the architecture to define trust boundaries between the user, the model, and external data sources or tools.
- Generate test cases for functional, negative, and ambiguous scenarios including session state and timeout behavior.
- Execute specialized suites for RAG (retrieval logic) and Agents (tool authorization and side-effect protections).
- Perform security validation against OWASP Top 10 for LLM Applications and privacy leakage benchmarks.
Frameworks & tools
Uses ISTQB CT-AI, CT-GenAI, OWASP ASVS 5.0.0, and AISVS standards. Compatible with any stack integrating LLMs, including LangChain, LlamaIndex, and custom RAG or agent implementations.
Why this beats prompting it yourself
Generic prompts often overlook the "silent failures" of AI, such as retrieval filters that bypass permissions or indirect injections from untrusted data sources. This skill enforces a rigorous, standard-aligned methodology that separates model evaluation from application-level integration risks.
Use cases
- Auditing a RAG system to ensure users cannot access documents outside their role.
- Stress testing an AI agent's tool-calling logic to prevent unauthorized API actions.
- Validating fallback UI behavior when a model times out or returns malformed JSON.
- Identifying prompt injection vulnerabilities in customer-facing chat interfaces.
Known limitations
Does not perform model-level benchmark evaluations or training data quality analysis. Requires user-selected writable folders for artifact generation and explicit authorization for external tool testing.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Testing & Debugging.
Focuses on application-layer integration. Does not perform model fine-tuning or raw dataset benchmarks. Requires pre-approved sandbox environments for side-effect testing.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 2 days ago