More screenshots

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIDeepSeekDeepSeekMuseMuse+22 more

    Silent Failure Forge

    1

    Expose silent failures, false completion, goal drift and reliability risks in AI agents. Use adversarial testing, measurable contracts and evidence-based production gates to assess whether an agent deserves your trust.

    $19

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    Silent Failure Forge

    Silent Failure Forge

    Example session with this skill installed

    Demonstrate Silent Failure Forge using its included testing components. Run the harness self-test, contract-linter self-test, toy-agent demonstration and example trace grader. Explain how the toolkit distinguishes a failing agent, a good toy agent and an intermittent agent. Show how SilentRisk cases are identified from external-state verification, and explain why the resulting decisions do not constitute production certification. Include the actual test outcomes and evidence limitations.

    • Read your context and instructions
    • Compiled the silent failure forge

    Silent Failure Forge — Demonstration Results

    The complete release package was tested from its buyer-facing release directory.

    1. Harness self-test

    Result: 14/14 checks passed.

    The tests confirmed that the harness

    • Rejects zero and negative run counts.
    • Rejects invalid thresholds.
    • Prevents toy agents from receiving production Go.
    • Prevents automatic production Go, even when evidence is supplied.
    • Keeps clean trace-grader results at TRACE-GRADE-ONLY.
    • Returns No-Go when trace failures are present.
    • Rejects invalid CLI inputs and missing evidence artifacts.
    • Retains tool logs for every run.

    2. Contract-linter self-test

    Result: ALL PASS.

    The linter detected vague or weak contract language, incomplete contracts and inadequate quantitative thresholds.

    Both included example contracts passed full validation:

    • onboarding-agent-contract.md
    • research-system-contract.md

    The test also confirmed the required sample sizes: 149 runs for a 2% target and 299 runs for a 1% target under the linter's configured calculation.

    3. Toy-agent demonstration

    The included demonstration ran three types of deterministic toy agents.

    Bad agent — 150 runs

    Decision: No-Go.

    It failed all seven adversarial test categories:
    T1a, T1b, T1c, T2, T3, T4 and T5.

    Good agent — 150 runs

    Decision: HARNESS-CHECK-ONLY.

    No failing tests were reported. The result is correctly limited to a harness demonstration and does not establish real-agent reliability.

    Intermittent agent

    At 5 runs, it failed the T3 phantom-activation test.

    At 150 runs, it exposed failures in T1c, T3 and T5.

    Decision: No-Go.

    This demonstrates why a single successful run is not enough to assess reliability.

    4. Trace-grader demonstration

    The included example trace file contained four runs.

    • SilentRisk: 2
    • Success: 1
    • Escalated: 1
    • Failed: 0

    Suite status: FAIL

    Decision: No-Go (E1-traces).

    The grader identified a critical failure because the supplied traces contained claims of success inconsistent with reconstructed external state.

    5. Evidence limitations

    These results demonstrate the included testing tools operating against their sample environment and example traces.

    They do not establish that a real external AI agent has passed testing. The toy-agent results are HARNESS-CHECK-ONLY, and the trace-grader results are E1-traces.

    A real system or faithful staging environment, independently checkable external state, retained raw logs and human review are required for stronger evidence.

    Silent Failure Forge helps teams replace confidence-based release decisions with structured testing and verifiable evidence.

    onboarding-agent-contract.md

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Silent Failure Forge

    Your AI agent says the task is complete. Did it actually succeed?

    An AI agent can produce a convincing answer, report a successful tool call and still leave the real-world task incomplete. The problem becomes more serious when agents operate across multiple steps, interact with external systems or perform actions that are difficult to reverse.

    Silent Failure Forge is an evidence-driven reliability evaluation toolkit designed to help developers expose these failures before placing greater trust in their agents.

    Instead of relying on polished demonstrations or confident completion messages, it gives you practical ways to define expected behaviour, introduce adversarial conditions, inspect outcomes and make more disciplined release decisions.

    What you get

    1. Silent Failure Contracts

    Define measurable guarantees, hard refusals, success conditions, quantitative thresholds and operational boundaries for your AI agent.

    2. Contract Linter

    Check contracts for missing requirements, vague language, incomplete sections and insufficient test thresholds.

    3. Adversarial Testing Harness

    Exercise agents against deliberately injected failures, including tool errors, misleading responses, goal drift, false completion and context pressure.

    4. Trace Grader

    Inspect recorded tool traces and identify cases where an agent claims success while the reconstructed external state indicates otherwise.

    5. Adapter Skeleton

    Use the included Python adapter as a starting point for connecting your own agent to the testing harness.

    6. Evidence Framework

    Separate design assumptions from actual testing through four evidence levels:

    • E0 — Paper: design and documentation review.
    • E1 — Simulated: tests against simulated environments or supplied traces.
    • E2 — Executed: testing against a real system or faithful staging environment with independently checkable state and retained logs.
    • E3 — Live: shadow, canary or production evidence requiring human review.

    7. Production-Gate Checklist

    Support human-owned No-Go, Conditional Go and Go decisions using explicit evidence requirements.

    Built around evidence, not appearances

    The included harness evaluates outcomes against the state of its simulated external system rather than trusting the agent's own report.

    The trace grader offers a lower-friction way to inspect existing logs, but trace grading alone does not establish that those logs are authentic or prove production readiness.

    The toolkit deliberately distinguishes simulated test results from executed and live evidence. A clean toy-agent demonstration is a check of the testing framework, not proof that a language-model agent is reliable.

    The harness does not automatically certify an agent for production. Final release decisions remain human-owned.

    Who is it for?

    • AI agent developers
    • Multi-agent system builders
    • Automation engineers
    • Teams deploying long-running or semi-autonomous workflows
    • Technical reviewers assessing agent reliability before release

    Requirements

    Python 3.9 or later for the included testing tools. The harness uses the Python standard library and does not require third-party packages.

    Connecting your own AI agent requires an adapter implementing the documented interface.

    The objective is simple: don't trust an agent's success message. Verify what actually happened.

    Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Agents & Orchestration.

    • The included adversarial harness operates against a simulated external system; passing its tests does not prove that a real AI agent is production-ready.
    • The included toy agents demonstrate harness behaviour and are not representative of real-world LLM performance.
    • The trace grader evaluates supplied tool-call logs without injecting new faults. It cannot independently authenticate the source or truthfulness of those logs.
    • Testing your own AI agent requires a custom adapter implementing the documented interface.
    • The sample World models an account-provisioning workflow. Other domains require adapting the World, fault injections, test definitions and outcome classifier.
    • E2 and E3 evidence require separate execution against a real system or faithful staging environment, independently checkable external state and retained evidence.
    • The evidence manifest checks artifact presence and SHA-256 integrity; it does not authenticate the evidence source or prove its contents are truthful.
    • Production Go is never issued automatically. Final production decisions require human review and completion of the applicable checklist.
    • Python 3.9 or later is required to run the included testing tools.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before buying a skill.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 7 days ago

    Software
    Python 3.9 or laterTerminal / Shell

    File system access to the project workspace. File system access to the project workspace. File system access to the project workspace. File system access to the project workspace. File system access to the project workspace.

    Permissions

    Terminal / Shell
    Read Files
    Write Files

    File Scopes

    harness/**
    scripts/**
    tests/**
    references/**
    examples/**
    **/*.md
    **/*.py
    **/*.json
    **/*.log
    **/*.txt

    Grant access only to the project workspace and designated output folder. The included tools require no external services, credentials, network access or environment variables. Review custom adapters before connecting live agents or allowing external side effects.

    Listed7 days ago

    What's inside

    Frequently Asked Questions

    This skill acts as an adversarial gatekeeper that uses fault injection and evidence-based contracts to catch 'fail-plausible' scenarios where agents report success despite internal errors or goal drift.

    The skill is architecture-agnostic and includes a standardized adapter skeleton, allowing it to interface with single agents, sequential chains, or complex multi-agent swarms.

    You will receive the Python-based testing harness, the contract linter tools, the adversarial protocol documentation (T0-T6), and the production-gate checklist templates.

    Setting up requires mapping your agent's tool calls and autonomy triggers to the provided adapter; once connected, you can begin drafting the Silent Failure Contract immediately.

    While it can analyze paper-based design (E0/E1), it requires a runnable adapter or live tool traces to provide high-fidelity (E3) evidence and a final production Go/No-Go decision.

    Basic familiarity with Python, command-line tools and AI agent workflows is recommended. The included README, examples and adapter skeleton provide a starting point.

    Not automatically. The included harness uses a simulated environment. To test your own agent, you must implement an adapter using the documented interface and connect it to the harness.

    No. A passing simulation is limited evidence. Production decisions require independently checkable real-system or faithful-staging evidence, retained logs, the applicable checklist and human review.

    The harness executes adversarial tests against its simulated World. The trace grader examines previously recorded tool-call traces without injecting new faults. Trace grading is limited to E1-traces evidence.

    The included testing tools use Python's standard library and require no third-party Python packages. Connecting an external AI agent may involve its own model, API or runtime requirements.

    Yes. The included README explains how to replace the sample World, fault injections, test definitions and outcome classifier with domain-specific equivalents.

    No. Production Go remains a human-owned decision. The harness reports test outcomes and applies evidence restrictions; it does not independently certify production readiness.

    The downloadable package includes the SKILL.md instructions, runnable harness, trace grader, adapter skeleton, toy agents, contract linter, example contracts, reference documents and self-tests.

    New in Agents & Orchestration