Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIDeepSeekDeepSeekMuseMuse+22 more

    Flaky Test Triage Card

    1

    Sort flaky failures into timing, state, order, network and nondeterminism, with the diagnostic command and the fix for each.

    Free

    Security scanned
    Flaky Test Triage Card

    Flaky Test Triage Card

    Example session with this skill installed

    Triage these failures: test_user_signup failed with 'timeout 5000ms' on worker 4 but passed on retry. test_delete_post passed alone but failed when run after test_create_post in the full suite.

    • Read your context and instructions
    • Compiled the flaky test triage

    I have classified these failures based on the provided log patterns. The signup failure points to an async timing issue, while the post deletion failure indicates a clear order dependence.

    Flaky triage — Backend Integration Suite — Last 24 Hours

    Collected: 2 failures across 48 runs

    TestCauseConfidenceEvidenceFixOwnerQuarantine expiry
    test_user_signupTiming / asyncHighTimeout 5000ms; passed on retryReplace sleep with wait_for_selector@dev_alpha2023-11-20
    test_delete_postTest-order dependenceHighPasses in isolation; fails in suiteEnsure test_create_post cleans up DB@dev_beta2023-11-18

    Diagnostic commands

    
    # Verify timing for signup
    pytest tests/auth/test_signup.py --count=20
    
    # Verify order dependence for post deletion
    pytest tests/blog/test_posts.py -p no:randomly
    

    Suite health

    Flaky rate: 4.1% of runs · Tests quarantined: 2 · Oldest quarantine: 2023-11-18

    Next steps

    1. Run the pytest --count=20 command to confirm the signup flake frequency.
    2. Update the test_create_post fixture to use a proper teardown or unique IDs.
    3. Set a calendar reminder for the quarantine expiry dates.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Retries hide flakiness and turn a signal into noise. This card tells you which kind of flaky you have, because each kind has a different one-line fix.

    What you get

    • Five causes with their log fingerprints: timing, shared state, test-order dependence, network, and nondeterminism in the code under test.
    • The commands that split them apart in minutes - repeat a single test 50 times, then vary the worker count.
    • Three rules that keep a suite honest: quarantine entries expire, retries are a measurement rather than a fix, fix the cheap ones first.
    • A triage report with a confidence column, an owner, and a quarantine expiry date.

    How it works

    Collect failures over a window instead of from one red build. If a test fails 1-in-50 alone, it is timing or nondeterminism. If it only fails alongside others, it is state or order. That split removes most of the argument.

    Who it is for

    Anyone whose CI is intermittently red and whose team has started ignoring it.

    Limits

    It classifies what the logs show. If more than a third of failures are unknown, fix the logging before the tests.

    Free starter. For automated clustering by cause, a cost model that ranks which flakes are worth fixing, and quarantine automation with expiry, see Flaky Test Triage: https://www.agensi.io/skills/flaky-test-triage

    Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Testing & Debugging.

    Requires failure logs and retry data. Cannot fix underlying product bugs or suites with zero isolation. Classification depends on log quality.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before buying a skill.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Free to use

    No account required to browse

    Trust & safety

    Security scan

    Security scan passed 2 days ago

    • Free with an Agensi account

    Software
    PlaywrightNode.js

    Permissions

    Terminal / Shell
    Browser
    Listed2 days ago

    What's inside

    Frequently Asked Questions

    The skill analyzes log signatures and failure patterns to categorize flakes into specific buckets like 'order dependence,' 'shared state,' or 'timing,' rather than just labeling them as generic errors.

    It is compatible with any AI agent that supports structured text input, and it provides specific diagnostic support for Pytest, Playwright, and Jest frameworks.

    You will receive the core skill logic designed for your AI agent, along with a library of diagnostic command templates and structured triage card formats for immediate use.

    Simply provide the skill with your test names, error logs, and CI retry results; it will then output a structured triage card with reproduction steps and quarantine recommendations.

    Unlike generic prompts that often suggest 'adding a wait,' this skill follows a structured diagnostic process to identify the true root cause, preventing you from just masking the problem.

    Yes, the skill includes a quarantine rule generator that sets mandatory expiry dates and owner assignments to ensure tests are eventually fixed rather than ignored indefinitely.

    New in Testing & Debugging

    Free