More screenshots

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIDeepSeekDeepSeekMuseMuse+22 more

    Playwright Scraper: Extract Pages That Fight Back

    1

    Extract content from sites that need a real browser — the escalation ladder from plain fetch to stealth, waiting that works, and scraping ethics that keep you out of trouble.

    $5

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    Playwright Scraper: Extract Pages That Fight Back

    Playwright Scraper: Extract Pages That Fight Back

    Example session with this skill installed

    A React job board with infinite scroll returns empty HTML to plain fetch. Scrape all listings — title, company, location, link — and make it survive the next redesign.

    • Read your context and instructions
    • Compiled the playwright scraper

    Level 1 first: plain HTTP fetch returned the shell — empty content, 200 OK — which is the classic JavaScript-rendered signature. Level 2 sufficed: headless Chromium, no tricks, because the board has no anti-bot. Never start at level 3. Script: launched headless Chromium (headed mode kept one flag away for debugging), set viewport, user agent, locale, and timezone to match a plausible visitor. Navigated and waited for the content selector [data-testid='job-card'] — never a fixed sleep, never the page load event, which SPAs fire long before data arrives. Paginated by infinite scroll: scroll to bottom, wait 1.5s, count cards, repeat until the count stops growing. Two consecutive scrolls adding nothing means done. Extracted title, company, location, and link per card via $$eval, preferring semantic selectors over brittle CSS paths, and pulled the embedded JSON where the page offered it — cleaner than DOM scraping. Persisted raw HTML per page plus a jobs.json. 340 listings in about 4 minutes, browser closed in a finally block. When the site redesigns next quarter, the saved HTML shows exactly which selectors broke — the extractor gets fixed without a re-scrape. The ladder is the discipline: the lightest tool that works, proven by trying, not by reaching for the heaviest tool first.

    than DOM scraping. Persisted raw HTML per page plus a jobs.json

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Some pages only exist after JavaScript runs; some actively fight automation. This skill is the escalation ladder: start with the lightest tool that works and only bring out the browser — and the stealth — when the page demands it. It covers Playwright setup, the script anatomy (launch, context, navigate and wait, extract, paginate, persist, close), waiting strategies that fix flaky scrapers, stealth honestly (what anti-bot systems actually look for and where the line is), and a failure playbook that maps symptoms to causes to fixes.

    Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse All Claude Skills.

    Bypassing a CAPTCHA or terms of service to scrape at scale can be unlawful depending on jurisdiction and target — the skill teaches the line, it does not cross it.
    Stealth is countermeasure work: expect to iterate, and know that a residential or mobile IP is often the actual fix, not fancier JavaScript.
    Specialized sources (YouTube transcripts, Reddit threads) are better served by purpose-built scrapers than a generic browser script.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before buying a skill.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 3 days ago

    Software
    PlaywrightNode.js

    Permissions

    Terminal / Shell
    Browser
    Listed3 days ago

    What's inside

    Frequently Asked Questions

    The page is JavaScript-rendered and fetch saw the shell. Use the browser — level 2, headless Chromium, wait for the content selector.

    Stealth pass: patch the webdriver flag and fingerprint leaks, randomize delays, and check whether a datacenter IP is the actual problem. Know the line — do not bypass CAPTCHAs to scrape at scale.

    A/B test or redesign. Save raw HTML on every run and anchor on semantic selectors — the saved HTML shows exactly what broke.