More screenshots

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIDeepSeekDeepSeekMuseMuse+22 more

    E-commerce Competitor Pricing Scraper Setup

    1

    Build a local competitor-price monitor with resilient selector fallbacks, SQLite history, markup-drift detection, polite request controls, and cron-ready execution.

    $9

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    E-commerce Competitor Pricing Scraper Setup

    E-commerce Competitor Pricing Scraper Setup

    Example session with this skill installed

    Build a local price monitor for the 35 competitor product URLs in competitors.csv.

    I also saved representative product pages in ./html-samples/.

    Requirements

    • prefer JSON-LD or product metadata before CSS selectors;
    • create multiple fallback selectors for price and availability;
    • map every competitor product to our merchant_sku;
    • store all observations in SQLite;
    • never convert failed extraction into price 0;
    • add markup-drift and access-review statuses;
    • use a polite per-domain delay;
    • stop on CAPTCHA or explicit bot-block pages rather than trying to bypass them;
    • create a cron example for recurring runs;
    • generate a maintenance guide for future HTML changes.
    • Read your context and instructions
    • Compiled the e-commerce competitor pricing
    • Generated the data export

    Competitor price-monitor setup complete

    Competitor products configured: 35
    Representative HTML pages inspected: 12
    Products using JSON-LD as primary source: 24
    Products using metadata fallback: 7
    Products requiring CSS fallback: 4
    Product-match reviews required: 2

    Extraction strategy

    SourcePrimary methodFallbacksStatus
    Competitor A / SKU-100JSON-LD offers.price3READY
    Competitor B / SKU-205product:price:amount2READY
    Competitor C / SKU-311data-testid="price"2READY
    Competitor D / SKU-411.product-price1REVIEW

    Runtime safeguards

    • Per-domain delay: 2.5 seconds
    • Request timeout: 20 seconds
    • Transient retries: 2
    • 401 / 403 / 429: stop and review
    • CAPTCHA / challenge page: stop and review
    • Failed price parse: never stored as 0.00
    • Historical observations: append-only SQLite

    Example observation

    FieldValue
    Merchant SKUSKU-100
    CompetitorCompetitor A
    Price79.99 USD
    Methodjsonld:offers.price
    StatusOK

    Markup-drift behavior

    If all configured extraction methods fail, the scraper records:

    SELECTOR_DRIFT_REVIEW

    The previous valid price remains in history. No fake replacement value is written.

    SQLite history

    Database

    data/price_history.sqlite

    Stored data includes

    • merchant SKU
    • competitor
    • URL
    • observed price
    • currency
    • availability
    • timestamp
    • extraction method
    • content hash
    • HTTP status
    • review status

    Files generated

    • scraper.py
    • scraper_config.json
    • sqlite_schema.sql
    • run_scraper.sh
    • cron.example
    • selector_strategy.md
    • maintenance_guide.md
    • latest_prices.csv
    • price_history.sqlite

    This package is designed for permitted public-page monitoring with polite request rates. Review site terms, access rules, selector drift, and product matching before production use.

    latest_prices.csv

    CSV · data export

    Generated

    Example file from a real run - the skill writes it into your workspace.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    The problem

    Competitive prices change constantly, while home-grown monitoring scripts break whenever markup changes.

    A scraper that works today can fail tomorrow because a competitor removes a class name, changes its product markup, renders price data differently, or starts returning an access challenge.

    What it does

    E-commerce Competitor Pricing Scraper Setup creates a complete local monitoring package from representative HTML or permitted public product pages.

    • Inspects sample HTML for stable price sources
    • Prefers JSON-LD and product metadata over brittle DOM paths
    • Generates ordered selector fallbacks
    • Stores append-only price history in SQLite
    • Preserves currency and availability data
    • Detects selector drift and extraction failures
    • Uses bounded retries and polite per-domain delays
    • Stops on blocked or CAPTCHA pages instead of bypassing them
    • Creates a cron-ready background runner
    • Includes a maintenance workflow for future markup changes

    Why this beats prompting it yourself

    A generic AI response might generate one BeautifulSoup selector.

    This skill builds the complete operating workflow:

    HTML Inspection → Selector Strategy → Runtime → SQLite History → Drift Detection → Scheduling → Maintenance

    Resilience without anti-bot evasion

    The workflow uses multiple legitimate extraction paths and structured-data fallbacks.

    It does not include CAPTCHA bypass, stealth browser plugins, proxy rotation, fingerprint spoofing, or authentication bypass.

    Local-first price history

    Observations are stored locally in SQLite so teams can maintain their own historical pricing dataset without depending on a hosted monitoring service.

    Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills to Reverse Engineer and Scrape Websites.

    Cannot bypass CAPTCHAs, 403 blocks, or paywalls. Requires local Python environment. Does not include proxy rotation or browser fingerprinting.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before buying a skill.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 1 month ago

    Permissions

    Terminal / Shell
    Browser
    Network Access
    Listed1 month ago
    Updated20 days ago

    What's inside

    Frequently Asked Questions

    This skill generates a complete local Python-based monitoring environment, including a script with fallback extraction logic, an SQLite database for price history, and a cron-ready execution workflow.

    Unlike basic scrapers, this setup prioritizes structured metadata (JSON-LD) and uses a multi-layered selector strategy that automatically tries alternative paths if the primary HTML class changes.

    No, this skill is designed for ethical and sustainable data collection; it uses polite request delays and stops upon encountering blocks rather than using fingerprint spoofing or CAPTCHA bypass services.

    This skill is compatible with advanced LLM-based agents (such as AutoGPT or custom GPT-4 agents) that have filesystem access to write Python scripts and initialize local SQLite databases.

    The setup includes a built-in drift detection and maintenance workflow that allows your AI agent to quickly identify broken selectors and regenerate the logic without rebuilding the entire database.

    Yes, the generated script uses a local SQLite database, allowing you to build and own a permanent, private historical pricing dataset without paying for a monthly SaaS subscription.

    New in E-commerce