2

    scrapling-technical-seo-crawler

    by Roy Yuen

    Professional technical SEO crawler and site auditor for deep architectural analysis and issue detection.

    Updated May 2026
    Security scanned
    One-time purchase

    $5

    One-time purchase · Own forever

    ⚡ Also available via Agensi MCP — your AI agent can load this skill on demand via MCP. Learn more →

    Included in download

    • Perform automated site audits for client onboarding or monthly reporting.
    • Identify indexability issues like missing canonicals and noindex tags.
    • network automation included
    • Includes example output and usage patterns
    • Instant install

    See it in action

    Found 12 pages.
Issues Identified:
- 4 pages missing Meta Description.
- 2 pages with Multiple H1 tags.
- 15 images missing Alt Text.
- 1 broken internal link on /about-us.
Report generated: technical-seo-report.md and issue-register.csv.

    About This Skill

    What it does

    This skill provides a professional-grade technical SEO crawler for AI agents, powered by the Scrapling framework. It performs deep structural audits of public websites to identify indexability issues, metadata gaps, and structural flaws. The crawler traverses same-domain pages to extract mission-critical SEO signals such as canonical tags, robots directives, heading hierarchies, and schema availability.

    Why use this skill

    Writing a custom web crawler that respects robots.txt, manages depth, and correctly handles dynamic content is complex. This skill handles the heavy lifting of state management and signal extraction. Unlike generic LLM browsing, this tool produces structured, multi-format artifacts (JSON, CSV, Markdown) ready for client delivery or data analysis. It provides deterministic, evidence-backed findings rather than LLM "hallucinations" about a site\'s structure.

    Supported tools & outputs

    • Scrapling Engine: Supports both standard HTTP and Dynamic JS-rendered fetching.
    • Rich Data Extraction: Captures meta signals, H1-H2 tags, link graphs, and JSON-LD schema.
    • Comprehensive Export: Automatically generates a technical audit report, issue register, link graph CSV, and structured JSON project files.
    • Flexible Modes: Supports live crawling with depth control or fixture mode for local HTML file auditing.

    📖 Learn more: Best Frontend & Design Skills for Claude Code →

    Use Cases

    • Perform automated site audits for client onboarding or monthly reporting.
    • Identify indexability issues like missing canonicals and noindex tags.
    • Map internal link structures and identify broken links across a domain.
    • Audit image accessibility and metadata health at scale.

    Reviews

    No reviews yet — be the first to share your experience.

    Only users who have downloaded or purchased this skill can leave a review.

    Security Scanned

    Passed automated security review

    Permissions

    Network Access

    Allowed Hosts

    example.com
    www.example.com

    File Scopes

    scrapling-technical-seo-crawler/**

    Tags

    seo
    web-scraping
    technical-audit
    python
    crawler

    Scrapling is required for live crawl mode pip install "scrapling[fetchers]" && scrapling install

    Creator

    Roy Yuen

    Frequently Asked Questions

    Learn More About AI Agent Skills

    Similar Skills

    seo-optimizer

    SEO optimizer and banned-word scanner for Chinese social media. Keyword optimization and advertising law compliance.

    Free42 installs

    code-reviewer

    Reviews your code for bugs, security vulnerabilities, logic errors, performance issues, and style violations. Organizes findings by severity and suggests fixes with code examples.

    Free179 installs

    git-commit-writer

    Writes conventional commit messages by analyzing your staged git changes. Detects commit type, scope, and breaking changes automatically.

    Free91 installs

    readme-generator

    Generates a complete, polished README.md by scanning your actual project structure, dependencies, and code.

    Free57 installs

    $5

    One-time