Browse The Skill Store

🤖 Robots Sitemap Validator

by JustHandled Labs

Free

Check your robots.txt and sitemap.xml for the mistakes that quietly block crawlers. Flags a site-wide Disallow: /, malformed directives, a missing Sitemap: line, sitemap XML that is not well-formed, <loc> URLs missing a scheme, mixed http and https, and sitemaps over the 50,000-URL or 50MB limits. Reads local files or pasted text and never touches the live site.

A high-performance scraping engine with Playwright stealth, proxy rotation, and anti-bot bypass capabilities.