Browse The Skill Store

A high-performance scraping engine with Playwright stealth, proxy rotation, and anti-bot bypass capabilities.

🤖 Robots Sitemap Validator

by JustHandled Labs

Free

Check your robots.txt and sitemap.xml for the mistakes that quietly block crawlers. Flags a site-wide Disallow: /, malformed directives, a missing Sitemap: line, sitemap XML that is not well-formed, <loc> URLs missing a scheme, mixed http and https, and sitemaps over the 50,000-URL or 50MB limits. Reads local files or pasted text and never touches the live site.

rsl pay per crawl setup

by Elyas Shukri Elmi

$39

Generates machine-readable AI licensing terms and robots.txt configs for pay-per-crawl and bot management.