π€ Robots Sitemap Validator
Check your robots.txt and sitemap.xml for the mistakes that quietly block crawlers. Flags a site-wide Disallow: /, malformed directives, a missing Sitemap: line, sitemap XML that is not well-formed, <loc> URLs missing a scheme, mixed http and https, and sitemaps over the 50,000-URL or 50MB limits. Reads local files or pasted text and never touches the live site.
- Prevent accidental site-wide de-indexing in robots.txt
- Verify that sitemaps referenced in robots.txt exist locally
- Check sitemap.xml files for valid XML syntax and structure
Free
Included in download
- Downloadable skill package
- Works with Cursor, Codex CLI
- 2 permissions declared
Sample input
Audit the robots.txt and sitemap.xml in my public directory to make sure I'm not blocking search engines from the main site.
Sample output
Scan results for /public:
- [CRITICAL] robots.txt L4: User-agent: * Disallow: / (Blocks all crawling)
- [WARNING] robots.txt L12: Sitemap: /sitemap_index.xml (File not found in local directory)
- [INFO] sitemap.xml: Valid XML structure found with 42 URLs.
Check your robots.txt and sitemap.xml for the mistakes that quietly block crawlers. Flags a site-wide Disallow: /, malformed directives, a missing Sitemap: line, sitemap XML that is not well-formed, <loc> URLs missing a scheme, mixed http and https, and sitemaps over the 50,000-URL or 50MB limits. Reads local files or pasted text and never touches the live site.
Free
Included in download
- Downloadable skill package
- Works with Cursor, Codex CLI
- 2 permissions declared
- Instant install
Sample input
Audit the robots.txt and sitemap.xml in my public directory to make sure I'm not blocking search engines from the main site.
Sample output
Scan results for /public:
- [CRITICAL] robots.txt L4: User-agent: * Disallow: / (Blocks all crawling)
- [WARNING] robots.txt L12: Sitemap: /sitemap_index.xml (File not found in local directory)
- [INFO] sitemap.xml: Valid XML structure found with 42 URLs.
About This Skill
Ensure Search Engines Can Crawl Your Code
The Robots Sitemap Validator is a specialized tool for developers and SEO teams to catch critical crawling mistakes before they hit production. It automates the tedious process of auditing local sitemap and robots.txt files, ensuring your site's architecture is search-engine friendly.
What it does
- Identifies syntax errors in
robots.txtandsitemap.xmlfiles. - Detects "orphan" sitemaps where a robots.txt references a file that doesn't exist locally.
- Flags high-severity blocking rules that might accidentally de-index important sections of your site.
- Provides structured feedback including rule IDs, severity levels, and exact line numbers.
Why use this skill?
Manual SEO audits are prone to human error, and production-only crawlers find issues too late. This skill integrates into your local development or CI/CD workflow to detect issues while you're coding. It supports standard web protocols and works with any directory structure, providing a layer of safety that generic LLM prompting cannot match in terms of precision and file-system awareness.
Use Cases
- Prevent accidental site-wide de-indexing in robots.txt
- Verify that sitemaps referenced in robots.txt exist locally
- Check sitemap.xml files for valid XML syntax and structure
- Identify high-severity crawl-blocking rules before deployment
Known Limitations
It validates the files you give it, so it cannot confirm that a referenced sitemap exists unless that file is provided alongside (it flags those for review). It does not fetch the live site, and the URL-count and size checks are guidance notes against the documented limits.
How to Install
mkdir -p ~/.claude/skills && curl -sL https://www.agensi.io/api/install/robots-sitemap-validator -o /tmp/robots-sitemap-validator.zip && unzip -o /tmp/robots-sitemap-validator.zip -d ~/.claude/skills && rm /tmp/robots-sitemap-validator.zipFree skills install directly. Paid skills require purchase - use the download button above after buying.
Reviews
No reviews yet - be the first to share your experience.
Only users who have downloaded or purchased this skill can leave a review.
No reviews yet - be the first to share your experience.
Only users who have downloaded or purchased this skill can leave a review.
Security Scanned
Passed automated security review
Permissions
Allowed Hosts
File Scopes
Notes: Read-only. Also accepts pasted text via --stdin. Prints findings with rule id, severity, file, and line. It does not crawl, render JavaScript, submit to search engines, or read environment variables.
Works with any agent that can read a file and run a local Python script (Claude Code, Cursor, Codex CLI, and other SKILL.md-compatible agents). Standard library only, no install step. Parses XML with the standard library and makes no network calls.
Creator
JustHandled Labs creates focused agent skills and workflow packs for Claude, Codex, Cursor, and AI-assisted builders. Each tool is designed around a real repeatable task: cleaner commits, better PRs, stronger handoffs, safer repo hygiene, clearer documentation, and less copy-paste chaos. The goal is not generic AI productivity. The goal is specific workflows that are easier to run, review, and repeat. Maintained by H.J. Westerfield, with a background in communications, editing, project coordination, customer support, and practical AI systems. JustHandled Labs builds tools for people who want useful automation without theatrical complexity.
Frequently Asked Questions
Learn More About AI Agent Skills
More Premium Skills
keyword-research
Transform URLs or product lists into SEO keyword research packs with Google Ads data and intent-based clustering.
local-seo-audit
Automated local SEO auditing and keyword generation for agencies and small businesses.
cinematic-sites
Turn any basic business URL into a high-end cinematic landing page with AI-generated 4K assets and GSAP animations.
endless-loop
Autonomous research and task loop that builds on previous findings to solve complex objectives while you sleep.