ai rag markdown extractor

    1

    Convert noisy web pages into clean Markdown and semantic RAG chunks for vector databases and LLM context.

    $29

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIMuseMuseDotsDots+22 more

    ai rag markdown extractor

    Example session with this skill installed

    Extract the content from https://example-docs.com/api-reference into clean Markdown and provide RAG chunks of 500 tokens each.

    • Read your context and instructions
    • Compiled the ai rag markdown

    I have processed the requested URL, stripped all navigation and footer noise, and formatted the API reference into clean Markdown. Below is the structured output and the semantic RAG chunks.

    Markdown Extraction

    API Reference - Authentication

    To interact with the API, you must include a Bearer token in your Authorization header.

    ParameterTypeDescription
    api_keystringYour unique project key
    tenant_idUUIDThe target organization ID

    Endpoints

    GET /v1/user

    Retrieves the current user profile.

    curl -X GET https://api.example-docs.com/v1/user \
  -H "Authorization: Bearer YOUR_TOKEN"

    RAG Semantic Chunks

    Chunk 1 (Tokens: ~180)

    Context: Authentication Overview
    To interact with the API, you must include a Bearer token in your Authorization header. Keys can be managed in the developer dashboard under the 'Security' tab. All requests must be made over HTTPS.

    Chunk 2 (Tokens: ~240)

    Context: User Endpoints
    GET /v1/user: Retrieves the current user profile. Returns a JSON object containing the user's UUID, email address, and active role within the tenant. Requires the 'user:read' scope.

    Next steps

    1. Copy the Markdown content into your project's /docs folder.
    2. Upsert the provided RAG chunks into your Pinecone or Supabase vector store.
    3. Run the llms.txt generator command to index these new files.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Web pages are filled with noise that degrades LLM performance and increases token costs. This skill strips away headers, footers, and ads to produce clean, structured Markdown ready for RAG pipelines or prompt context.

    What it does

    • HTML purification: Removes navigation menus, cookie banners, sidebars, and tracking scripts from raw source code.
    • Table conversion: Transforms complex HTML table structures into valid GitHub-Flavored Markdown tables.
    • Link resolution: Converts relative hyperlinks into absolute URLs to maintain reference integrity.
    • Semantic chunking: Segments long documents into chunks based on character or token budgets while preserving paragraph boundaries.
    • Specification drafting: Generates compliant llms.txt files to help search engines and agents index your documentation.

    How it works

    1. Fetch: Provide a URL or raw HTML source for processing.
    2. Clean: The skill executes DOM cleaning to discard non-content elements and CSS noise.
    3. Format: Content is converted to Markdown with preserved headers and lists.
    4. Chunk: Specify a token limit to receive segmented text blocks for vector database ingestion.

    Frameworks & tools

    This tool outputs data optimized for Pinecone, ChromaDB, Qdrant, and Supabase Vector. It is designed for use within Cursor, Claude Code, LangChain, and LlamaIndex workflows.

    Why this beats prompting it yourself

    Generic prompts often leave behind invisible HTML artifacts or break table formatting. This skill uses a systematic DOM decomposition approach to ensure 100% clean Markdown and consistent RAG chunk boundaries every time.

    Use cases

    • Convert messy online documentation into clean context for a coding agent.
    • Prepare a website's content for ingestion into a vector database.
    • Generate a standardized llms.txt file for a new project.
    • Extract structured data from web articles without manual copy-pasting.

    Known limitations

    Does not include built-in proxy rotation for sites with aggressive bot detection. Token estimates are approximations based on standard character-to-token ratios.

    How to use

    Skills on Agensi run inside your AI agent. Copy the prompt, paste it into your agent, and it connects to Agensi and installs this skill for you. If a payment is needed, your agent shows it to you first.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill passes our trust checks below.

    Security scanned

    Passed our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scanned

    Verified clean 1 day ago

    • Passed all security checks, Safe to install

    Requires

    Python
    Ycombinator

    Listed1 day ago

    What's inside

    Frequently Asked Questions