baoyu url to markdown

    1

    Convert dynamic JavaScript-rendered web pages into clean, structured Markdown files with full metadata.

    Free

    Security scanned

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIDeepSeekDeepSeekMuseMuse+22 more

    baoyu url to markdown

    Example session with this skill installed

    Fetch the content from https://example.com/blog/ai-trends and save it as a clean markdown file with metadata.

    • Read your context and instructions
    • Compiled the baoyu url to

    Successfully captured content from example.com. The page was rendered using Chrome CDP and converted to Markdown. The output has been saved to url-to-markdown/example-com/ai-trends-2024.md.

    Captured Content Preview

    ---
    url: https://example.com/blog/ai-trends
    title: Top AI Trends for 2024
    description: An analysis of the shifting landscape in generative AI and LLM orchestration.
    author: Jane Doe
    published: 2024-05-20
    captured_at: 2024-08-25T14:30:00Z
    ---
    
    # Top AI Trends for 2024
    
    The landscape of artificial intelligence is evolving rapidly. This year, we see a significant shift from monolithic models to specialized agentic workflows.
    
    ## The Rise of Agentic Workflows
    Developers are no longer just prompting; they are building systems that can reason and use tools.
    
    ### Key Drivers:
    *   **Reduced Latency**: Smaller, optimized models.
    *   **Better Tooling**: Improved SDKs for browser automation.
    *   **Reliability**: Multi-step verification processes.
    
    > "The future of AI is not just better models, but better systems around those models."
    
    1. Increased focus on RAG stability.
    2. Deployment of local-first LLMs.
    3. Standardization of agent communication protocols.
    

    Next steps

    • Verify the metadata in the YAML front matter.
    • Move the file to your long-term knowledge repository.
    • Run with --wait if the page contains lazy-loaded images that were missed.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Extracting content from modern web pages is often blocked by dynamic JavaScript rendering or authentication walls. Standard scrapers return empty shells or garbled HTML that requires hours of manual cleanup.

    This skill automates the extraction of clean, structured Markdown from any URL using Chrome CDP. It renders full JavaScript components and provides a dedicated manual override for pages behind logins or paywalls. You get a production-ready file with YAML front matter, including original metadata like author and publish date, ready for your local knowledge base or RAG pipeline.

    What it does

    • JS Rendering uses Chrome CDP to execute JavaScript and capture dynamic content.
    • Clean Conversion strips HTML boilerplate and transforms core content into readable Markdown.
    • Metadata Extraction parses titles, authors, and dates into YAML front matter.
    • Authentication Support provides a wait-for-user mode to handle logins and paywalls manually before capture.
    • Automated Filing organizes outputs into domain-specific directories with kebab-case slugs.

    How it works

    1. Run the script with a target URL to initiate the automated Chrome capture process.
    2. Use the --wait flag for complex pages, allowing you to manually confirm the page state before triggering the snapshot.
    3. Review the generated .md file in the structured url-to-markdown/ output directory.

    Frameworks & tools

    This tool runs on Bun and uses Chrome CDP for browser automation. It supports custom configuration via EXTEND.md and environment variables for Chrome profile management.

    Why this beats prompting it yourself

    Writing a custom scraper for dynamic sites requires complex browser orchestration and DOM cleaning logic. This skill handles the Chrome lifecycle, network idle states, and Markdown transformation out of the box, saving you from writing brittle regex or selector-heavy scripts.

    Use cases

    • Archiving technical blog posts for offline documentation.
    • Scraping authenticated dashboard data or private documentation.
    • Building a local knowledge base from research papers and articles.
    • Feeding clean web content into LLM context windows or RAG systems.

    Known limitations

    Requires a local Chrome installation or path defined in environment variables. Complex layouts may require the manual wait mode for accurate capture.

    Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for APIs & Backend.

    Requires local Chrome installation. Extremely complex single-page apps may require manual triggering via the --wait flag. Target directory must be writable.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before buying a skill.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Free to use

    No account required to browse

    Trust & safety

    Security scan

    Security scan passed 23 days ago

    • Free with an Agensi account

    Software
    BunNode.js

    Permissions

    Terminal / Shell
    Write Files
    Network Access
    Environment Variables
    Listed23 days ago

    What's inside

    Frequently Asked Questions

    Unlike basic scrapers, this skill uses Chrome CDP to execute JavaScript, allowing it to capture content from React, Vue, or Angular apps that would otherwise appear as blank pages.

    Yes, by using the `--wait` flag, the skill pauses the automated process, allowing you to manually log in or bypass paywalls before it triggers the Markdown extraction.

    You will receive the complete source code, installation instructions, and documentation for integrating the skill into your local environment or AI agent workflows.

    The skill requires a local installation of the Bun runtime and a Chrome/Chromium browser; you can specify your custom browser path via environment variables if needed.

    The output includes a structured YAML front matter containing the original URL, author, and publish date, followed by clean Markdown optimized for RAG pipelines and LLMs.

    New in APIs & Backend

    Free