- Home
- Skills
- Workflow Automation
- Batch Download: Orchestrated Multi-File Collection
More screenshots
Works with the AI Agents you already use
Batch Download: Orchestrated Multi-File Collection
Orchestrate large multi-file downloads with parallel workers: plan-first decomposition, URL validation, worker contracts, and verification that proves every file arrived intact.
$5
Batch Download: Orchestrated Multi-File Collection
Example session with this skill installed
Collect 120 papers (PDFs) from a conference proceedings index for a literature review. The source list is the index page itself. Prove the collection is complete or say exactly what's missing.
- Read your context and instructions
- Compiled the batch download
- Generated the document
COLLECTION COMPLETE-ISH — fetched 112 of 120, with every gap explained. Reported as partial, honestly.
Phase 1 — decompose: completeness test is all papers listed on the proceedings index page. URLs discovered from the index (each with the index as its source). Partitioned into 4 chunks of 30 by list segment; chunks independent.
Phase 2 — validate: deduplication found 6 URLs that were the same paper linked twice (index + author page) — normalized and deduped to 120. Probing: 3 URLs returned 404, 2 returned text/html where a PDF was expected (login redirects). Work order: 115 URLs, with the 5 flagged listed in the failure log before any worker started.
Phase 3 — workers: four workers, 28–29 URLs each, naming rule (first-author-year-title-slug), verification rubric. Politeness: 1s delay, real User-Agent. One worker hit a 429 on the proceedings domain and backed off for 60 seconds, then resumed — the rate limit was in the plan, so it wasn't a crisis.
Phase 4 — verify: count check — 112 files on disk vs 115 in the work order. The 3 gaps: two files failed twice (server errors, logged with the error), one file's magic bytes showed HTML (an error page, deleted and logged). Type sweep: all 112 are real PDFs. Spot check: 10 random PDFs opened, all have pages and the right titles. Manifest written with all 112 files plus the 8 failures (5 from validation, 3 from fetching) with reasons.
Report: "Fetched 112 of 120. 5 URLs were dead or login-walled before fetching began; 3 failed during fetching. Failure list attached with reasons. Manifest attached." A partial batch presented as complete would have been a lie; a partial batch presented honestly is a usable collection with a known gap.
batch-download-orchestrated-multi-file-c.pdf
PDF · document
Example file from a real run - the skill writes it into your workspace.
Connects securely to your tools. The creator never sees your data.
About this skill
Downloading 500 files is a project, not a command. The failure mode is always the same: start fetching immediately, discover halfway that half the URLs are dead, and end with an unverifiable pile. This skill front-loads the planning so the fetching is mechanical.
Phase 1 decomposes before downloading: define the set and what counts as complete (a count, a source list, a quality bar), discover URLs with evidence (each URL from an actual source — never guessed, never reconstructed from a pattern), and partition into independent chunks so a failed chunk retries alone. Phase 2 validates the URL list before any worker fetches a byte: deduplicate (normalized, redirects resolved once), probe each URL with a HEAD request (status, content type, content length), and check robots.txt and rate limits so the batch doesn't get IP-blocked mid-run. The validated list — URL, expected type, expected size, source — is the work order, and workers fetch from it; they don't discover.
Phase 3 gives each worker a written contract: its chunk, a deterministic naming rule (never "file(1).pdf"), per-file verification (size > 0, magic bytes match the expected type, size roughly matches the probed length), a failure protocol (retry twice with backoff, then log and move on), and politeness (delay per domain, real User-Agent, back off on 429s). Phase 4 is integration and verification: count check (files on disk vs work order, every gap with a reason), type sweep with file, a random spot check with human eyes, and a manifest with source URL, fetched date, byte size, and checksum per file. The standing rule: never present a partial batch as complete — report fetched N of M, with the failure list and reasons.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Workflow Automation.
- It cannot fetch login-walled or paywalled content without authorization — a 403 ends that URL's story.
- A domain that blocks the batch mid-run stops that domain's workers; the rest continue, and the block is recorded.
- Magic-byte checks catch error pages disguised as files, but they can't catch a wrong-but-valid file — that's what the spot check is for.
- A batch bigger than the free disk space fails halfway; the probed total size must be checked against free space first.
- Checksum verification proves the file arrived as fetched, not that the source itself is trustworthy.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 3 days ago
Permissions
No special permissions declared or detected