More screenshots
Works with the AI Agents you already use
NeMo Curator: Build LLM Training Data That Holds Up
Curate LLM training corpora with NeMo Curator — quality, dedup, and safety in the right order, every filter validated on samples before it runs at scale.
$5
NeMo Curator: Build LLM Training Data That Holds Up
Example session with this skill installed
We are fine-tuning on 40M support documents scraped from forums, docs, and tickets. Build a NeMo Curator pipeline that cuts the junk without cutting the good stuff, and prove each filter earns its place.
- Read your context and instructions
- Compiled the nemo curator
Named the risk first: a support corpus fails on boilerplate, SEO spam, and outdated guidance presented as fact, so the filters were chosen for that failure, not from a default list. Exact dedup ran first — nearly free — and cut 40M documents to 31M. Quality heuristics (length bounds, symbol ratios, repetition detectors) were validated on 5,000 stratified samples with 97% removal precision before scaling, then cut the corpus to 22M. Fuzzy dedup at Jaccard 0.8 took it to 17M; semantic dedup was skipped because the fuzzy pass left no evidence of meaningful paraphrase overlap, and the cost was not justified. PII redaction ran last on the final set, with 200 redacted documents reviewed by hand to catch over-redaction of technical identifiers — order numbers that look like SSNs. Benchmark quarantine: n-gram overlap against the eval set came back 0.3%, documented, with the overlapping eval items removed from training. Final corpus 16.4M documents with a curation log holding per-stage counts, thresholds, and sample-validation numbers. The model trained on it beats the uncut-corpus baseline by 6 points on support-bench — and the team can say why, because the log answers what it was fed. The log is what makes the next corpus better instead of just smaller.
Connects securely to your tools. The creator never sees your data.
About this skill
Training data quality is the highest-leverage variable in LLM training and the hardest to inspect after the fact, so this skill is curation judgment, not just tooling: which filters to run, in what order, and how to prove each one helped instead of just shrinking the corpus. It stages the pipeline as quality heuristics, then exact dedup, then fuzzy dedup, then semantic dedup only with evidence, then PII redaction and safety classifiers last — because running classifiers on ten billion tokens of exact duplicates is burning money. Every filter is validated on stratified samples before it touches the full corpus, with removal precision and recall cost recorded in a curation log, and benchmark quarantine keeps eval text out of training data so a benchmark win is a real win.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for LLM Ops.
It teaches curation judgment and the NeMo Curator pipeline, not training itself — you still run the training job and the model.
Filter quality depends on sample validation: a heuristic tuned on the head of the distribution can still damage the long tail if samples are not stratified.
Semantic dedup is the costliest stage; the skill shows when the evidence justifies it and when fuzzy dedup is enough.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 3 days ago
Permissions
No special permissions declared or detected