More screenshots

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIDeepSeekDeepSeekMuseMuse+22 more

    LLM Dataset Annotator

    1

    Your agent labels every row of a CSV, JSONL or Excel file against a written spec, checks every batch, measures agreement, scores against your own labels and hands back a short human review queue.

    $5

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned

    LLM Dataset Annotator

    Example session with this skill installed

    Label every review in reviews.csv with an overall sentiment (Positive, Negative, Mixed, Neutral) and the topics it mentions (Quality, Price, Delivery, Support, Usability). I hand-labeled 20 of them in gold.csv, so check yourself against those first.

    • Read your context and instructions
    • Compiled the llm dataset annotator
    • Generated the data export

    Labeled all 40 reviews against the spec we agreed (a definition for every label, six decision rules for the hard cases, three worked examples). Both batches passed validation. Your file is untouched; the labels are in reviews_annotated.csv.

    Checked against your 20 gold labels

    FieldRowsScore
    sentiment20accuracy 100%, kappa 1.00
    topics (multi-label)20exact-set accuracy 95%, mean Jaccard 0.97

    The one topics miss is

    r28 ("Oh great, another lamp that dies right after the return window closes"): you tagged it Quality + Support, I tagged Quality only. It's a judgment call on whether the return window is a Support complaint; worth a rule either way.

    Label distribution

    sentimentcounttopiccount
    Negative17 (42.5%)Quality19 (47.5%)
    Positive15 (37.5%)Usability12 (30.0%)
    Mixed4 (10.0%)Price7 (17.5%)
    Neutral4 (10.0%)Support6 (15.0%)
    Delivery5 (12.5%)

    Confidence: 33 high, 7 medium, 0 low.

    Review queue: 2 rows (review.csv)

    idreviewmy labelwhy it's flagged
    r21"Assembly instructions were missing a step, but it went together eventually."Neutralminor complaint that resolved itself; Neutral vs Negative is a judgment call
    r26"I ordered white and received black. Exchange was quick though."Positiverule 3 (resolved problem praised) says Positive, but the reviewer gives no overall verdict; could be Mixed

    Output columns added

    sentiment, topics (joined with ; ), annotation_confidence, annotation_agreement, needs_review, annotation_note.

    Next steps

    1. Settle r21 and r26 in review.csv (fill final_sentiment), then I'll re-export.
    2. Decide the r28 rule (is a return-window complaint also Support?) and I'll add it to the spec.
    3. For the full dataset, I'll also run the independent second pass on a 300-row sample to report agreement.

    reviews_annotated.csv

    CSV · data export

    Generated

    Example file from a real run - the skill writes it into your workspace.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Asking an AI agent to "label this CSV" usually gets you one of two things: a quick keyword script pretending to be judgment, or a pile of labels with no way to tell which ones are shaky. This skill makes your agent work like a careful annotation team instead: it writes the labeling rules down with you, labels every row by actually reading it, checks its own work, and tells you exactly which rows still need a human.

    No API key and no extra cost: the agent you already use does the reading. A bundled Python script (standard library only) does the bookkeeping, so no row is skipped, duplicated or given a label that isn't in your spec.

    What it does

    • Writes the label spec with you: the fields, a definition for every label, decision rules for the hard cases and worked examples. Already have a codebook? It converts it.
    • Pilots first on 30-50 rows, or on the rows you labeled yourself, and turns its mistakes into sharper rules before the full run.
    • Labels every row in validated batches. Invented labels, broken rules, missing or duplicated rows and bad output are rejected with the exact problem named, and the agent fixes them before anything counts.
    • Measures consistency by re-labeling a sample in an independent second pass (a fresh subagent where your agent supports one): percent agreement, Cohen's kappa and the label pairs it confuses most.
    • Scores against your gold labels: accuracy, kappa, and precision, recall and F1 per label, with the exact rows to look at.
    • Builds a review queue of disagreements, low-confidence rows and the agent's notes. Labels you correct there flow into the export.
    • Gives your file back intact: label columns plus confidence, agreement and review status appended; every original column, value and row unchanged. It never overwrites the input.

    How it works

    1. Inspect: columns, row count, empty cells, text lengths, duplicates.
    2. Spec: agree the labels with you; the script validates the spec against the data.
    3. Pilot: label a sample or your gold rows, score, refine the rules.
    4. Pass 1: label every row in shuffled batches; each batch is checked on submit. Runs can be paused and resumed.
    5. Pass 2: independent re-label of a sample for agreement.
    6. Report: label distribution, agreement, confusions, review queue.
    7. Review and export: you settle the flagged rows; the annotated file is written.

    Use cases

    • Sentiment, emotion, stance, intent or topic labels for NLP training and evaluation sets.
    • Qualitative coding of open-ended survey answers or interview excerpts against a codebook.
    • Triage of support tickets, feedback, app reviews or moderation queues.
    • Extending a small human-labeled set to the rest of your data, with accuracy measured on the part you labeled.

    Single-label and multi-label fields. CSV (comma, semicolon, tab or pipe, BOM-safe), TSV, JSONL and Excel.

    Known limitations

    • Agreement between two passes of the same model measures consistency, not correctness. For accuracy, label 30-100 rows yourself and use the gold comparison; the report says this too.
    • It is as fast as your agent reads: hundreds of rows are comfortable; for many thousands, expect a long session or label a sample.
    • Labels come from fixed label sets. Span highlighting and free-text answers are out of scope.

    Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Data & Databases.

    Agreement between two passes of the same model measures consistency, not correctness; use your own gold labels for accuracy. Speed depends on your agent: hundreds of rows per session are comfortable, many thousands take a long time. Labels come from fixed label sets; span highlighting and free-text answers are not supported. Excel files need openpyxl.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before buying a skill.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 5 days ago

    Software
    Python 3.8+
    Optional
    openpyxl (only for Excel files)

    No API key needed: your own agent does the labeling. For Excel files run `pip install openpyxl`, or save the sheet as CSV.

    Permissions

    Terminal / Shell
    Read Files
    Write Files

    Runs one bundled Python script locally (standard library only) to read your data file and write the labeled copy next to it. It makes no network calls, reads no environment variables and never modifies the input file.

    Listed5 days ago

    What's inside

    Frequently Asked Questions

    No. The agent you already use (Claude Code, for example) reads the rows and decides the labels, so labeling is part of your normal agent session. The bundled script uses only the Python standard library and makes no network calls.

    Asked directly, agents tend to write a keyword script, skip or duplicate rows on long files, invent labels outside your set, and give no sign of which labels are shaky. This skill forbids labeling with code, checks every batch against your spec before it counts (so no row is lost or given an invented category), measures consistency with an independent second pass, scores itself against any rows you labeled, and gives you a short review queue instead of a file you have to re-check line by line.

    Three ways. If you hand-label 30-100 rows, the skill scores itself against them: accuracy, Cohen's kappa and precision/recall/F1 per label, with the rows it got wrong. A second, independent pass on a sample reports agreement and the label pairs it confuses, which shows where your rules are vague. And every row where the passes disagree, the agent was unsure, or it left a note goes into a review file for you. Agreement alone measures consistency, not correctness, and the report says so.

    CSV (comma, semicolon, tab or pipe, with or without BOM), TSV, JSONL and Excel (.xlsx, needs openpyxl). Any number of label fields, each single-label or multi-label, with labels that must be used alone (like "None"). The output keeps your file's format, appends the label columns plus confidence, agreement and review status, and leaves every original column and row exactly as it was.

    It works at the speed your agent reads, in batches of about 40 short rows (fewer for long texts). Hundreds of rows are comfortable in one session, and a run can be paused and resumed at any time without losing work. For several thousand rows the agent tells you up front how long it will take, can split batches across subagents where supported, and offers to label a sample instead.

    New in Data & Databases