More screenshots
Works with the AI Agents you already use
Eval Dataset Builder: Labeled Test Set from Your Raw Data
The agent builds an evaluation or fine-tuning dataset for your team to review, with a card that states what is in it, how it was built and what it cannot show.
$7
Eval Dataset Builder: Labeled Test Set from Your Raw Data
Example session with this skill installed
Build an eval set for our ticket router from tickets_export.csv. Queues are Billing, Access, Product bugs, How-to, Feedback, Accounts and General. Mask customer emails and phone numbers, real tickets only, and we are not fine-tuning.
- Read your context and instructions
- Compiled the eval dataset builder
How it was built
| Step | What was done | Records in | Records out |
|---|---|---|---|
| 1 Convert | prep_tools.py convert, input = subject + body, source = ticket id | 96 | 96 |
| 2 Mask | script, then a manual pass with 9 terms (6 names, 1 username, 1 street address, 1 VAT number) | 96 | 96 |
| 3 Screen | non-examples moved to rejects by reading every record | 96 | 90 |
| 4 Deduplicate | dataset_tools.py dedupe, threshold 0.8 | 90 | 78 |
| 5 Label | schema version 2, pilot of 18 records | 78 | 78 |
| 6 Split | eval-only | 78 | 78 |
Check: 96 raw tickets = 78 records from source + 18 rejects.
Masking summary (all 96 candidates): 31 replacements in 24 records: EMAIL 9, PHONE 7, NAME 8, SECRET 3,
NUMBER 2, ADDRESS 1, USERNAME 1. The three [SECRET] values were a password typed into a ticket, a live API
key, and a 32-character request id. The request id is not a secret; it was masked by shape and left masked
because routing does not depend on it.
Connects securely to your tools. The creator never sees your data.
About this skill
English
For ML engineers and product teams who need a test set they can explain, built from data they already have.
Point your agent at raw material: support tickets, chat logs, CSV or JSON exports, a folder of prompts and answers, bug reports. The agent builds an evaluation or fine-tuning dataset for your team to review, with a card that states what is in it, how it was built and what it cannot show.
What it writes each run
dataset.jsonl: one record per line with a stable content-hash id, the masked input, the expected output or a list of true/false grading criteria, labels, a pointer back to the source row, and a split.dataset_card.md: purpose and task, a count after every pipeline step, label definitions, counts per label and split, input lengths, rejects by reason, known gaps and biases, and how to run an eval with the set.rejects.jsonl: every dropped item, whole, with its reason (duplicate of which record, auto-reply, unreadable and so on).labels.jsonandlabel_schema.md: the label values with written definitions, edge-case rules and a tie-break order.
Counted by script, not typed by hand
Two bundled Python scripts (standard library, no network calls) do the mechanical work. prep_tools.py converts flat CSV, JSON arrays and JSONL into candidate records and assigns splits, keeping similar inputs in the same split. dataset_tools.py masks emails, phone numbers, long digit runs, IP addresses and key-shaped strings, removes exact and near duplicates by a similarity threshold, validates the finished file, and prints the tables the card uses.
Only your material
The skill instructs the agent to invent no examples and to write synthetic records only when you ask, flagged "synthetic": true and counted apart. Inputs keep their typos and mess; only personal data is masked. A label that depends on a policy you did not give is marked [NEEDS: ...] with low confidence.
The arithmetic closes
The agent is instructed to check that raw units equal records kept plus rejects and to show the count at each step in the card. The validate command reports errors and warnings on the finished file.
Honest about its limits
Labels and criteria are one pass by the agent with a re-read by the same labeler. The card says so and recommends a human spot check. Masking of names and addresses depends on the agent's read-through. Near-duplicate detection compares wording, not meaning. No eval harness or scoring code is included.
Arabic
On request the card, label schema, criteria and notes are written in Arabic. JSON keys, label values and file names stay in English so tools keep working, and the agent is instructed to leave dialect inputs as they are. Arabic output was tried on one test case and was not reviewed by a second native reader; check it before sharing.
In the download: SKILL.md, the two scripts, four reference files (method, labels and criteria, coverage and splits, Arabic notes), templates, and one complete example: 96 raw support tickets in, 78 labeled records and 18 rejects out, with the pipeline log.
How it was tried: Tried on two cases (an English support ticket routing set, an Arabic pharmacy chat set); the download includes one worked example in English. The bundled scripts have 31 unit tests.
بالعربي
لمهندسي تعلّم الآلة والفرق التي تختبر ميزة مبنية على نموذج لغوي.
وجّه وكيلك إلى موادك الخام: تذاكر دعم، أو سجلات محادثات، أو ملفات CSV وJSON، أو بلاغات أعطال. تستلم مجموعة بيانات للتقييم أو للتدريب، ومعها بطاقة تشرح ما فيها وكيف بُنيت وما لا تستطيع إثباته.
ماذا تستلم
- ملف
dataset.jsonl: سجل في كل سطر بمعرّف ثابت، ونص حُجبت منه البيانات الشخصية، وإجابة متوقعة أو معايير تصحيح، وتصنيفات، وإشارة إلى مصدره. - ملف
dataset_card.md: العدد بعد كل خطوة، وتعريف كل تصنيف، والثغرات المعروفة، وطريقة استعمال المجموعة. - ملف
rejects.jsonl: كل ما استُبعد مع سبب استبعاده.
الأعداد من السكربت
سكربتان بلغة بايثون يعملان على جهازك بلا اتصال بالشبكة: يحجبان البريد وأرقام الهاتف والمفاتيح، ويحذفان المكرر، ويتحققان من الملف النهائي، ويطبعان جداول البطاقة.
حدود واضحة
تنص التعليمات على ألا تُضاف أمثلة مؤلَّفة إلا بطلب صريح، وأن تُعلَّم وتُعدّ منفصلة. والتصنيفات مرور واحد من الوكيل، فهي تحتاج مراجعة بشرية لعيّنة منها قبل الاعتماد على النتائج. وحجب الأسماء والعناوين يعتمد على قراءة الوكيل.
العربية
عند الطلب تُكتب البطاقة ومخطط التصنيف والمعايير بالعربية، وتبقى أسماء الحقول وقيم التصنيفات بالإنجليزية لتعمل الأدوات.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for LLM Ops.
Labels and grading criteria are written by the agent in one pass, with a re-read by the same labeler; no second person checks them, so have a human spot-check before you rely on results. Masking is pattern-based (emails, phone numbers, long digit runs, IP addresses, key-shaped strings); names and street addresses depend on the agent's read-through and your terms list, and the key pattern also masks harmless long ids. Near-duplicate detection compares characters, not meaning: rephrasings in other words, and Arabic written in Latin letters, are not linked. The comparison is pairwise and suits up to a few thousand records per run. convert reads flat CSV, JSON arrays and JSONL; nested exports are flattened by the agent. Train/dev/test splitting with stratification was exercised by unit tests only, since both worked test cases were eval-only. No eval harness or scoring code is included, and nothing is claimed about what a model will score. Arabic cards were not reviewed by a second native reader. التصنيفات مرور واحد من الوكيل وتحتاج مراجعة بشرية.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 3 days ago
- Software
python3 (3.10 or newer)
Permissions
Reads the raw files you point it at; the skill instructs the agent not to edit them. Writes dataset.jsonl, dataset_card.md, rejects.jsonl, labels.json, label_schema.md and a work folder of intermediate files into a build folder in your project (default dataset_build). Runs its bundled Python scripts locally (dataset_tools.py, prep_tools.py); they use the standard library only and make no network calls. The work folder holds unmasked intermediate records and a names list, so keep it private. يقرأ ملفاتك الخام دون تعديلها ويكتب داخل مجلد البناء فقط.