- Home
- Skills
- Prompt & Skill Engineering
- Blind Test
More screenshots
Works with the AI Agents you already use
Blind Test
Prove that a skill, prompt, rule file or MCP server changes what your agent does: a hidden key written first, traps for false alarms, runs with and without it, graded blind
$9
Blind Test
Example session with this skill installed
I'm about to publish my UX-writing skill. Build a blind test for its main claims: the brand's voice as rules, slop-free copy, whole flows written state by state. Run it on Haiku with and without the skill, two runs each
- Read your context and instructions
- Compiled the blind test
The key had 11 discriminating problems, 3 traps and one folklore line ("make it fun and energetic, with emojis — that's our brand"), written before the runs. Graded blind, then scored with score.mjs:
| Arm | Discriminating (of 11) | Traps flagged (of 3) | Folklore |
|---|---|---|---|
| Without the skill | 6.5–7.5 | 0–1 | repeated |
| With the skill | 10 | 0 | corrected |
The report also listed what the round exposed in the skill itself: a script that suggested the wrong form of address, a check blind to Cyrillic, buttons it couldn't recognise. Each was fixed as a class before release.
Blind Test was also checked with its own method: two runs of Haiku preparing a test with Blind Test covered all 13 items of the key and all 7 discriminating ones; two runs without it covered 5–6 plus partials and none of the discriminating ones. Limits, for both: one target, one model, keys written by the skill's author.
Connects securely to your tools. The creator never sees your data.
About this skill
You wrote a skill, a system prompt or a CLAUDE.md, ran it once and liked the output. Would the agent have done the same without it? Did it find more real problems, or did it simply flag more things? Asking a model to judge two outputs puts its taste in place of an answer.
Blind Test checks it the way a careful reviewer would. Your agent builds a fictional target with problems planted on purpose, writes the key before the first run, runs the same request with and without the change in isolated agents, and grades every output against the key without knowing which arm wrote it.
What it can test. Anything you can switch on in one arm and off in the other, with the target, the request and the rest of the setup the same:
- Skills, before you publish them, and a new version against the old one on the same key.
- System prompts and rule files: CLAUDE.md, AGENTS.md, Cursor rules.
- Plugins and MCP servers: whether a connected tool improves the result or only adds steps.
- Context: a knowledge file, a design system export, a set of examples.
- Models: two models on the same instructions, everything else fixed.
What a fair test needs, built in.
- A hidden key, written first. Planted problems tied to the claims being tested, each with where it is, how to confirm it and what an output must say to count. The decision rule is written before any run.
- Traps. Legitimate decisions that look like mistakes: a 20 px button that passes WCAG with spacing, a translation that's long and wraps freely. A skill or a prompt that makes the agent flag everything pays for each one.
- Folklore in the request. A common wrong belief, phrased the way a stakeholder would say it. The test shows whether the agent follows the request or the rule.
- Isolated runs. The same request, word for word, in every arm, each agent in its own folder with the target and the request only. Optionally a competitor skill in a sandbox.
- Blind grading. Outputs are relabelled A, B, C before grading; the arms are revealed after the grades are written.
- Side effects logged. The target server records every request that would change something, so a review that submits a form shows up.
Review and build tasks. Review tests plant problems in something that exists. Build tests give a brief and check the result against data at ten times the size, empty states and awkward names.
An honest report. The table per arm, what only the skill found, what it missed, the fixes made as classes, a rerun, and the limits written next to every number.
Blind Test came out of testing my own skills before release. Design Taste Pro was checked this way on a fictional fleet dashboard: 15–16 of 16 planted problems with the skill against 10 and two partial without, and no trap flagged by the runs with the skill.
- The agent writes the key, so the key carries its author's blind spots. Have someone else read it before the runs when the numbers go public.
- Two runs per arm show whether a result is stable; a number for a listing wants three or more, and one more target the skill was never tuned on.
- It tests one model at a time. A skill that helps a small model a lot may help a large one less.
- Grading is done by an agent or a person following the key. It's blind, and the evidence line for each grade is kept, but it's still a reading.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before any payment.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed today
Requires
Permissions
File Scopes
`target-server.mjs` serves the target folder on 127.0.0.1 and nothing outside it; it answers anything except GET and HEAD with 405 and logs it, and never serves the key, the log or dotfiles. `blind-labels.mjs` and `score.mjs` read and write files in the test folder. The scripts make no network requests beyond the local server and read no environment variables.