More screenshots
Works with the AI Agents you already use
Model Output Drift Monitor
Detect, measure, and act on drift in model outputs before users notice it.
$5
Model Output Drift Monitor
Example session with this skill installed
Apply the skill to this case: A support-answer model began giving shorter answers. Complaints rose slowly over two weeks, but no single answer looked obviously wrong.
- Read your context and instructions
- Compiled the model output drift
Situation: A support-answer model began giving shorter answers. Complaints rose slowly over two weeks, but no single answer looked obviously wrong. Evidence: Baseline mean length was 142 words with a 6 percent refusal rate. Current mean length was 98 words with a 14 percent refusal rate, concentrated in billing questions. No model version change had occurred, but a retrieval source had been replaced nine days earlier. Decision: The drift was traced to the new retrieval source returning shorter passages, not to the model. The team reverted that source for billing questions and kept the new source elsewhere. Outcome: Mean length recovered to 138 words and refusal rate fell to 7 percent within three days. The monitor now runs weekly, with the billing segment flagged separately.
Connects securely to your tools. The creator never sees your data.
About this skill
Model outputs rarely fail loudly. They slide: more refusals, shorter answers, shifted labels, rising toxicity flags, or a slow change in score distribution. Teams notice weeks later, after bad outputs reached users, and then argue about whether anything really changed. This skill turns that argument into a measured comparison between a frozen baseline and current behaviour, with a decision about what to do next. This skill gives you a counted diagnosis, a decision table, a fill-in template, a checklist, and a scoring rubric, plus a runnable script (drift_report.py) that Reads baseline and current CSV files with a score column and writes a markdown drift report with means, spread, shift, and a flag at a configurable threshold. It is built for the bottleneck moment: when the usual quick fix has already failed, or when a silent failure is worse than a loud one. Follow the protocol in order, record numbers rather than impressions, and use the quality gate before you call the work done.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse All Claude Skills.
Results depend on the quality of your evidence set: a missing baseline, a small or unrepresentative sample, or outputs that were never labelled will limit the diagnosis, and the skill says so rather than filling gaps with guesses. The script is a helper for counting and reporting; it does not replace judgement about safety, permission, or whether a fix is correct. Thresholds are starting points to adapt to your context, not universal constants.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 23 days ago