- Home
- Skills
- Agents & Orchestration
- data leakage detection lab
data leakage detection lab
Inspects ML pipelines to identify and fix data leakage, contaminated features, and accidental hindsight.
$5
Works with the AI Agents you already use
data leakage detection lab
Example session with this skill installed
Review this pipeline. I am using StandardScaler.fit(X) then train_test_split. My features include a rolling 30-day mean of the target variable. Is there leakage?
- Read your context and instructions
- Compiled the data leakage detection
Leakage detected. 1) Preprocessing Leak: StandardScaler must be fit only on X_train to avoid test set statistics influencing training. 2) Future Leak: Rolling target means must be lagged or computed cumulatively to ensure future outcomes do not inform past predictions.
Connects securely to your tools. The creator never sees your data.
About this skill
The problem
Machine learning models often report suspiciously high performance that collapses in production. This usually stems from data leakage, where information from the future or the test set accidentally contaminates the training process.
What it does
- Identifies future information leakage by auditing feature timestamps against label timestamps.
- Detects preprocessing contamination where scalers, encoders, or imputers are fit on the full dataset before splitting.
- Spots sample and group leakage where duplicate entities or near-duplicate records straddle the train-test boundary.
- Exposes accidental hindsight by flagging features derived from revised, updated, or backfilled data.
- Performs adversarial validation to determine if training and test sets are statistically distinguishable.
Frameworks & tools
Designed for Python-based data science stacks including scikit-learn, pandas, NumPy, and XGBoost/LightGBM. Works with Jupyter notebooks and automated ML pipelines.
Why this beats prompting it yourself
Standard LLMs often miss subtle leakage patterns like target encoding inside cross-validation or group-based dependencies. This skill uses specific diagnostic playbooks and adversarial tests to catch technical debt that manual reviews overlook.
Use cases
- Audit a Jupyter notebook before moving a model to a production environment.
- Debug why a model achieved 0.99 AUC on real-world noisy data.
- Verify that a temporal split correctly handles rolling windows and lag features.
- Ensure multi-user datasets don't leak information via group identity.
Known limitations
Requires access to the data pipeline code and feature engineering logic. It cannot detect leakage in pre-compiled black-box models without seeing the training workflow.
Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Agents & Orchestration.
Requires access to data pipeline code and feature engineering logic. Cannot audit pre-compiled black-box models without training workflow visibility.
How to use
- 1
Connect your agent to the Agensi MCP
Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup
- 2
Sign in to Agensi
Your agent shows a sign-in link. Sign in and click Allow.
- 3
Ask for what you need
Your agent loads this skill and uses it on your task. It always asks before buying a skill.
Reviews
No reviews yet
Be one of the first to try it. Every listed skill goes through the trust checks below.
Security scanned
Checked by our 8-point scan before listing
Fresh listing
Recently published to Agensi
Money-back guarantee
Doesn't do what its page promises? You get your money back.
Trust & safety
Security scan
Security scan passed 3 months ago
Permissions
No special permissions declared or detected