More screenshots

    Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIDeepSeekDeepSeekMuseMuse+22 more

    ML Notebook Audit & Repair

    1

    Finds and fixes data leakage, test-set misuse and unsupported conclusions in ML notebooks, then re-audits until clean.

    $25

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    ML Notebook Audit & Repair

    ML Notebook Audit & Repair

    Example session with this skill installed

    Audit breast_cancer_classifier.ipynb with the ML notebook audit skill. Fix anything severe and tell me if I can trust the conclusion.

    • Read your context and instructions
    • Compiled the ml notebook audit

    Summary

    The notebook claimed "99% accuracy on unseen data". The audit found 3 severe defects, patched them, executed the notebook end to end and re-audited it.

    Status: PASS. The honest result is 97.4%, and the "clearly better" model is not clearly better.

    Severe findings (all patched)

    IDSev.LocationProblemFix
    F-0019cell 5StandardScaler fit on all data before the split: test statistics leak into trainingSplit first; scaler inside a pipeline
    F-00210cells 7, 10C tuned on the test set, then the same test set reports the headline metric5-fold CV on train only; test used once
    F-0039markdownClaims "99% on unseen data", "balanced" (it is 37/63), "clearly beats", "fully reproducible" (no seeds)Seeds, balanced metrics, bootstrap CI, rewritten conclusion

    The original gave 0.947, 0.982 and 0.982 across three runs, and the "best" C changed between runs.

    Corrected results (held-out test, 114 samples)

    ModelAccuracyBalanced acc.F1 malignant
    Logistic regression0.9740.9690.964
    Random forest0.9560.9500.940

    Bootstrap 95% CI for the difference: [-0.027, 0.067]. It includes 0, so there is no clear winner.

    | Random forest | 0.956 | 0.950 | 0.940 |

    Bootstrap 95% CI for the difference: [-0.027, 0.067]. It includes 0, so there is no clear winner.

    lr_search = GridSearchCV(
        make_pipeline(StandardScaler(), LogisticRegression(max_iter=5000)),
        {"logisticregression__C": [0.001, 0.01, 0.1, 1, 10, 100]},
        cv=StratifiedKFold(5, shuffle=True, random_state=SEED),
        scoring="balanced_accuracy",
    ).fit(X_train, y_train)
    

    Final status

    Unresolved findings > 8: 0 · Evidence level 3 (fully executed, reproducible across runs) · PASS

    Next steps

    • Report 0.969 balanced accuracy, not 99%.
    • Present logistic regression as "simpler and at least as good", not "better".
    • Rerun with other seeds if you need a stronger claim.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Most ML notebooks run fine and are still wrong. The scaler was fit before the split, the hyperparameters were tuned on the test set, the plot says "validation" but shows test data, and the conclusion claims more than the numbers support.

    This skill makes your agent audit the notebook as one experiment, not cell by cell.

    How it works

    1. Traces data → split → preprocessing → training → model selection → evaluation → metrics → plots → conclusions.
    2. Rates every finding from 1 to 10 and patches only the severe ones (above 8), with the smallest correct change.
    3. Propagates each fix to metrics, plots, tables, artifacts, markdown and conclusions.
    4. Re-audits the whole notebook after every patch and checks for regressions.
    5. Ends with an honest PASS or FAIL report that states what was actually executed.

    What it catches

    • Data leakage and overlapping train/validation/test sets
    • Model selection on the test set
    • Misleading metrics on imbalanced data
    • Training, checkpoint and train/eval-mode bugs
    • Unfair comparisons between frozen, fine-tuned, PEFT/LoRA and from-scratch models
    • Missing seeds and non-reproducible splits
    • Markdown and conclusions that don't match the results

    What's included

    Full audit checklist, finding schema, patch protocol, report template, and two worked examples (scikit-learn and ResNet18).

    Works best for PyTorch and scikit-learn notebooks.

    Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Agents & Orchestration.

    Automatic patches only apply to findings with severity > 8. Requires Level 4 evidence for full experimental validation. Static analysis only if datasets are missing.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before buying a skill.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 24 days ago

    Software
    Python 3jupyter

    Permissions

    Terminal / Shell
    Read Files
    Write Files

    Reads the notebook, writes the repaired version, and runs Python/Jupyter in the terminal to execute it. No network access, no accounts, no credentials.

    Listed24 days ago

    What's inside

    Frequently Asked Questions

    The skill specifically identifies structural ML flaws like data leakage between splits, improper stratification, and hidden state dependencies that standard linters miss. It ensures your Jupyter notebook is a valid experimental artifact, not just a collection of cells that happen to run.

    To maintain code integrity, the agent only auto-patches 'Critical' issues (severity 8-10) like data leakage. For lower-severity issues, it provides a detailed audit report with recommendations so you can decide how to refactor your code.

    The skill is optimized for Python workflows using PyTorch and scikit-learn, specifically supporting advanced techniques like PEFT and LoRA. It requires access to the notebook file (.ipynb) and ideally the local environment context to perform a full-pipeline trace.

    Yes, the skill performs consistency mapping to ensure that your Markdown descriptions and final visualizations actually align with the results generated by the code. If a patch changes a metric, the agent flags the discrepancy in your documentation.

    Every purchase includes the core auditing and repair agent, a set of pre-configured experimental validation protocols, and access to all minor version updates to support new PyTorch or scikit-learn releases.

    No. It only patches severe defects with minimal changes, never removes experiments, and reports FAIL if problems remain.

    No. Without execution it audits statically and states the evidence level in the report. Execution gives stronger verification.

    Any Python ML notebook. Checks are deepest for PyTorch and scikit-learn, including transfer learning and PEFT/LoRA.

    The repaired notebook plus a report with every severe finding, its root cause, the patch, and a PASS or FAIL status.

    New in Agents & Orchestration