Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIDeepSeekDeepSeekMuseMuse+22 more

    classification model development

    1

    Builds a binary classifier from an approved labelled dataset: leakage and consent screening, holdout precision, recall and calibration, cost-based threshold, group evaluation, model card. Decision support only.

    $2.99

    Secure checkout via Stripe

    Money-back guarantee

    Converts to your local currency at checkout

    Security scanned
    classification model development

    classification model development

    Example session with this skill installed

    Develop a binary classifier for 'loan_default' using E-122 data. Target approved by Sarah Miller. Costs: FP 500, FN 2000. Features in loan_data.csv. Ensure calibration is reported.

    • Read your context and instructions
    • Compiled the classification model development

    The classification model for loan_default has been developed using the deterministic pipeline. All records without explicit consent were excluded prior to training.

    Classification Model Development — loan_default, 2023-10-24

    1. Scope, sources and assumptions

    • Sources: E-122 (Loan application records)
    • Approvals: Target and dataset approved by Sarah Miller.
    • Method: Stratified split, seed 42.

    2. Data, split and class balance

    MetricValue
    Records Supplied10,000
    Records Excluded (No Consent)150
    Training Count6,895
    Holdout Count2,955
    Imbalance Ratio1:8.4

    3. Features and leakage screening

    • income: Numeric, included.
    • credit_score: Numeric, included.
    • current_loan_status: Excluded (Available only after outcome).

    4. Model selection and artifact

    • Selected Model: Gradient Boosted Tree
    • OOF Log Loss: 0.241
    • Training Hash: sha256:7b9a...

    5. Threshold analysis

    • Selected Threshold: 0.32
    • Method: Cost-minimization (FP: 500, FN: 2000)
    • Constraints: Met (Min Recall > 0.70)

    6. Holdout evaluation and calibration

    MetricEstimate95% CI
    Precision0.682[0.651, 0.713]
    Recall0.741[0.710, 0.772]
    ROC AUC0.884[0.865, 0.903]
    ECE0.012N/A

    7. Group evaluation

    Recall gap between region_A and region_B is 0.04. Selection rate ratio is 0.92.

    8. Warnings and items for human review

    • Information gap — organizational validation required: Confirm cost weights represent current fiscal year risk.
    • Analytical estimate — validation required: Threshold selection assumes stable default rates.

    9. Model card summary and approval boundary

    Model exhibits strong calibration (ECE < 0.02). Adopting the 0.32 threshold is a

    Management decision required. This analysis supports, and does not replace, professional judgment and management's decision.

    Next steps

    1. Review the model_artifact.bin for integration testing.
    2. Validate the cost-ratio assumptions with the finance team.
    3. Approve the model card for the internal compliance registry.

    Connects securely to your tools. The creator never sees your data.

    About this skill

    Manual classification modeling is prone to data leakage, arbitrary threshold selection, and poor calibration. This skill automates the development of a binary classifier through a deterministic, reproducible pipeline that enforces rigorous holdout evaluation.

    What it does

    • Leakage screening identifies and excludes features that would not be available at the time of prediction.
    • Automated model selection evaluates logistic or tree-based models using out-of-fold log-loss and Brier scores.
    • Cost-based thresholding selects decision boundaries based on specific costs of false positives versus false negatives.
    • Fairness reporting calculates recall gaps and selection-rate ratios across defined groups to highlight disparities.
    • Model card generation produces a technical summary including training hashes, coefficients, and limitations for audit.

    How it works

    1. The developer provides a labeled dataset, target definition, and explicit approval/consent markers.
    2. The agent builds a request.json configuration, mapping features to their types and availability.
    3. A deterministic execution script (run.py) performs the splitting, training, and evaluation.
    4. The agent interprets the resulting envelope, reporting metrics, calibration curves, and any triggered warnings.

    Frameworks & tools

    The skill utilizes a Python-based execution environment (python3) to run a specialized scripts/run.py utility. It outputs structured Markdown reports based on input.schema.json and output.schema.json definitions.

    Why this beats prompting it yourself

    Generic prompts often lead to "lazy" evaluation where models overfit on the entire dataset or ignore class imbalance. This skill enforces a strict temporal or stratified split and requires explicit cost-benefit inputs for thresholding, ensuring the model solves the actual business problem rather than just maximizing accuracy.

    Use cases

    • Build a churn prediction model with a threshold optimized for the cost of retention offers.
    • Evaluate a fraud detection flag while screening for data leakage and group bias.
    • Generate a reproducible model card and artifact for a risk assessment classifier.
    • Perform a calibration check on an existing classification target to ensure predicted probabilities match real-world outcomes.

    Known limitations

    This skill is for binary classification only and does not support multi-class or regression tasks. It requires pre-approved datasets with explicit consent markers and cannot be used for causal inference or time-series forecasting.

    Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse All Claude Skills.

    Binary classification only. No multi-class or regression. Requires pre-approved data and explicit consent markers. No causal inference or live deployment.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before buying a skill.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    Fresh listing

    Recently published to Agensi

    Money-back guarantee

    Doesn't do what its page promises? You get your money back.

    Trust & safety

    Security scan

    Security scan passed 3 days ago

    Permissions

    Terminal / Shell
    Read Files
    Write Files
    Environment Variables
    Listed3 days ago

    What's inside

    Frequently Asked Questions

    Most LLMs struggle with data leakage and proper evaluation; this skill enforces a deterministic pipeline with strict holdout sets and cost-based thresholding to ensure your model is business-ready, not just statistically accurate.

    The skill requires a labeled dataset and a configuration file (request.json) where you map feature types, availability, and the specific costs associated with false positives and false negatives.

    Currently, this skill is purpose-built for binary classification tasks (e.g., Yes/No outcomes) and does not support multi-class classification or continuous numerical regression.

    You receive the specialized execution scripts (run.py), the input/output schemas for integration, and the logic required to generate automated model cards and fairness reports.

    The skill utilizes a standard Python 3 environment and is compatible with any AI agent capable of executing local scripts and handling JSON-based configurations.

    Yes, the skill includes a fairness reporting module that automatically calculates recall gaps and selection-rate ratios to help you identify and mitigate bias across different demographic groups.