Mlops Pipeline Generator

MLOps Pipeline Generator

Your model works in the notebook. Now make it work in prod — with retraining, monitoring, and rollbacks. This skill is the production playbook for the 80% of MLOps that takes 80% of the time.

What it does

Generates a complete MLOps pipeline:

• Training — reproducible, parameterized, versioned data + code
• Evaluation — accuracy, fairness, calibration, regression tests
• Registry — MLflow model registry with stage transitions
• Deployment — canary, blue/green, shadow modes
• Monitoring — data drift, prediction drift, performance decay
• Retraining — schedule-based, drift-triggered, performance-triggered
• Rollback — automatic on performance regression
• Audit trail — every prediction traceable to model + data version

When to use it

• You have a working model in a notebook and need to ship it
• Your model is in prod but you can't reproduce training
• Drift is killing your model performance and you don't know
• You need to retrain regularly but it's a manual mess
• Your team can't deploy ML models without a 2-week process
• Compliance asks "how do you know the model in prod is the one you tested?"

Why it's better than ad-hoc prompting

Most "productionize my ML" prompts give toy examples. This skill is different:

• End-to-end — from training to rollback, not just deployment
• Framework-agnostic — scikit-learn, PyTorch, XGBoost, HuggingFace
• Real orchestration — Airflow / Kubeflow / Prefect, not "just cron it"
• Drift detection done right — KS-test, PSI, embedding distance
• Cost-aware — spot instances, autoscaling, model quantization

Architecture

┌─────────────────────────────────────────────────────────┐
│               Agent (Claude/Cursor)                     │
│  - Reads your notebook / training script                │
│  - Asks about data, framework, deployment target        │
│  - Generates pipeline from template                     │
└───────────────┬─────────────────────────────────────────┘
                │
                ▼
┌─────────────────────────────────────────────────────────┐
│        skills/mlops-pipeline-generator/                 │
│  scripts/                                                │
│    ├── gen_pipeline.py    # Main pipeline generator     │
│    ├── gen_dag.py         # Airflow DAG                 │
│    ├── gen_evaluate.py    # Eval + fairness tests       │
│    ├── gen_drift.py       # Drift detection             │
│    ├── gen_retrain.py     # Auto-retrain triggers       │
│    ├── gen_deploy.py      # Canary/blue-green           │
│    ├── gen_rollback.py    # Auto-rollback               │
│    └── gen_audit.py       # Audit trail + lineage       │
│  references/                                             │
│    ├── architecture.md     # End-to-end diagram          │
│    ├── drift-guide.md      # Data + prediction drift    │
│    ├── deployment-modes.md # Canary vs blue-green       │
│    ├── cost-optimization.md                             │
│    └── compliance-evidence.md                            │
│  templates/                                              │
│    ├── airflow-dag.py                                │
│    ├── mlflow-train.py                                │
│    ├── seldon-deploy.yaml                             │
│    ├── drift-monitor.py                               │
│    └── retrain-trigger.py                             │
└─────────────────────────────────────────────────────────┘

Quick start

# 1. Install
pip install mlflow scikit-learn evidently pandas

# 2. Generate the full pipeline
python scripts/gen_pipeline.py --framework pytorch --data csv --target col_y --out pipeline/

# 3. Generate just the Airflow DAG
python scripts/gen_dag.py --schedule "0 6 * * *" --tasks train,evaluate,deploy --out dag.py

# 4. Generate drift detection
python scripts/gen_drift.py --reference data/train.csv --features col1,col2,col3 --out drift.py

# 5. Generate retrain trigger
python scripts/gen_retrain.py --trigger drift --threshold 0.1 --out retrain.py

# 6. Generate canary deployment
python scripts/gen_deploy.py --mode canary --traffic-split 10/90 --out deploy.yaml

The 6 pipeline stages

┌─────────────┐   ┌──────────────┐   ┌─────────────┐
│ 1. Extract  │──▶│ 2. Validate  │──▶│ 3. Train    │
│   data      │   │   schema     │   │   model     │
│   versioned │   │   quality    │   │   versioned │
└─────────────┘   └──────────────┘   └─────────────┘
                                              │
                                              ▼
┌─────────────┐   ┌──────────────┐   ┌─────────────┐
│ 6. Monitor  │◀──│ 5. Deploy    │◀──│ 4. Evaluate │
│  + retrain  │   │  canary/blue │   │  + register │
│  triggers   │   │  green       │   │  in MLflow  │
└─────────────┘   └──────────────┘   └─────────────┘

Stage 1: Extract

• Pull from data warehouse (Snowflake, BigQuery, Postgres)
• Version the dataset (DVC, lakeFS, delta lake)
• Snapshot to object storage with content hash

Stage 2: Validate

• Schema match (Great Expectations, Pandera)
• Data quality (nulls, outliers, drift vs reference)
• PII detection (if applicable)
• BLOCK on quality failure

Stage 3: Train

• Parameterized (config + CLI args)
• Reproducible (pinned dependencies, random seed, data version)
• Distributed (Dask, Ray, Horovod for big models)
• Track with MLflow (params, metrics, artifacts)

Stage 4: Evaluate

• Holdout accuracy + regression tests
• Fairness across protected groups
• Calibration (if classification)
• Performance vs previous model (BLOCK if worse)
• SHAP / feature importance

Stage 5: Deploy

• Canary (10% traffic) → shadow (no traffic) → blue-green
• Health check (latency, error rate, prediction distribution)
• Auto-rollback if performance drops
• A/B test for business metrics

Stage 6: Monitor + retrain

• Data drift (KS-test, PSI, embedding distance)
• Prediction drift (output distribution shift)
• Performance decay (need labels — delayed feedback)
• Auto-retrain triggers (drift, schedule, performance)
• Audit trail (every prediction → model version → data version)

Drift detection methods

| Method | Best for | How it works | |--------|----------|--------------| | KS test | Continuous features | Compare distributions of new vs reference | | PSI | Categorical + continuous | Population Stability Index, threshold 0.1-0.25 | | Chi-square | Categorical | Compare category frequencies | | Embedding distance | Text / image | Cosine similarity of embeddings | | Prediction drift | Any | Output distribution shifts | | Performance decay | With labels | Accuracy drop over time window |

Pricing

Single-purchase, lifetime access. $18.00.

Includes:

• 7 Python pipeline scripts
• 5 reference docs (architecture, drift, deployment, cost, compliance)
• 5 production templates (Airflow, MLflow, Seldon, drift, retrain)
• Framework support: scikit-learn, PyTorch, XGBoost, HuggingFace
• Future updates for the same major version

Example usage

"I have a scikit-learn model in a notebook. It's a fraud classifier. Generate the full MLOps pipeline."

The skill will:

1. Extract the training code from the notebook
2. Generate parameterized version
3. Create Airflow DAG with 6 stages
4. Generate MLflow training script
5. Generate evaluation script (with fairness checks)
6. Generate Seldon deployment YAML (canary)
7. Generate drift monitoring (Evidently)
8. Generate retrain trigger (drift + performance)
9. Output mlops-pipeline/ ready to deploy

Compatibility

Works with any agent that supports the SKILL.md standard and can execute Python: Claude Code, OpenClaw, Codex CLI, Cursor, Gemini CLI, Cline, Windsurf, Aider. Frameworks: scikit-learn, PyTorch, XGBoost, LightGBM, HuggingFace, TensorFlow. Orchestrators: Airflow, Kubeflow, Prefect, Dagster. Deployment: Seldon, KServe, BentoML, Ray Serve. Tested on Linux, macOS, Windows.

Tags

mlops, ml, machine-learning, pytorch, sklearn, mlflow, airflow, deployment, monitoring