prepro auto

Dirty, unoptimized data is the primary bottleneck for supervised learning. Manually writing scripts for cleaning, scaling, and encoding is repetitive and error-prone. PrePro Auto provides a structured agentic workflow to profile local datasets, apply verifiable cleaning steps, and compare model performance without manual boilerplate.

What it does

• Data Profiling returns dataset shape, missingness patterns, and actual-row evidence for inspection.
• Verifiable Cleaning generates proposed cards for outlier removal, scaling, and imputation before execution.
• Deterministic Transforms applies exact row/column rules using expression previews to validate logic against boundary cases.
• Model Comparison runs cross-validation across multiple supervised candidates to identify the champion model.
• Managed Exports handles dataset revisions and model artifacts in a local workspace with undoable edits.

How it works

1. Bootstrap the environment by piping the agent script into Python to initialize the workspace and install dependencies.
2. Upload a local CSV, Excel, Parquet, or JSON file to create a tracked job and background server instance.
3. Analyze the dataset using inspect to review cards and recommendations for cleaning or encoding.
4. Execute approved proposals to commit changes, update the dataset revision, and generate training-ready artifacts.

Frameworks & tools

This skill utilizes a self-contained Python helper (3.10+) to interface with the MIT-licensed PrePro Auto package. It manages local datasets in CSV, Excel, Parquet, JSON, and Feather formats.

Why this beats prompting it yourself

Generic LLMs often hallucinate data distributions or write cleaning scripts that fail on edge cases. This skill uses a local runtime to provide real-row evidence and stale-check protection, ensuring every transformation is verified against actual data before it is applied.

Use cases

• Automating feature engineering and encoding for tabular ML datasets.
• Comparing XGBoost, Random Forest, and Linear models on a target variable.
• Identifying and fixing data quality issues in large local CSV or Parquet files.
• Exporting cleaned, staged data and trained model artifacts for production.

Known limitations

Requires a local Python 3.10+ environment with process execution and loopback HTTP. Not compatible with remote connectors that lack local filesystem or background process persistence.