Make Data Imports Safe To Rerun

The problem

Data imports often fail halfway, create duplicates on retry, or lack proper validation. Developers waste time manually cleaning up database corruption or writing custom recovery logic for every new ETL script.

What it does

• Analyzes import scripts to identify duplicate risks and partial-failure vulnerabilities.
• Implements idempotent logic like upserts, unique constraints, and record fingerprinting.
• Adds dry-run modes and validation layers to catch errors before data hits the database.
• Builds checkpointing systems to allow interrupted migrations to resume exactly where they stopped.
• Configures quarantine mechanisms for rejected rows so invalid data doesn't block the entire import.

Frameworks & tools

Works with CSV, JSON, SQL, and API-based data flows. Compatible with any environment where code can be edited or shell commands executed to verify database state.

Why this beats prompting it yourself

Standard LLM prompts often overlook edge cases like transaction boundaries, race conditions, or memory exhaustion in large batches. This skill enforces a strict integrity-first hierarchy, ensuring recovery and deduplication are baked into the architecture rather than added as an afterthought.

Use cases

• Fixing a legacy CSV import script that duplicates users every time it runs.
• Adding a resume feature to a large-scale database migration that keeps timing out.
• Implementing a dry-run flag for a production API sync to verify transformations.
• Building a robust ETL pipeline that isolates malformed records in a quarantine table.

Known limitations

Cannot guarantee idempotency if no stable record identity or business key exists. Requires explicit approval before executing against production systems.