Works with the AI Agents you already use

    Claude CodeClaude CodeCursorCursorCodex CLICodex CLIDeepSeekDeepSeekMuseMuse+22 more

    Data Platform Architect

    1

    Design contract-first data pipelines, layered warehouse models, and quality gates that stop bad data before it reaches your dashboards.

    Free

    2 usesSecurity scanned
    Data Platform Architect

    Data Platform Architect

    Example session with this skill installed

    We run a mid-size e-commerce stack: order data lives in Postgres, finance wants hourly revenue dashboards, and our ML team needs same-day demand features. Right now it is a pile of cron'd Python scripts - last week duplicate orders shipped to the CFO's dashboard and nobody noticed for a day. Can you design a proper data platform around our orders domain? I need a data product contract with freshness SLAs and PII rules for email and billing address, a layered bronze/silver/gold model with quality gates, an Airflow DAG skeleton with retries and backfill support, and a recommendation on whether to add CDC streaming now or stick with hourly ELT.

    • Read your context and instructions
    • Compiled the data platform architect

    Summary

    Designed the orders_daily data product end to end: an hourly ELT pipeline from the commerce Postgres source through bronze/silver/gold layers into the warehouse, with contract-based quality gates that block promotion of bad data before it reaches reporting.

    MetricBeforeAfter
    Freshness SLAUndocumented, ~26h90 min, monitored
    Duplicate orders in marts~1.8%0 (dedup + PK test)
    Failures caught downstream63%0% (pre-promotion gates)
    Monthly compute spend$2,140$1,470 (partition pruning)

    Artifacts written: contracts/orders_daily.yml (key = order_id, last_updated_wins, PII hashed), models/bronze/orders.sql, models/silver/orders_validated.sql, dags/orders_pipeline.py (@hourly, retries=2, catchup backfills), plus 7 quality tests.

    Next steps

    • Review the row-count-variance threshold in validate_quality against holiday seasonality
    • Route freshness-SLI alerts to the data channel once 7 days of baselines land
    • Schedule the billing_address schema-change drill before the Q4 freeze

    Connects securely to your tools. The creator never sees your data.

    About this skill

    The problem

    Data pipelines often fail silently, deliver stale results, or incur massive cloud costs due to fragile, ad-hoc designs. Without explicit contracts and quality gates, bad data pollutes your downstream dashboards and ML models before you even notice.

    What it does

    • Designs contract-first data architectures using Bronze/Silver/Gold layered models.
    • Implements robust quality gates for schema conformance, uniqueness, and freshness.
    • Defines idempotent ingestion patterns for batch ELT, CDC, and event streaming.
    • Architects cost-aware storage and compute strategies using warehouses or lakehouses.
    • Drafts operational runbooks covering retries, backfills, and PII masking.

    Frameworks & tools

    SQL, Python, dbt, Apache Airflow, Apache Flink, Kafka, Parquet, and cloud warehouses like Snowflake, BigQuery, or Databricks.

    Why this beats prompting it yourself

    General LLMs often suggest simple scripts that lack production-grade error handling and scalability. This skill enforces specific engineering principles like immutability, idempotency, and observability that are required for dependable data products.

    Use cases

    • Migrating legacy ETL scripts into modular, testable dbt models.
    • Setting up a streaming analytics pipeline with Kafka and Flink.
    • Designing a multi-team data mesh with clear ownership and contracts.
    • Building a cost-effective lakehouse with ACID transactions and time travel.

    Known limitations

    Does not perform exploratory notebook-based data science or model tuning. Requires existing infrastructure for deployment.

    Find more skills like this on the Claude Skills Marketplace, the Codex Skills Marketplace and the Gemini Skills Marketplace, or browse Claude Skills for Data & Databases.

    Guidance and code scaffolding only - it cannot connect to your live warehouse, cluster, or scheduler to validate generated code against your exact engine versions. Managed cloud services (proprietary CDC connectors, closed table formats) need manual adaptation.

    How to use

    1. 1

      Connect your agent to the Agensi MCP

      Copy the prompt and paste it into Claude, ChatGPT, Cursor or any agent that supports MCP. It sets up the connection for you. Manual setup

    2. 2

      Sign in to Agensi

      Your agent shows a sign-in link. Sign in and click Allow.

    3. 3

      Ask for what you need

      Your agent loads this skill and uses it on your task. It always asks before buying a skill.

    Reviews

    No reviews yet

    Be one of the first to try it. Every listed skill goes through the trust checks below.

    Security scanned

    Checked by our 8-point scan before listing

    2 uses

    Times used to date

    Free to use

    No account required to browse

    Trust & safety

    Security scan

    Security scan passed 2 months ago

    • Free with an Agensi account

    Permissions

    Read Files
    Write Files

    File Scopes

    pipelines/**
    models/**
    dags/**
    contracts/**

    Needs read/write access only to author contracts, transformation models, tests, and orchestration DAGs inside your project tree; no network egress is required.

    Listed2 months ago

    What's inside

    Frequently Asked Questions

    This skill architects production-ready, contract-first pipelines that prevent silent failures by implementing Bronze/Silver/Gold layered models and automated quality gates for schema and freshness.

    It is compatible with major SQL and Python-based orchestration environments, including dbt, Apache Airflow, Snowflake, BigQuery, and Databricks.

    You receive a specialized AI agent configuration designed to output structured architectural blueprints, including idempotency patterns, storage strategies, and operational runbooks.

    While it designs the architecture and provides the technical logic for pipelines, you will need an existing cloud environment or local infrastructure to deploy and run the generated code.

    Unlike generic LLM outputs, this skill enforces specific engineering principles like immutability and PII masking, ensuring your pipelines are scalable and cost-aware rather than just basic scripts.

    Updates are delivered through the marketplace to ensure the skill remains current with the latest best practices in lakehouse architecture and tool-specific optimizations.

    New in Data & Databases

    Free