What it does

The Databricks Auditor for AWS identifies hidden waste in your Databricks environment by running a deterministic rule engine against your cluster configurations, job schedules, Delta table metadata, and PySpark code. It moves beyond generic advice by calculating dollar-denominated savings estimates based on 2026 DBU list pricing and AWS EC2 rates.

Why use this skill

Unlike general AI prompting, this skill uses a specialized rule engine and inlined pricing data to provide quantifiable ROI. It distinguishes between DBU costs and EC2 infrastructure costs—a critical nuance often missed. It identifies high-leverage arbitrages like converting All-Purpose clusters to Jobs Compute (73% savings) and right-sizing SQL Warehouses.

Key Features

• Compute Right-sizing: Detects misconfigured Spot/Fleet instances, Graviton opportunities, and Photon misuse.
• Delta Lake Optimization: Identifies small file issues, over-partitioning, and missing Z-ORDER/Liquid Clustering.
• Code Anti-pattern Detection: Scans PySpark for expensive collect() calls, Python UDFs that break Photon, and withColumn loops.
• SQL Warehouse Audit: Analyzes auto-stop settings and serverless candidacy for BI workloads.

The Output

You receive a ranked list of findings starting with the biggest cost leaks. Each finding includes the current cost, estimated monthly savings (with the arithmetic shown), a technical "why," and a copy-paste fix snippet or Terraform configuration change.