market data quality inspector

The problem

Bad market data ruins backtests and triggers erroneous trades. Manually hunting for missing bars, stale quotes, or unadjusted price gaps in large datasets is tedious and error-prone.

What it does

• Identifies missing intraday or daily bars by comparing data against official exchange trading calendars.
• Detects stale quotes and frozen prices where volume is zero or price remains static across multiple bars.
• Validates OHLC relationships, ensuring highs are greater than lows and prices are non-negative.
• Flags potential corporate action inconsistencies like unadjusted splits or dividend gaps.
• Cross-references multiple data feeds to find conflicting price levels or trade timestamps.

Frameworks & tools

Compatible with CSV, Parquet, and JSON formats. Handles Pandas DataFrames and structured market data feeds for XNAS, XNYS, and other major exchanges.

Why this beats prompting it yourself

General LLMs often overlook exchange-specific nuances like early closes, holiday calendars, and timezone-aware timestamp shifts. This skill uses a structured inspection framework to ensure mathematical consistency and calendar alignment that simple prompting misses.

Use cases

• Sanitizing historical OHLCV data before running a backtest.
• Verifying live feed integrity before an automated strategy goes live.
• Auditing data vendor quality by comparing overlapping feeds for inconsistencies.
• Detecting symbol mismatches and casing errors across heterogeneous data sources.

Known limitations

Identifies issues but does not automatically repair data unless requested. Statistical outlier detection may flag genuine high-volatility events as warnings.