Api Compatibility Checker

Api Compatibility Checker

# API Compatibility Checker

Shipping an API update that breaks your consumers is a trust-killer. Every time you add a field, change a response, or rename a parameter, there is a risk that existing clients will fail silently or crash outright. Manual review of API diffs is error-prone, slow, and does not scale across microservice architectures. Most teams discover breaking changes only after deployment, when customer integration tests start failing and support tickets pile up. Existing tools like oasdiff are powerful but sit outside the agent workflow. You have to switch context, run a separate CLI command, and manually interpret the diff. This skill brings breaking change detection directly into your AI agent's workflow, so you can verify compatibility during code review, before a PR merges, or as part of a CI pipeline.

What It Does

• Compares two OpenAPI 3.0/3.1 specs (JSON or YAML) and produces a detailed diff report
• Classifies every change as breaking, warning, or safe/additive
• Detects removed endpoints, removed HTTP methods, added required parameters, schema type changes, removed properties, enum value changes, constraint changes, request body changes, and response schema changes
• Generates reports in Markdown (human-readable) or JSON (machine-readable for CI)
• Supports --fail-on-breaking flag for CI/CD pipelines (exit code 1 if breaking changes found)
• Includes sample specs for immediate testing

Frameworks/Standards Covered

| Standard/Framework | Support | |---|---| | OpenAPI 3.0.3 | Full spec comparison | | OpenAPI 3.1.0 | Full spec comparison | | JSON Schema | Deep property comparison, type checking, constraint validation | | Swagger 2.0 | Partial (YAML/JSON loading supported, but some structures differ) | | REST API Versioning | URL-based and header-based versioning patterns documented | | Semantic Versioning | Breaking change detection maps to MAJOR version bumps | | CI/CD Integration | Jenkins, GitHub Actions, GitLab CI, CircleCI |

Detailed Feature Breakdown

### Endpoint Change Detection The tool identifies every path and HTTP method in both specs. It flags removed endpoints (breaking), added endpoints (safe), and modified endpoints. For modified endpoints, it drills into parameters, request bodies, and responses. ### Parameter Analysis For each operation, the tool compares all parameters across four locations: path, query, header, and cookie. It detects: - Removed parameters (breaking: existing clients will send invalid data) - Added required parameters (breaking: existing clients will fail) - Added optional parameters (safe: backward compatible) - Parameters that became required (breaking) - Parameters that became optional (safe) - Schema changes within parameters (type changes, constraint changes) ### Schema Deep Comparison The recursive schema comparator walks the full property tree of both objects. It detects: - Removed properties (breaking: clients reading those fields get undefined) - Added required fields (breaking: new responses will fail validation) - Type changes (breaking: parsers expecting string will fail on integer) - Enum value removals (breaking: exhaustive switch statements break) - Enum value additions (warning: may break exhaustive clients) - Format, pattern, min/max, minLength/maxLength, minItems/maxItems changes ### Response Body Analysis Compares every response status code and media type. Removed responses are breaking (clients expecting 200 will fail on 404). Common response schemas are compared recursively. ### Request Body Analysis Detects added/removed request bodies, required/optional transitions, and schema changes within body content types. ### CI/CD Integration The --fail-on-breaking flag causes the tool to exit with code 1 when any breaking changes are detected. This makes it suitable for use in GitHub Actions, GitLab CI, Jenkins, and other CI pipelines. The JSON output format allows programmatic parsing of results.

Usage

### Quick Start ```bash # Compare two OpenAPI specs python3 scripts/cli.py samples/petstore-v1.json samples/petstore-v2.json # Generate JSON report for CI python3 scripts/cli.py samples/petstore-v1.json samples/petstore-v2.json --format json --output reports/breaking-report.json # Fail CI pipeline if breaking changes found python3 scripts/cli.py base.yaml revision.yaml --fail-on-breaking ``` ### Configuration The config/config.yaml file controls default behavior: ```yaml api_compatibility_checker: default_format: markdown # Default output format output_dir: ./reports # Default output directory fail_on_breaking: false # Exit with code 1 on breaking changes max_recursion_depth: 10 # Max schema recursion depth ```

Output Format

### Markdown Report ``` # API Compatibility Report Base spec: `samples/petstore-v1.json` Revision spec: `samples/petstore-v2.json`

Summary

FAIL: 3 breaking change(s) detected. | Metric | Count | |---|---| | Total changes | 7 | | Breaking changes | 3 | | Warnings | 2 | | Safe changes | 2 | ``` ### JSON Report ```json { "summary": { "base_spec": "samples/petstore-v1.json", "revision_spec": "samples/petstore-v2.json", "total_changes": 7, "breaking_changes": 3, "compatible": false }, "changes": [ { "type": "endpoint_removed", "severity": "breaking", "path": "GET /pets", "detail": "Endpoint removed" } ] } ```

Why This Beats Prompting It Yourself

Prompting an AI This Skill --- --- --- Accuracy Misses subtle breaking changes like enum removals or constraint changes Systematic recursive comparison catches everything Consistency Varies by model and context Deterministic, same results every time Schema depth Shallow comparison, misses nested type changes Full recursive comparison with $ref resolution CI integration Cannot be automated --fail-on-breaking flag for CI pipelines Speed Takes multiple prompt rounds Single command, instant results Documentation No report generation Markdown and JSON reports Reproducibility Depends on model state Fully deterministic

Use Cases

• CI/CD Pipeline Gate: Run the compatibility checker as a GitHub Action step before every API deployment. Block the build if breaking changes are detected without explicit approval.
• Pull Request Review: Automatically compare the proposed OpenAPI spec against the current production spec. Post a compatibility report as a PR comment so reviewers can see exactly what changed.
• API Version Planning: Before creating a new API version, diff the proposed spec against the current version. Use the output to write accurate migration guides and deprecation notices.
• Third-Party API Monitoring: Periodically compare known third-party API specs against newly fetched versions. Detect when a dependency's API has changed in ways that will break your integration.
• Microservice Contract Verification: In a microservice architecture, each service publishes its OpenAPI spec. Run the checker across service boundaries to verify that consumer-driven contracts are still satisfied.