Api Contract Tester v2

About This Skill

# API Contract Tester Your docs say the API returns `{id, name, email}`. The code returns `{id, name, email_hash, is_verified}`. The client breaks in prod. This skill catches that mismatch before merge — automatically, on every PR. ## What it does - **Generates contract tests** from OpenAPI 3.x or GraphQL SDL - **Validates live or recorded responses** against the spec - **Detects drift** — fields added, removed, type-changed - **Catches breaking changes** — required field added, type narrowed, enum removed - **Generates clients** — TypeScript, Python, Go, Java from spec - **Mock server** — stand up a fake API from spec for local dev - **CI gate** — fail PR if contract violated ## When to use it - Your API has an OpenAPI/GraphQL spec but you don't know if the code matches - You ship breaking changes by accident - You want to generate clients in multiple languages from one source - You need a mock API for frontend dev - You're adding a new endpoint and want CI to enforce the spec - You're integrating a third-party API and want to know if it changed ## Why it's better than ad-hoc prompting Most "test my API" prompts give generic advice. This skill is different: - **Spec-driven** — every test comes from the contract, not vibes - **Drift-aware** — distinguishes breaking vs non-breaking changes - **Multi-protocol** — REST + GraphQL in one tool - **CI-friendly** — exits non-zero on contract violation - **Mock included** — no need for a separate Prism/Mockingjay setup ## Architecture ``` ┌─────────────────────────────────────────────────────────┐ │ Agent (Claude/Cursor) │ │ - Points at OpenAPI/GraphQL spec + code │ │ - Runs contract tests │ │ - Reports drift + breaking changes │ └───────────────┬─────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────┐ │ skills/api-contract-tester/ │ │ scripts/ │ │ ├── test_rest.py # OpenAPI contract tests │ │ ├── test_graphql.py # GraphQL contract tests │ │ ├── detect_drift.py # Compare code vs spec │ │ ├── check_breaking.py # Semver-aware change detect│ │ ├── generate_client.py # TS/Py/Go/Java clients │ │ └── mock_server.py # Spec → mock server │ │ references/ │ │ ├── breaking-changes.md │ │ ├── openapi-tips.md │ │ └── graphql-tips.md │ │ examples/ │ │ ├── petstore.yaml │ │ └── github.graphql │ └─────────────────────────────────────────────────────────┘ ``` ## Quick start ```bash # 1. Install pip install openapi-spec-validator prance httpx graphql-core # 2. Run contract tests against live API python scripts/test_rest.py --spec openapi.yaml --base-url https://api.example.com # 3. Detect drift between code and spec python scripts/detect_drift.py --spec openapi.yaml --code ./src/api/ # 4. Check a PR for breaking changes python scripts/check_breaking.py --old openapi-v1.yaml --new openapi-v2.yaml # 5. Generate a TypeScript client python scripts/generate_client.py --spec openapi.yaml --lang typescript --out clients/ # 6. Stand up a mock server python scripts/mock_server.py --spec openapi.yaml --port 4010 # → mock API at http://localhost:4010 ``` ## Sample output ``` ## API Contract Test Report **Spec**: openapi.yaml (v2.3.1, 47 endpoints) **Base URL**: https://staging.api.example.com **Endpoints tested**: 47/47 ✅ **Total assertions**: 312 **Pass**: 305 | **Fail**: 7 ### Failures #### GET /users/{id} (3 failures) - ❌ **Response missing field** — spec requires `created_at`, response lacks it - ❌ **Response extra field** — `internal_id` in response, not in spec (potential leak) - ❌ **Type mismatch** — spec says `id: string`, response has `id: number` #### POST /users (4 failures) - ❌ **Required field missing** — request must include `email`, test sent without - ❌ **Status code wrong** — spec says 201, got 200 - ❌ **Response missing field** — spec returns `verification_token`, response lacks it - ❌ **Header missing** — spec requires `X-RateLimit-Remaining` header ### Drift between code and spec - `internal_id` field in User schema (src/models/user.py:23) but not in spec - `is_verified` field in spec but not in code (src/api/users.py:89) ### Breaking changes vs v2.2.0 - 🛑 `User.email` changed from optional to required - 🛑 `User.id` type changed from string to integer - ⚠️ `User.role` enum lost the `guest` value ### Verdict: 🛑 BLOCK — 7 contract violations + 2 breaking changes ``` ## Breaking change detection (semver-aware) | Change | Severity | Action | |--------|----------|--------| | New optional field | Non-breaking | ✅ OK | | New required field | Breaking | 🛑 MAJOR bump | | Field type narrowed (any → string) | Non-breaking | ✅ OK | | Field type broadened (string → any) | Breaking | 🛑 MAJOR bump | | Enum value added | Breaking | 🛑 MAJOR bump | | Enum value removed | Breaking | 🛑 MAJOR bump | | New endpoint | Non-breaking | ✅ OK (MINOR bump) | | Endpoint removed | Breaking | 🛑 MAJOR bump | | New optional parameter | Non-breaking | ✅ OK | | Parameter made required | Breaking | 🛑 MAJOR bump | | Response status added | Non-breaking | ✅ OK | | Response status removed | Breaking | 🛑 MAJOR bump | ## Pricing Single-purchase, lifetime access. $7.50. Includes: - 6 Python scripts (REST, GraphQL, drift, breaking, client, mock) - 3 reference docs (breaking changes, OpenAPI tips, GraphQL tips) - 2 example specs (Petstore, GitHub GraphQL) - Future updates for the same major version ## Example usage > "Here's our OpenAPI spec and the API base URL. Tell me if the code matches, and flag any breaking changes since v2.2." The skill will: 1. Fetch the spec 2. Run 300+ contract assertions against the live API 3. Diff code vs spec for drift 4. Compare current spec to old version 5. Output `contract-report.md` with deployable/hold decision ## Compatibility Works with any agent that supports the SKILL.md standard and can execute Python: Claude Code, OpenClaw, Codex CLI, Cursor, Gemini CLI, Cline, Windsurf, Aider. Supports OpenAPI 3.0/3.1, GraphQL SDL, JSON Schema. Tested on Linux, macOS, Windows. ## Tags api, testing, openapi, graphql, rest, contract-testing, qa, ci-cd, backend