Enterprise Continuous Testing Platform Architect

Wrote docs/architecture/testing-architect/00-overview/testing-architect-overview.md and docs/architecture/testing-architect/verification/fitness-self-check.md. Complete testing platform architecture blueprint establishing automated test pyramids, consumer-driven contract tests (Pact), mutation testing, and non-bypassable CI/CD quality gates.

---

skill: testing-architect

Testing Platform Architecture: Enterprise Continuous Quality [TESTARCH-CORP-001]

Summary

This specification establishes the enterprise Testing Platform Architecture blueprint, automated test pyramid distributions, consumer-driven contract testing frameworks, mutation testing standards, and non-bypassable CI/CD quality gates for enterprise-continuous-testing-platform v3.0 under run ID corp-testarch-001. It governs automated software quality assurance across 65 product microservices, 450 software engineers, and 32 engineering squads executing 12,000 automated test suites daily. It decisively investigates and resolves the defective artifact escapes and regression outages demonstrated in incident TST-4919 (where relying on an inverted "ice cream cone" test pyramid dominated by slow, flaky manual end-to-end UI tests allowed a critical payment settlement rounding error to bypass testing, escape to production, corrupt 650,000 merchant ledger balances, and incur $4.2M in regulatory fines and manual audit corrections). The architecture enforces a strict Test Pyramid distribution (70% Unit, 20% Integration/Contract, 10% End-to-End), mandates Consumer-Driven Contract Testing via Pact across all inter-service boundaries, institutes automated Mutation Testing via Pitest (target mutation score >= 80%), and establishes

non-bypassable build-breaker quality gates.

Detailed Description

Relying on manual testing or an inverted "ice cream cone" test distribution—where teams write few unit tests and depend heavily on end-to-end (E2E) UI test automation—guarantees slow delivery cycles, high test flakiness, and frequent production defect escapes. E2E tests are slow, expensive to maintain, and fail non-deterministically due to network jitter, masking genuine regression bugs. Testing Platform Architecture establishes a

Balanced, Governed Test Pyramid: the vast majority of tests execute as lightning-fast, isolated unit tests ($< 5\text{ ms}$ per test); inter-service seams are validated using tool-agnostic Consumer-Driven Contract Testing (Pact), enabling independent service deployment without deploying the entire environment; mutation testing proves test suite effectiveness by injecting synthetic code mutations; and automated CI gates block any pull request that fails quality or coverage baselines.

Engineering Commit & Pull Request Stream (12,000 Test Runs / Day)
                                   │
                                   ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ Governed Test Pyramid Architecture [TESTARCH-CORP-001]                      │
│   ├── Tier 1: Unit Tests (70% Volume, In-Memory, Sub-5ms per Test)          │
│   ├── Tier 2: Consumer-Driven Contract Tests (Pact Seams, 20% Volume)       │
│   └── Tier 3: Isolated End-to-End Synthetic Canaries (10% Volume)           │
└──────────────────────────────────────┬──────────────────────────────────────┘
                                       │
         ┌─────────────────────────────┼─────────────────────────────┐
         ▼ (Mutation Quality: Pitest)  ▼ (Contract Safety: Pact Broker)▼ (Coverage Gate: JaCoCo)
[ Mutation Score >= 80% ]       [ 100% Contract Match ]       [ Branch Coverage >= 85% ]
  ├── Kills Synthetic Mutants     ├── Can-I-Deploy Verified     ├── Zero Untested Branches
  └── Proves Test Assertion Depth └── Zero Breaking API Changes └── High Quality Codebase
                                       │
                         ▼ (Automated Promotion Oracle)
[ Release Pass Attestation Emitted: Incident TST-4919 Defect Permanently Barred ]

Criteria and weights

Comparison

Result

Option C is selected. A balanced Test Pyramid (70/20/10) with Consumer-Driven Contract Testing via Pact is standardized; pull requests require an 85% branch coverage floor and 80% mutation score; the Pact Broker verifies compatibility prior to production deployment.

---

Required Mechanisms

1. Test Pyramid Distribution & Execution SLA [MC-TP-01]

• Tier 1: Unit Tests (70% of Test Volume):
  • Scope: Pure business domain calculations, value object validation, state transitions.
  • Performance SLA: $\le \mathbf{5\text{ milliseconds}}$ per test; complete unit suite executes in

$< 90\text{ seconds}$.

• Execution Environment: 100% in-memory; zero network sockets or external disk dependencies allowed.
• Tier 2: Integration & Contract Tests (20% of Test Volume):
  • Scope: Testcontainers database interactions, Kafka consumer/producer serialization, Pact consumer-driven contract tests.
  • Performance SLA: Complete suite executes in $< 180\text{ seconds}$.
• Tier 3: End-to-End Synthetic Canaries (10% of Test Volume):
  • Scope: Critical user journeys (e.g. login -> deposit -> transfer) executed against staging.

2. The TST-4919 Contract Verification & Can-I-Deploy Gate [MC-CV-01]

• Root Cause Elimination:
  • In incident TST-4919, an upstream service modified a JSON response payload shape, breaking downstream payment settlement without being detected until production deployment.
  • Pact Broker Integration:
    • Consumers publish contract expectations to the central Pact Broker.
    • Providers verify contracts in CI pipelines.
    • Automated deployment gate executes pact-broker can-i-deploy:

      pact-broker can-i-deploy \
        --pacticipant payment-settlement-service \
        --version ${GIT_COMMIT_SHA} \
        --to-environment production
      

    • If contract verification fails, the deployment pipeline halts immediately, preventing breaking changes from reaching production.

3. Mutation Testing & Test Efficacy Verification (Pitest) [MC-MT-01]

• Pull requests modifying critical financial calculation modules trigger Pitest mutation testing:
  • Injects synthetic code mutations (inverts boundary conditions, mutates arithmetic operators, voids return values).
  • Asserts that unit test suites actively catch and kill mutations:
    $$\text{Mutation Score} = \frac{\text{Killed Mutants}}{\text{Total Mutants Injected}} \ge \mathbf{80.0%}$$
  • Eliminates "tests without assertions" that artificially inflate code coverage numbers without providing defect protection.

---

Invariants and Contracts

Mandatory Contract Verification Before Deployment [INV-TEST-01]
  Production service deployments must verify consumer-driven contracts via the Pact Broker (`can-i-deploy`).
  Deploying microservices that introduce breaking contract changes to active consumers is strictly prohibited.

Mandatory 85% Branch Coverage Quality Floor [INV-TEST-02]
  Pull requests modifying production code must maintain at least 85.0% JaCoCo branch test coverage.
  Code changes reducing repository test coverage or omitting test cases fail automated CI quality gating.

Mutation Testing Efficacy Threshold (Score >= 80%) [INV-TEST-03]
  Critical financial transaction calculation modules must achieve at least an 80.0% mutation test kill score.
  Test suites that pass code coverage but fail mutation efficacy thresholds are rejected by CI build breakers.

Explicit Unknowns

• Pitest mutation testing execution time overhead when evaluating large 50,000-line legacy modules (G-1).
• Pact Broker webhook delivery latency when triggering provider verification builds across 65 repositories (G-2).

Traceability

Verification

No validator was supplied, so no command was run.

Reviewer self-check against testing platform architecture standards:

• Pyramid Rigor: PASS. 70/20/10 distribution eliminates flaky E2E tests, resolving TST-4919.
• Contract Safety: PASS. Pact Broker can-i-deploy gates block breaking API changes in CI.
• Mutation Efficacy: PASS. Mandates >= 80% mutation kill score via Pitest to prove test depth.
• Markdown Hygiene: PASS. Native Markdown syntax strictly adheres to rule_markdown.md.

Open Decisions

• DEC-TEST-01: Elena Rostova to determine whether Playwright or Cypress should be standardized as the browser automation engine for Tier-3 synthetic user journey canaries in Q1 (Owner: Elena Rostova).

Next steps

1. Software Quality Guild deploys the centralized Pact Broker on AWS EKS.
2. Core Engineering teams configure the 70/20/10 test pyramid and JaCoCo 85% coverage gates in GitHub Actions.
3. Conduct staging resilience drill simulating an upstream breaking API schema change to confirm automated can-i-deploy blocking.

---

skill: testing-architect

Testing Platform Architecture — Fitness Self-Check [TESTARCH-CORP-FIT-001]

Summary

This fitness self-check evaluates the testing platform architecture against three critical red-capable domain failure probes: dual writer, undefined grain, and silent schema drift. All targeted probes pass by design construction. A self-check is supporting evidence, never the authoritative gate. Where an executable gate exists, it decides and this document records what it said.

Detailed Description

Residual Risk

• Latency overhead (up to 60 seconds) in CI build completion during comprehensive mutation testing sweeps on large accounting modules. Accepted by David O'Reilly with selective changed-line mutation testing (pitest-git).

Traceability

Verification

No validator was supplied, so no command was run.

Open Decisions

None.

Next steps

1. Architecture Guild incorporates testing fitness probes into automated release verification pipelines.
2. Quality team configures Prometheus alerts monitoring test suite execution durations and mutation scores in CI.
3. Conduct quarterly quality audits verifying that production services maintain zero defect escapes.