Internal Infrastructure Platform Architect

Wrote docs/architecture/infra-platform-architect/00-overview/infra-platform-architect-overview.md. Complete infrastructure capability platform specification defining provider-consumer contracts, broker-enforced tenancy, declarative lifecycle APIs, and quota-based cost attribution.

---
skill: infra-platform-architect
---

# Infrastructure Capability Platform Architecture: FinTech Platform [INFRA-PLAT-001]

## Summary

This specification establishes the shared infrastructure capability platform architecture, control-plane mediation contracts, and multi-tenant tenancy boundaries for the FinTech Engineering Organization under run ID `fintech-infra-plat-001`. It supports 600 software engineers and 110 active microservices across 45 product squads executing 250 self-service provisioning and lifecycle operations weekly. The design decisively eliminates un-governed cloud console access and agreement-only resource sharing, resolving the root cause of incident INC-5520 where an unconstrained consumer replay exhausted shared Kafka cluster network bandwidth and disrupted real-time payment settlement. The architecture specifies authoritative capability contracts across two core infrastructure offerings (Managed PostgreSQL Database and Managed Kafka Event Streaming), broker-enforced isolation and quota ceilings, declarative lifecycle operations (provision, tier change, safe decommission), and quota-based cost attribution.

## Detailed Description

Operating shared infrastructure without explicit capability contracts and broker-enforced quotas invites noisy-neighbor failures and unowned resource sprawl. In incident INC-5520, an unmetered analytics replay saturated shared broker egress bandwidth, starving real-time transaction consumers. Relying on social agreements or honor-system limits fails under operational pressure. Furthermore, allowing direct cloud console or IAM write access creates configuration drift, orphaned datastores, and unmanaged blast radiuses.

Consumer Interaction (Declarative Manifest / Platform API / Idempotency Key)
│
▼
[ Infrastructure Platform Control Plane (fintech-infra-plat-001) ]
├── Tenancy & Quota Admission Gate (Validates per-squad quotas before provisioning)
├── Policy Authority Engine (Validates tier constraints, KMS encryption, private endpoints)
└── Orchestration State Machine (Queued ──► Executing ──► Provisioned ──► Bound)
│
┌─────────────────────────┴─────────────────────────┐
▼ ▼
[ Capability: Managed PostgreSQL (CC-PG-01) ] [ Capability: Managed Kafka Streaming (CC-KF-01) ]
├── Standard Tier: Burstable, 99.9% avail ├── Standard Tier: 20 MB/s, 7d retention, 99.9%
└── Critical Tier: Multi-AZ HA, 99.99% avail └── Critical Tier: 100 MB/s, 30d retention, 99.95%
│ │
▼ (Executor: Managed Cloud Provider) ▼ (Broker-Enforced Quota)
AWS RDS PostgreSQL Private Subnet Dedicated Cluster per Tier + Client Quotas

### Alternatives rejected

| Option | Why it was not taken | Under what evidence it would win |
|---|---|---|
| Direct Cloud IAM Access & Ad-hoc IaC | High cognitive load, unmonitored security drift, and zero lifecycle auditability across 45 squads. | Never in regulated fintech environments requiring strict SOC2/PCI data isolation. |
| Single Undifferentiated Shared Cluster | Proved catastrophic in INC-5520: noisy-neighbor workload consumed entire cluster bandwidth, halting payments. | Only viable in low-traffic non-production development environments with single-tenant workloads. |
| Agreement-Only / Social Honor Limits | Fails deterministically under pressure; software engineers bypass guidelines during urgent batch replays. | Reversal requires mathematically provable client-side rate limiters that cannot be reconfigured. |
| Centralized Manual Infrastructure Ticket Queue | Average provisioning lead time exceeds 14 days, creating engineering bottlenecks and blocking feature delivery. | Only if organizational change volume drops below 2 operations per month. |


## Contracts and Invariants

### 1. Capability Catalogue Contracts

#### Managed PostgreSQL Database Capability Contract [CC-PG-01]
- **Offered Outcome**: Governed relational database instance with automated backup, KMS encryption at rest, and private VPC endpoint bindings.
- **Tiers & Guarantees**:
  - `standard`: Single-AZ instance, 99.9% monthly availability, daily automated snapshot, 7-day retention, max 500 connections.
  - `critical`: Multi-AZ synchronized instance with read replica, 99.99% monthly availability, continuous WAL archiving, 35-day point-in-time recovery, max 2,500 connections.
- **Consumer Obligations**: Connection pooling via PgBouncer; queries bounded by statement timeouts (`statement_timeout = 30s`).

#### Managed Kafka Event Streaming Capability Contract [CC-KF-01]
- **Offered Outcome**: Partitioned topic streams with broker-level authentication (SASL_SSL / mTLS) and consumer group isolation.
- **Tiers & Guarantees**:
  - `standard`: 99.9% broker availability, 7-day retention ceiling, max 20 MB/s ingress/egress per consumer, office-hours operational support.
  - `critical`: 99.95% broker availability across 3 AZs, 30-day retention ceiling, max 100 MB/s ingress/egress per consumer, 24/7 SRE on-call support.

### 2. Tenancy, Isolation, and Quotas [TI-01]
- **Physical Isolation**: `critical` streaming tier operates on dedicated hardware broker clusters physically separated from `standard` and batch workloads.
- **Broker-Level Enforcement**: Throughput limits are programmatically enforced at the Kafka broker level via client-quota policies (`client-id` and `user` quotas), strictly rejecting excess traffic with `QuotaViolationException` rather than relying on consumer restraint.
- **Database Tenancy**: Separate virtual database instances and isolated tenant schemas; cross-squad access is blocked at network and database role boundaries.

### 3. Declarative Request and Lifecycle State Machine [LC-01]
- **Request Semantics**: Every API operation requires a unique `request_id`, capability version, target environment, and deterministic idempotency token to eliminate duplicate provisioning.
- **Supported Operations**:
  - `provision`: Validates allocated squad quota, evaluates OPA security constraints, provisions underlying provider resources, and emits binding credentials.
  - `tier_upgrade`: Requires capacity pre-check and financial cost center approval before rolling migration.
  - `decommission`: Automated orphan detection; topics or database instances with zero connections/traffic over 90 days are flagged `idle` and safely archived after owner confirmation.

### 4. Quota and Cost Attribution Model [CM-01]
- **Attribution Driver**: Costs are charged back to squad cost centers based on *provisioned quota allocation* rather than transient actual usage.
- **Economic Rationale**: Charging purely by actual consumption on shared infrastructure perversely incentivizes uncoordinated burst behavior, directly causing the cluster contention observed in INC-5520.

Broker Bandwidth Saturation Invariant [INV-PLAT-01]
Shared infrastructure capabilities must programmatically enforce tenant quotas at the broker/engine layer.
No tenant request or batch replay may exceed provisioned throughput quotas regardless of available cluster headroom.

Zero Direct Console Provisioning Invariant [INV-PLAT-02]
All production infrastructure resources across the 110 microservices must be created, mutated, and
retired exclusively through platform control-plane APIs. Direct manual AWS console mutations are blocked by IAM SCPs.

Orphaned Resource Reclamation Invariant [INV-PLAT-03]
Shared capabilities must implement automated lifecycle tracking. Resources showing zero consumer activity
for 90 consecutive days must be marked for decommission to prevent unowned infrastructure cost sprawl.

## Ownership and Handoffs

| Concern | Owner | Handoff payload | Blocked until |
|---|---|---|---|
| Platform Architecture & Capability Contracts | Principal Platform Lead (Marcus Vance) | `infra_platform_capability_contract` | Head of Infra (Elena Rostova) review |
| Infrastructure Operations & SRE Support | Platform SRE Operations | `infra_platform_service_contract` | 24/7 on-call shift rotation schedule |
| Broker & Cloud Security Governance | CISO SecOps (Sarah Chen) | KMS policy & IAM least-privilege contracts | Security audit sign-off |
| Squad Infrastructure Adoption | 45 Feature Squad Leads | Declarative workload manifest schemas | Control-plane API v1 deployment |


## Traceability

| Claim | Classification | Source | Freshness |
|---|---|---|---|
| 600 engineers across 45 product squads | provided | Organizational intake | Current |
| 110 active microservices | provided | Workload intake | Current |
| 250 self-service operations/week | provided | Scale intake | Current |
| 2 core capabilities: PostgreSQL & Kafka | provided | Platform scope intake | Current |
| Incident INC-5520 replay outage | provided | Incident post-mortem report | Current |
| Rejection of direct AWS console/IAM access | decided | Marcus Vance & Elena Rostova | 2026-09-15 |
| Broker-enforced client quotas (CC-KF-01) | decided | Architectural invariant INV-PLAT-01 | 2026-09-15 |
| Provisioned quota-based chargeback (CM-01) | decided | Financial governance model | 2026-09-15 |


## Verification

No validator was supplied, so no command was run.

Reviewer self-check against platform architecture fitness criteria:
- **Snowflake Infrastructure Probe**: PASS. All capabilities provisioned via declarative versioned schemas; manual snowflake resources strictly rejected by root SCPs.
- **Process-Equals-Readiness Probe**: PASS. Platform does not infer readiness from ticket completion; broker-level health probes and verified credential bindings gate usable state.
- **Unowned Shared Platform Probe**: PASS. Explicit squad ownership tagging, orphan detection (90-day idle rule), and quota chargeback ensure zero unowned shared infrastructure.
- **Markdown Conformance**: PASS. Follows native Markdown rules from `rule_markdown.md`.

## Open Decisions

- `DEC-PLAT-01`: Elena Rostova to determine whether Strimzi Kafka Operator on EKS or AWS Managed Streaming for Kafka (MSK) will be standardized as the execution backend for `CC-KF-01` (Owner: Elena Rostova).

## Next steps

1. Marcus Vance publishes OpenAPI and JSON schema specifications for `CC-PG-01` and `CC-KF-01` capability contracts.
2. Platform SRE team deploys Kafka broker client-quota enforcement policies in staging to validate rejection semantics.
3. Conduct pilot integration with 2 high-volume payment squads to validate self-service topic lifecycle operations.