ML Serving Architecture Selector

Choose a serving pattern your team can actually operate

ML serving decisions are often made from technology preference instead of workload constraints. This skill turns latency, throughput, freshness, cost shape, team capacity, rollout needs, and compliance requirements into a structured architecture decision.

What it produces

• A binding-constraint register tagged Verified, Inferred, or Assumed
• An elimination pass for structurally unsuitable patterns
• A six-axis comparison matrix for the surviving options
• Cost formulas using supplied values and CALIBRATE markers where values are missing
• An ML-SERVING-ADR with the decision, strongest counterarguments, and revisit triggers

Patterns covered

Batch, real-time, streaming, edge, managed platforms, Kubernetes-based serving, and serverless inference. The skill compares architecture patterns; it does not deploy or configure any platform.

Best for

• Teams deciding whether online inference is actually necessary
• Architects comparing managed serving with self-operated infrastructure
• ML teams documenting why an option was eliminated
• Reviewers who need explicit trade-offs and conditions that reopen the decision

Boundaries

It does not query live prices, benchmark latency, provision infrastructure, prove compliance, or guarantee cost or performance. Provider-specific claims must be verified against current authoritative documentation.