Developer Technical Interview Rubric Creator

Senior Backend Engineer Interview Rubric: Distributed Rate Limiter

1. Interview Overview & Scoring Weights

• Role: Senior Backend Engineer - High-Throughput Ingestion

Format: 60-minute practical coding and architecture session. Suggested split: 0-10 min requirements and capacity math, 10-20 min architecture, 20-45 min core implementation, 45-60 min failure-mode probing.

• Stack: Java 21, Spring Boot, Kafka, Redis, DynamoDB
• Problem: API rate limiter with distributed sliding windows and burst handling at 100k RPS
• Core Evaluation Dimensions:
  • Problem Solving & Architecture (30%)
  • Code Quality & Reliability (25%)
  • Edge-Case Resilience (25%)
  • Pragmatism & Debt Management (20%)
  • Communication and trade-off articulation is scored inside each dimension, not separately.

2. Standardized Scoring Matrix

3. Targeted Technical Edge Cases

• Case 1: Hot Tenant Saturates a Single Redis Key

  • Trigger Condition: One API key generates 30-40% of the 100k RPS. All traffic for that key hits one Redis key, and therefore one shard and one thread.
  • Expected Senior Response: Recognizes the single-key ceiling. Proposes splitting the budget across sub-keys (e.g., key:{tenant}:{0..N}), or leasing batches of tokens to each app instance and syncing periodically. States the accuracy cost of each approach.
  • Red Flag: Says "Redis is fast enough" without computing per-key ops/sec, or proposes scaling Redis Cluster nodes without noticing that one key maps to one slot.

• Case 2: Check-Then-Increment Race

  • Trigger Condition: Two or more instances read the current count, both see it under the limit, and both increment, so the limit is exceeded under concurrency.
  • Expected Senior Response: Names the race immediately. Moves the read, evaluation, and write into a single atomic Lua script (or equivalent atomic command sequence). Explains why MULTI/EXEC alone does not give conditional logic.
  • Red Flag: Uses GET then INCR as separate calls, or adds a JVM-level synchronized lock and believes it solves a distributed problem.

• Case 3: Redis Outage or Latency Spike

  • Trigger Condition: Redis failover takes several seconds, or p99 latency jumps above the request budget. The interviewer asks "what happens to the API now?"
  • Expected Senior Response: Sets a tight client timeout and a circuit breaker. Makes an explicit, justified fail-open or fail-closed choice per endpoint class. Proposes a coarse local limiter as a fallback. Considers recovery behavior (avoiding a thundering herd or counter reset exploit).
  • Red Flag: No timeouts, so request threads (including virtual threads) pile up behind a dead dependency. Or defaults to fail-open for everything without discussing abuse risk.

• Case 4: Window Boundary Burst and Clock Skew

  • Trigger Condition: Fixed-window counters allow 2x the limit across a boundary. Separately, instances compute window start from their own clocks, which drift.
  • Expected Senior Response: Explains why the sliding window fixes the boundary problem. Uses the sorted-set log (exact, memory-heavy) or the weighted two-window counter (approximate, cheap) and states the trade-off. Takes time from Redis (TIME inside the script) rather than from app servers.
  • Red Flag: Uses System.currentTimeMillis() on each app node as the source of truth, or cannot explain why the fixed-window approach fails.

• Case 5: Retry Storm After Throttling

  • Trigger Condition: Clients receive 429s and retry immediately and in sync, amplifying load. Also tests whether the candidate's design handles bursts without punishing legitimate spikes.
  • Expected Senior Response: Returns Retry-After and rate-limit headers. Recommends client backoff with jitter. Combines a sliding window for the sustained rate with a token bucket for burst capacity. Notes that rejected requests should be cheap (reject before expensive work). Mentions Kafka only for async usage and audit events, not for the decision path.
  • Red Flag: Treats burst handling as "raise the limit". Puts Kafka or DynamoDB in the synchronous decision path. Has no plan for synchronized client retries.

4. Technical Debt Tolerance Indicators

• Healthy Pragmatism (Positive Indicators):

  • Says, for example, "I'll hardcode per-tier limits now and load them from DynamoDB later. The PolicyProvider interface lets me swap that in without touching the limiter." The shortcut is named, scoped, and replaceable.
  • Defines clear seams (RateLimiter, WindowStore, PolicyProvider, Clock) so Redis, the algorithm, and the config source can each change independently.
  • Proposes shadow mode or log-only enforcement before blocking real traffic.

• Reckless Shortcuts (Negative Indicators):

  • Ignores non-functional requirements: no latency budget, no memory estimate for the sliding log at 100k RPS, no metrics.
  • Omits timeouts, the Redis failure path, or the fail-open/fail-closed decision, and does not acknowledge the omission.
  • Leaves non-atomic check-and-update in place with "we'll fix it later".

• Over-Engineering (Negative Indicators):

  • Puts Kafka consumers or DynamoDB conditional writes in the per-request path.
  • Builds a generic multi-algorithm plugin framework, multi-region consensus, or a custom protocol before the single-region core works.
  • Spends more than 15 minutes on one component (e.g., the Spring configuration or a perfect abstraction) and leaves the core algorithm unfinished.

5. Decision Heuristic

• Clear Hire Threshold:

  • Weighted score of 3.0 or higher.
  • No dimension below 2, and at least two dimensions at 3 or higher, one of which must be Problem Solving & Architecture or Edge-Case Resilience.
  • The core atomic check-and-update logic is working or correctly specified by the end of the session.
  • A weighted score of 3.5 or higher with Edge-Case Resilience at 3 or higher should be flagged for consideration at a higher level.

• Hard Disqualifiers:

  • Cannot identify or fix the check-then-increment race even after a direct hint.
  • Proposes a per-request DynamoDB write or a Kafka round trip on the hot path and cannot be moved off it after discussion.
  • No fail-open/fail-closed reasoning at all after being asked about a Redis outage.
  • Fabricates or misrepresents technical claims (e.g., asserts MULTI/EXEC gives conditional atomicity) and does not correct after probing.
  • No working or coherent core implementation by minute 45 and no clear plan for completing it.

Interviewer note: Ask the same probes in the same order for every candidate (clock skew, hot key, Redis outage), and score from observed behavior, not from how polished the explanation sounds.

I can also turn this into a Word or PDF scorecard with blank scoring cells and a notes section for interviewers.