Api Latency Optimizer

Api Latency Optimizer

# API Latency Optimizer

Slow APIs are the #1 cause of user churn, revenue loss, and cascading system failures. Every 100ms of additional latency costs Amazon 1% of sales. Google found that 500ms of extra load time reduced search traffic by 20%. Yet most teams lack a structured approach to finding and fixing latency -- they react to symptoms rather than root causes. Existing tools are fragmented: APM tools tell you _that_ something is slow but not _why_ or _what to do about it_. Manual debugging wastes hours correlating log timestamps. This skill gives you a complete, methodical pipeline for measuring, diagnosing, prioritizing, and fixing API latency across the entire stack.

What It Does

• End-to-end latency measurement -- Times every stage: DNS resolution, TCP/TLS handshake, request queue wait, server processing, database queries, external API calls, serialization, and network transfer
• Percentile-based analysis -- Reports P50, P95, P99, P99.9 latency per endpoint so you see the real user experience, not just the average
• Bottleneck identification -- Automatically classifies latency sources by category (network, database, application, infrastructure, third-party) and ranks them by impact
• Targeted optimization blueprints -- Generates step-by-step remediation plans for each identified bottleneck with framework-specific code snippets
• Progressive optimization tracking -- Compares before/after measurements across optimization rounds and flags regressions
• Caching strategy designer -- Recommends caching layers (CDN, Redis, in-memory, HTTP caching) matched to your data volatility and access patterns

Frameworks/Standards Covered

| Standard/Framework | Application | |---|---| | OpenTelemetry (OTel) | Distributed tracing and span-based latency measurement | | Prometheus + Grafana | Metrics collection and latency dashboarding | | RED Method (Rate/Errors/Duration) | Service-level latency monitoring pattern | | USE Method (Utilization/Saturation/Errors) | Infrastructure-level bottleneck analysis | | Google SRE Golden Signals | Latency, traffic, errors, saturation context | | HTTP/1.1, HTTP/2, HTTP/3 | Protocol-level optimization (multiplexing, 0-RTT) | | gRPC | Streaming, protobuf serialization, connection management | | GraphQL | Query complexity analysis, data loader patterns, batching | | Redis | Caching strategies, TTL management, eviction policies | | CDN (CloudFront, Cloudflare, Fastly) | Edge caching, origin shielding, regional routing | | PostgreSQL / MySQL / MongoDB | Query optimization, indexing, connection pooling | | NGINX / Envoy / Kong | Reverse proxy tuning, upstream timeouts, load balancing | | AWS Lambda / Cloud Functions | Cold start mitigation, warmup strategies, concurrency |

Detailed Features

### 1. Structured Latency Measurement The skill guides you through measuring latency at every layer of the stack. Instead of vague "the API is slow" complaints, you get exact timing for each component: - Network layer: DNS lookup time, TCP handshake RTT, TLS negotiation duration, bandwidth delay product - Application layer: Request deserialization, authentication/authorization, business logic, middleware chain, response serialization - Data layer: Query execution time, index scan vs. full scan detection, N+1 query identification, connection acquisition wait - External dependencies: Third-party API call duration, retry overhead, circuit breaker state impact ### 2. Percentile-Based Reporting Averages lie. The skill computes and interprets latency percentiles: - P50 (median): The typical user experience. If this is high, most users are affected. - P95: Identifies the slow 5% -- often caused by cold caches, GC pauses, or resource contention - P99: The reliability floor. High P99 indicates systemic issues that will cascade under load - P99.9: For high-traffic systems, even 0.1% of requests being slow affects thousands of users per day The skill explains the relationship between percentiles and provides optimization targets based on the shape of your latency distribution. ### 3. Bottleneck Classification and Prioritization Bottlenecks are automatically classified and scored by potential impact: | Category | Common Causes | Typical Impact | |---|---|---| | Network | DNS latency, TLS overhead, bandwidth limits, geographic distance | 10-300ms | | Database | Missing indexes, N+1 queries, lock contention, full table scans | 50-5000ms | | Application | Slow algorithms, blocking I/O, serialization overhead, memory leaks | 10-1000ms | | Infrastructure | CPU starvation, memory pressure, disk I/O wait, connection pool exhaustion | 5-500ms | | Third-party | External API latency, rate limiting, slow SDK/library calls | 50-10000ms | Each finding includes a severity level (Critical/High/Medium/Low), an estimated time-to-fix, and a clear recommendation. ### 4. Optimization Blueprint Generation For each identified bottleneck, the skill generates a concrete action plan with: - Specific code changes or configuration updates - Expected latency reduction (e.g., "Expected: reduce P95 from 1200ms to 200ms") - Pre-requisites and dependencies - Rollback instructions for safety - Verification steps to confirm the fix ### 5. Caching Strategy Designer Selecting the right caching layer is critical. The skill evaluates: - Data volatility: How often does the data change? (seconds vs. hours vs. static) - Access patterns: Read-heavy vs. write-heavy, hot keys, temporal locality - Consistency requirements: Strong consistency vs. eventual consistency - Budget and complexity trade-offs: In-memory caching (free, simple) vs. Redis (moderate, powerful) vs. CDN (costly, global) Outputs a caching recommendation matrix with specific TTL values, eviction policies, and invalidation strategies. ### 6. Optimization Tracking and Regression Detection Each optimization round generates a diff report comparing before/after metrics. The skill flags: - Regressions in other endpoints caused by the change - Trade-offs (e.g., lower latency but higher error rate) - Drift over time after the change was applied ### 7. Connection Pooling and Concurrency Analysis Evaluates connection management: - Pool size relative to concurrent request volume - Idle timeout vs. connection reuse rate - Retry strategy (exponential backoff parameters) - Circuit breaker thresholds

Usage

### Quick Start ```bash # Install via Agensi npx agensi install api-latency-optimizer # Or manual: add to your agent's skills directory unzip api-latency-optimizer.zip -d /path/to/skills/ ``` ### Running the CLI Analyzer ```bash # Analyze an endpoint with sample requests python3 scripts/latency-optimizer.py analyze --url https://api.example.com/users --method GET --samples 20 # Generate a full optimization report python3 scripts/latency-optimizer.py report --url https://api.example.com --endpoints /users,/orders,/products --samples 50 --output report.md # Profile a specific bottleneck python3 scripts/latency-optimizer.py profile --url https://api.example.com/search --method POST --headers \'{"Content-Type": "application/json"}\' --body '{"query": "test"}' # Compare before/after optimization python3 scripts/latency-optimizer.py compare --before measurements/before.json --after measurements/after.json # Design a caching strategy python3 scripts/latency-optimizer.py cache-strategy --read-ratio 0.8 --write-ratio 0.2 --data-volatility static ``` ### Configuration Example ```yaml # config/latency-profiles.yaml profiles: default: sample_count: 20 timeout_seconds: 30 percentiles: [50, 95, 99, 99.9] include_dns: true include_tls_handshake: true follow_redirects: false production: sample_count: 100 timeout_seconds: 60 percentiles: [50, 95, 99, 99.9] include_dns: true include_tls_handshake: true follow_redirects: true headers: Authorization: "Bearer ${API_TOKEN}" User-Agent: "Latency-Optimizer-Prod/2.0" ```

Output Format

The skill produces structured reports in Markdown or JSON format: ```markdown # API Latency Optimization Report Generated: 2026-07-06T12:00:00Z Endpoint: GET /api/users

Summary

- P50: 42ms (target: <50ms) [PASS] - P95: 187ms (target: <200ms) [PASS] - P99: 1450ms (target: <500ms) [FAIL] - P99.9: 3200ms (target: <1000ms) [FAIL] - Sample count: 100

Bottlenecks Found

### [CRITICAL] Database: N+1 Query Pattern Detected - Impact: +1200ms on P99 - Location: User join with orders (6 additional queries per request) - Fix: Add eager loading with JOIN FETCH or Include() - Expected improvement: P99 -> 250ms - Risk: Low (read-only change) ### [HIGH] Serialization: JSON Serialization Overhead - Impact: +180ms on P95 - Location: Response serialization for large payloads (250KB avg) - Fix: Enable field selection via query params, add pagination - Expected improvement: P95 -> 80ms - Risk: Low (backward-compatible with defaults)

Caching Recommendations

| Strategy | Hit Rate Est. | Latency Reduction | Complexity | |---|---|---|---| | Redis: User profiles (TTL: 300s) | 85% | -150ms | Low | | CDN: Static assets | 95% | -200ms | Low | | In-memory: Reference data | 99% | -50ms | Very Low |

Optimization Plan (Priority Order)

1. Fix N+1 query pattern in users endpoint 2. Add response field selection 3. Enable HTTP/2 multiplexing 4. Configure Redis cache for user profiles 5. Add pagination defaults 6. Tune connection pool (max: 50, idle: 10) ```

Why This Beats Prompting It Yourself

Prompting It Yourself API Latency Optimizer --- --- --- Measurement You guess timings or look at one metric Systematic timing at every layer with percentiles Bottleneck detection Misses root causes, treats symptoms Classifies by category and impact score Optimization plan Generic advice you have seen before Concrete steps with code, expected improvement, and risk Caching design "Use Redis" without specifics TTL values, eviction policies, invalidation per data type Tracking Forget to remeasure after changes Automated before/after diff with regression detection Edge cases Missed entirely 10+ documented edge cases with remediations Setup time Hours of tool setup Zero dependencies, single Python script Consistency Depends on today's prompt quality Reproducible methodology every time

Use Cases

• Production incident response: A customer-facing API starts timing out. Run the analyzer to identify the bottleneck (e.g., a database query missing an index after a schema migration) and get an immediate fix plan.
• Pre-deployment performance validation: Before shipping a new endpoint, profile it against latency budgets and catch regressions before they hit production.
• API migration performance comparison: Moving from REST to GraphQL or HTTP/1.1 to HTTP/2? Use the compare mode to quantify the improvement and validate assumptions.
• Cloud migration latency audit: Moving services between regions or cloud providers. Run a full latency audit on all critical endpoints to establish baselines and catch degradation.
• Caching strategy redesign: Your Redis cluster is costing $500/month and you are not sure it is helping. The cache strategy designer validates ROI and recommends optimal configurations.