Incremental Sitemap Engine — Cold/Hot XML & <1ms Zero-Copy Nginx Serving

When large content websites (blogs, e-commerce, news platforms with 10k–500k+ URLs) rely on framework-level dynamic SSR routes (e.g. Next.js src/app/api/sitemaps/.../route.ts or Nuxt/Remix route handlers) to generate XML sitemaps on-demand:

1. Massive CPU & Database Thrashing: When Googlebot, Bingbot, or Ahrefs crawl sitemaps, they request dozens of weekly/monthly archive XML files concurrently. Each request triggers database pagination (fetching thousands of rows), JSON serialization, and heavy XML string concatenation in Node.js/Bun.
2. Gateway Timeouts (504s) & De-indexing: The Node.js event loop blocks, database connections exhaust, and responses take 5–15 seconds or timeout, causing search engines to abandon indexing.
3. 95% Waste on Immutable Historical Data: Articles published in past months/years are closed and immutable. Recomputing them dynamically on every crawl is pure waste.

This skill standardizes the Cold/Hot Incremental Generation + Nginx Zero-Copy Static Serving pattern to achieve sub-second delta builds and < 1ms TTFB under infinite crawler concurrency.

[!IMPORTANT] Service Placement Rule (CMS/Backend-Driven, Zero-FE Overhead): Always place the sitemap generator engine, database queries, and .sitemap-manifest.json state inside the CMS / Backend service that owns the database and content lifecycles (e.g., index-admin-cms), never inside the Frontend SSR app (index-web):

1. Event-Driven Auto-Sync: Hook debounced incremental regeneration (scheduleIncrementalSitemapSync) directly into CMS publish/update/delete lifecycles (lifecycles.ts) right before CDN cache purge.
2. Trigger & Status API: Expose POST /api/articles/sitemap/trigger and GET /api/articles/sitemap/status on the CMS so Frontend or Ops only makes a single lightweight HTTP call if a manual trigger is needed.
3. Zero Frontend Loops: Remove multi-query lastmod probes and unbounded Promise.all pagination loops from Frontend SSR routes to prevent socket pool exhaustion and Cloudflare 524 timeouts.