Cache Stampede (Dog-Pile)
criticalSummary
When a widely-shared cached value expires or is invalidated, all concurrent requests that miss simultaneously trigger identical expensive database queries, overwhelming the origin store before any single result can be computed and cached: a positive feedback loop that can collapse the database within seconds.
Description
Cache stampede is a specific manifestation of the thundering herd pattern focused on cache expiry. A hot cache key serves hundreds or thousands of requests per second. When that key expires (TTL expiry) or is deliberately invalidated, all requests that arrive in the next query-latency window simultaneously observe a cache miss. Each independently determines that it must query the database. Hundreds of identical queries execute concurrently against a database that was handling zero of those queries a moment ago.
The difference from a general thundering herd: stampede is triggered by cache management events (expiry, invalidation, cold start) rather than traffic spikes. The onset is instantaneous and predictable: every cache key will eventually expire. High-traffic systems with aggressive TTLs will experience stampedes continuously unless prevention is built in from the start.
Self-amplification mechanism: database queries during a stampede are slower than at normal load (connection pool contention, buffer cache eviction). Slower queries mean more concurrent requests accumulate before the first result is returned and cached. More concurrent requests consume more connections. Connection pool exhaustion causes subsequent requests to queue at the pool layer. The failure cascades.
Three distinct scenarios: 1. Single key expiry: one extremely hot key expires (home feed, front page, popular item) 2. Mass expiry: many keys set with the same TTL expire simultaneously (cache warming after deploy) 3. Intentional invalidation: cache flush on configuration change, deployment, or bug fix
Characteristics
Triggers
- ·Hot cache key reaches its TTL and expires under sustained concurrent access
- ·Full cache flush (Redis FLUSHDB, Memcached restart) causes all requests to miss simultaneously
- ·Coordinated cache invalidation across multiple keys at the same wall-clock time
- ·New service deployment where the cache is cold and traffic is immediately routed to the new instance
Detection Signals
Mitigation Strategies
On a cache miss, acquire a distributed lock (Redis SET NX PX 5000) before querying the database. Only the lock holder executes the query. Other concurrent misses poll the cache at 50ms intervals until the lock holder populates it. Reduces N database queries to 1 per stampede event.
Before a key expires, a fraction of reads voluntarily trigger a background refresh. The probability increases as the key approaches expiry: P(refresh) ∝ exp(-remaining_ttl / beta). A background worker refreshes the key while the current value continues to be served. Key never reaches zero: stampede window is eliminated.
Add random jitter to all cache key TTLs (±10–20% of base TTL) to prevent co-expiry of keys set simultaneously. Eliminates the mass expiry scenario. Does not prevent single-key stampede under sustained load.
Serve the expired (stale) cache entry while triggering an asynchronous background refresh. The stale entry is served for at most one request cycle; the background refresh updates the cache before the next request arrives. Requires storing the previous value with an extended TTL alongside the logical expiry.
Recovery Steps
- 1.Identify the expired/invalidated hot key(s) from cache miss metrics and database slow query log
- 2.Manually repopulate hot keys (execute the population query once and SET the result)
- 3.Terminate or limit duplicate in-flight database queries for the stampede pattern
- 4.After recovery, implement one of: mutex, PER, or stale-while-revalidate to prevent recurrence
- 5.Add TTL jitter to all cache key setters to prevent mass co-expiry in the future
Estimated recovery time: 30–120 seconds after manual cache repopulation. The database load subsides as soon as a cached value is available to serve concurrent requests. Without manual intervention, the stampede persists for the duration of the database query latency under overload conditions: potentially minutes if the database degrades into connection exhaustion.
Affected Systems
Patterns
Technologies
Basis
Cache stampede is a well-documented and precisely understood failure mode with multiple published prevention algorithms (PER, mutex, stale-while-revalidate); the failure mechanics are mathematically analyzable and occur predictably in any cache-aside system with hot keys under sustained concurrent load
Run This Failure
Blast radius analysis for this failure mode within each scenario that carries it.
Related Architecture Knowledge
Inbound: affects this entity
Read-heavy API workloads are vulnerable to cache stampede when popular cache keys expire under sustained concurrent load, causing all requests to simultaneously miss and query the origin database.
Full relationship →Used In Architecture Scenarios
Multi-Tenant SaaS
A multi-tenant API gateway providing authentication, distributed rate limiting, request routing, payload transformation, and per-tenant usage analytics for API publishers. The hot path: authentication check, rate limit evaluation, and routing decision: must complete in under 1ms using Redis-only data structures to avoid proxying latency dominating upstream service response time. PostgreSQL stores tenant configuration, subscription plans, and API key definitions. Kafka receives API usage events for downstream billing and analytics. Configuration changes (rate limit updates, routing rule edits) must propagate to all gateway replicas without restart.
Read-Heavy Application
A CMS for publishing and serving structured content: articles, documentation, product pages, and localized variants: where read APIs serve 50–100x more traffic than editorial write APIs. PostgreSQL stores the content graph (articles, authors, categories, taxonomy) and workflow state (draft, in-review, scheduled, published). Redis caches published content objects for read APIs. Elasticsearch powers full-text content search with faceting and relevance ranking. Cache invalidation on publish must be fast and complete; N+1 query patterns on content relationship traversal are the dominant database performance risk during reads.
AI / RAG Application
A low-latency feature serving platform for ML model inference, providing both batch (offline) and real-time (online) feature access with strict training-serving consistency. Redis serves the hot feature cache with p99 latency targets below 5ms for online inference requests; PostgreSQL provides point-in-time feature lookups for offline training jobs with temporal consistency guarantees; Cassandra stores high-cardinality feature entities at write scale beyond PostgreSQL's single-primary ceiling; Kafka streams feature computation events from online feature pipelines to update the cache; Qdrant stores vector features for embedding-based model inputs and similarity lookups; ClickHouse serves feature analytics and drift monitoring across training dataset populations. Feature versioning is first-class: every feature value is tagged with a pipeline_version and computed_at timestamp to support model reproducibility and training-serving skew diagnosis.