Relationship · Vulnerable To
Source: Technology·Target: Failure Mode
Summary
Redis is itself vulnerable to thundering herd when it restarts or flushes: all cache entries expire simultaneously, and many concurrent requests all miss and race to repopulate the same keys from the database, causing a stampede that can overwhelm the downstream database.
Evidence
- ·Redis FLUSHALL or restart clears all keys: all hot keys miss simultaneously on the first requests
- ·A Redis node serving 100,000 cache hits/second loses 100,000 requests/second to the database on cold start
- ·Facebook's memcache cold-start mitigation (warming from replica before switching traffic) addresses this pattern
- ·Twitter documented Redis cold-start incidents as a recurring operational failure mode
- ·Gradual traffic shifting to restarted Redis nodes (5% → 25% → 100%) prevents thundering herd
Operational Context
- ·Pre-warm Redis after restart by replaying recent cache population before accepting production traffic
- ·Use Redis Sentinel or Redis Cluster: replica failover maintains warm cache across node failures
- ·Stagger TTL expiry across keys using jitter (TTL = base + random(0, base/4)) to prevent synchronized expiry
Tradeoffs
- ·Pre-warming requires either a warm replica or a cache loader process: adds operational complexity
- ·Jitter reduces synchronized expiry risk but requires all cache writers to implement TTL jitter
- ·Redis Cluster (with replicas) reduces but does not eliminate cold-start risk: slot resharding warms new nodes
Evidence grounding
Grounded, 5 supporting itemsRedis restart-triggered thundering herd is a documented and common production incident pattern. Every system that caches in Redis must consider cold-start cache warming to prevent this.