Thundering Herd (Cache Stampede)

Before switching traffic to a new application instance, run a cache warm-up pass that pre-populates all known hot keys. Eliminates the cold-start stampede on deployment. Requires maintaining a list of hot keys or using a traffic-replay mechanism to identify them.

On a cache miss, acquire a distributed lock (Redis SETNX with short TTL, e.g., 5 seconds) before executing the database query. Only the lock holder executes the query and populates the cache; all other concurrent misses wait and then re-check the cache after the lock is released. Reduces N simultaneous database queries to 1 per cache miss event.

Instead of expiring a key exactly at TTL, each read recomputes expiry with a probability that increases as TTL approaches zero: should_refresh = (random() < exp((current_ttl - TTL) / beta)). A small fraction of reads trigger a background cache refresh before the key expires, eliminating simultaneous expiry. Beta = 1.0 is a standard starting point; tune based on query cost and acceptable staleness.

On cache miss, immediately return the last known (potentially expired) value while triggering an asynchronous background refresh. Caller receives stale data for at most one request cycle. Requires storing the previous value alongside the TTL metadata.

Redis distributed locks (via SET NX EX or Redlock) prevent thundering herd by ensuring only one caller repopulates a cache entry at a time, with other callers either waiting or returning a stale value until the cache is warm.

Add random jitter (±10–20% of TTL) to cache key expiry times to prevent multiple hot keys from expiring simultaneously. Reduces the probability of a simultaneous multi-key stampede; does not prevent single-key stampede under sustained load.