Rate Limit Cascade

When a downstream service begins rate limiting requests from an upstream service, the upstream's retry logic with insufficient backoff amplifies the request rate : exceeding the rate limit further and potentially pushing the rejection downstream to other upstream callers, producing a cascade of rate-limited retries across the call graph.

Rate limiting is intended to protect a service from overload. When rate limiting begins, callers receive 429 Too Many Requests. Well-behaved callers back off. Poorly-behaved callers (or callers without explicit 429 handling) retry immediately or with minimal delay, increasing the request rate beyond the limit: the exact opposite of the intended behavior.

Cascade mechanics: 1. Service B begins rate limiting Service A (A is over quota or B is under load) 2. A's retry logic retries the 429 immediately or within 1 second 3. A now sends 2× requests (original + retry) to B 4. B rate-limits more requests; A retries more 5. A's request backlog grows; A begins to exhaust its own connection pool and

thread pool waiting for successful responses from B

6. A starts returning errors to its own callers (Service C) 7. C retries its requests to A, compounding A's load further

This pattern is particularly common when: - The retry library handles all non-2xx responses with immediate retry - No distinction is made between retryable (503, 502) and non-retryable (429) status codes - The Retry-After header from the 429 response is ignored

A rate-limit cascade can cause a brief transient rate limit event to become a prolonged partial outage across multiple services in the dependency graph.