Rate Limiting
establishedSummary
Control the rate at which requests from a given client, tenant, or IP address are accepted by enforcing a maximum request count over a sliding or fixed window, protecting downstream systems from overload while preserving capacity for well-behaved callers.
Problem
Without admission control, a single misbehaving client (intentional abuse, buggy retry loop, viral traffic spike) can consume the entire API capacity, degrading service for all other clients. Rate limiting protects the system's aggregate capacity allocation and ensures fair resource distribution across identities.
Description
Rate limiting enforces admission control at the API or service boundary. Incoming requests are counted against a per-identity quota (user ID, API key, IP, tenant). Requests within the quota are accepted; excess requests are rejected (HTTP 429) or queued (throttled). The quota resets on a schedule (fixed window) or rolls continuously (sliding window, token bucket, leaky bucket).
Token bucket: each identity has a bucket that fills at a fixed rate (tokens/second) up to a burst ceiling. Each request consumes one token. When the bucket is empty, requests are rejected. Allows burst traffic up to the bucket size while enforcing an average rate limit. Redis INCR + EXPIRE or the lua script atomic increment is the standard implementation.
Leaky bucket: requests are added to a fixed-size queue; a drain process consumes them at a fixed rate. Excess requests are dropped when the queue is full. Produces smooth output even under bursty input but introduces queuing latency.
Fixed window: count requests in the current N-second window. Simple but susceptible to boundary attacks (a burst at window[N-1] + window[N+1] may double the limit).
Sliding window log: store timestamps of all requests in the last N seconds. More accurate but higher memory use per identity.
Rate limits are typically stored in Redis for sub-millisecond atomic increment and TTL-based window expiry. A Redis-backed token bucket does the read-modify-write of (tokens, last_refill_ts) atomically in a single Lua script keyed on the rate-limit key with the max tokens, refill rate, and current time as arguments, which prevents race conditions across multiple API server instances.
API gateway integration is usually preferable to a custom implementation: Kong, Envoy, AWS API Gateway, and NGINX all ship built-in rate limiting plugins, and are worth using directly when the gateway is already in the architecture.
Rate limit headers, per RFC 6585 and the draft RateLimit header specification, communicate the limit to clients: RateLimit-Limit (the window's request limit), RateLimit-Remaining (requests left in the current window), RateLimit-Reset (the Unix timestamp when the window resets), and Retry-After (seconds to wait before retrying, sent on a 429).
Distributed rate limiting requires every API server to share the counter rather than maintain a local one; local counters let traffic reach N times the limit across N servers. Redis Cluster, or a local-plus-global two-tier approach, balances latency against accuracy for the shared counter.
Tradeoffs
Protects downstream services from overload by bounding request rate per identity
Enables fair capacity allocation across tenants
Reduces abuse surface; bot traffic and credential stuffing attacks are limited by rate windows
Provides a clean capacity model for pricing and SLA design
Legitimate burst traffic is rejected when quota is exhausted, requiring client-side retry with backoff
Distributed rate limiting (multiple API servers sharing a counter) requires Redis or similar shared state, adding latency per request
Rate limit headers must be implemented correctly to guide client behavior
Choosing window type and limit values requires analysis of legitimate traffic patterns
When to use
Public or partner API exposed to clients you do not fully control
Clients can have bugs or malicious intent; rate limiting bounds their impact
Multi-tenant system where one tenant's traffic must not crowd out others
Without per-tenant limits, a high-volume tenant degrades all tenants
Downstream service has a fixed capacity ceiling (database, external API)
Rate limiting protects the downstream from overload when the downstream cannot auto-scale
Pricing model charges per API call (metered billing)
Rate limits enforce the billing tier's included call volume
When not to use
All callers are internal services under your control with known traffic patterns
Internal services can implement backpressure or circuit breaking instead of hard rejection
The system auto-scales infinitely relative to any conceivable traffic volume
If capacity is effectively unbounded, rate limiting only adds latency
Operational Requirements
Log and monitor 429 responses by identity
Distinguishes abuse patterns from bugs and legitimate traffic spikes.
Implement graceful degradation on 429
Return a helpful error body with the Retry-After header.
Consider a graduated response instead of a hard cutoff
Warn at 80% quota, throttle (queue) at 90%, reject at 100%.
Test limit values against real traffic profiles
Limits set too low cause false positives for legitimate burst traffic.
Characteristics
Relationships
Complements
Basis
Rate limiting is a foundational API protection pattern with well-established implementations across all major API gateways and Redis-based libraries; token bucket and leaky bucket algorithms are formally specified
Related Architecture Knowledge
Outbound: this entity affects
API gateways are the standard enforcement point for rate limiting; rate limiting rules configured on the gateway apply uniformly to all callers without code changes in downstream services.
Full relationship →Rate limiting at service ingress caps the load each upstream can place on a downstream, preventing the overload cascade triggered by burst traffic from multiple callers.
Full relationship →Rate limiting bounds the inbound request rate per identity, preventing any single caller from consuming the entire connection pool and exhausting capacity for other callers.
Full relationship →Rate limiting enforced at the API boundary prevents the retry amplification loop that causes rate limit cascades by ensuring callers never exceed the downstream quota in the first place.
Full relationship →Inbound: affects this entity
High-throughput OLTP workloads benefit from rate limiting to prevent individual tenants or clients from consuming the entire database write capacity during traffic spikes.
Full relationship →Redis atomic increment (INCR) with TTL (EXPIRE) is the standard implementation for sliding window and token bucket rate limiting, providing sub-millisecond rate limit enforcement.
Full relationship →Rate limiting enforces per-tenant quotas at the API level; tenant isolation enforces per-tenant resource boundaries at the database level. Together they provide multi-layered protection against noisy neighbors.
Full relationship →Used In Architecture Scenarios
Multi-Tenant SaaS
A multi-tenant API gateway providing authentication, distributed rate limiting, request routing, payload transformation, and per-tenant usage analytics for API publishers. The hot path: authentication check, rate limit evaluation, and routing decision: must complete in under 1ms using Redis-only data structures to avoid proxying latency dominating upstream service response time. PostgreSQL stores tenant configuration, subscription plans, and API key definitions. Kafka receives API usage events for downstream billing and analytics. Configuration changes (rate limit updates, routing rule edits) must propagate to all gateway replicas without restart.
Multi-Tenant SaaS
A multi-tenant developer tooling platform providing CI/CD pipeline execution, log aggregation, code analysis, and dependency scanning across isolated tenant organizations. Tenant isolation is the primary correctness constraint: a security boundary violation between tenants is a critical incident, not a performance event. PostgreSQL row-level security enforces data isolation; Redis manages job queues and distributed locks; Elasticsearch indexes pipeline log output for search; Kafka delivers webhook events to tenant-registered endpoints; MinIO stores pipeline artifacts. Resource quota enforcement prevents any single tenant's burst from affecting others.
Marketplace Platform
An e-commerce order lifecycle platform handling cart, checkout, payment, fulfillment, and returns across a mixed read/write workload where product discovery is read-heavy, checkout is write-transactional, and fulfillment is event-driven. The saga pattern orchestrates multi-step checkout: reserve inventory → charge payment → confirm order → notify fulfillment. PostgreSQL owns order records and inventory with row-level locking; Redis holds session state and cart contents with sub-millisecond access; RabbitMQ delivers fulfillment notifications with dead-letter handling; Elasticsearch serves product search and order history with faceted navigation. CQRS separates the write command path from the read model: the order read model is denormalized for fast order history queries without joining across domain tables.
Financial Ledger
An electronic health record (EHR) architecture built around strict auditability, HIPAA compliance, and append-only correctness. Clinical records are mutable by design (amendments, addenda) but corrections must be explicitly attributed, not silently overwritten. Event sourcing provides a reconstructable audit log; PostgreSQL row-level security enforces patient-level access control at the database layer; Kafka streams HL7 FHIR events to downstream clinical systems. The architecture must support breach detection, access auditing, and state reconstruction at any historical point: not just current state retrieval.
Write-Heavy Application
A high-rate device telemetry ingestion architecture designed for millions of devices emitting metrics at 1–60 second intervals. Kafka absorbs device writes as an ingestion buffer, decoupling device-facing ingest endpoints from the storage write path so that downstream storage pressure never propagates back to devices. TimescaleDB provides time-series storage with automatic chunk partitioning by time range, native compression, and continuous aggregate views for rollup queries. ClickHouse serves as the OLAP layer for device fleet analytics queries. Redis caches last-known device state (current readings per device) for real-time alerting queries that must not scan historical storage. Backpressure on the Kafka consumer side prevents storage write throughput from being overwhelmed by burst ingestion events from device reconnect storms.
Event-Driven System
A multi-channel notification delivery architecture that accepts upstream business events (order placed, payment received, comment posted, threshold alert triggered) and routes them to per-channel delivery workers (push via FCM/APNs, email via SendGrid, SMS via Twilio, in-app via WebSocket). Kafka carries raw business events from upstream producers. RabbitMQ handles per-channel fan-out with separate exchanges and queues per delivery channel, isolating email queue backlog from push notification delivery. PostgreSQL provides durable notification state tracking (sent, failed, bounced, suppressed). Redis enforces per-user rate limiting (notification frequency caps to prevent fatigue) and stores deduplication tokens to prevent duplicate sends across retry attempts. The inbox pattern on the consumer side ensures idempotent delivery even when Kafka produces duplicate events.
Analytics Pipeline
A metrics, logs, and traces ingestion and query platform built to absorb the telemetry output of a production system fleet: including the telemetry volume spikes that accompany the incidents the platform is meant to detect. ClickHouse stores metrics data with automatic time-based rollup via continuous materialized views; TimescaleDB provides complementary time-series storage for high-cardinality alert evaluation; Kafka buffers the ingestion stream against downstream write pressure, decoupling ingest acceptance rate from storage write throughput; Elasticsearch serves log full-text search and structured field filtering; Redis caches dashboard query results and active alert state for sub-100ms alert evaluation latency. The alert engine evaluates threshold and anomaly rules against pre-computed materialized views, not raw data, to bound alert evaluation cost independent of ingestion volume.
Event-Driven System
A video and audio streaming architecture where content ingestion triggers an async multi-variant transcoding pipeline, CDN delivery handles 95%+ of playback traffic, and Cassandra absorbs the write volume of per-user viewing history. Kafka decouples upload events from transcoding workers; MinIO stores raw and encoded assets; Redis maintains playback session state and view counters. The architecture must handle upload spikes without blocking delivery, and cache cold starts without cascading database load.