RabbitMQ
3.13.xSummary
AMQP-based message broker with flexible routing (exchanges, queues, bindings), acknowledgment-based delivery, and per-message TTL and dead-letter queue support.
Primary Use Case
Work queue distribution across consumers, RPC-over-messaging patterns, flexible message routing with fan-out and topic-based dispatch, and low-to-medium volume async task processing.
Workload Fit
Strengths
Best for
- ·Work queue distribution with acknowledgment-based at-least-once consumer processing
- ·RPC-over-messaging patterns requiring correlation ID and reply queues
- ·Flexible routing topologies (topic exchanges, fanout, header-based routing)
- ·Low-to-medium volume async processing where Kafka's complexity is unwarranted
Excels when
- ·Message delivery semantics (at-most-once, at-least-once) are more important than throughput
- ·Per-message routing logic varies and exchange bindings model the domain well
- ·Dead-letter queues and TTL-based expiry are required by the application
- ·Consumer counts are moderate and scale horizontally with queue consumers
Architectural advantages
- ·AMQP routing model (exchanges + bindings) enables flexible dispatch without application-layer routing code
- ·Dead-letter queues provide built-in error handling for failed message processing
- ·Publisher confirms and consumer acknowledgments enable reliable delivery semantics
- ·Low operational overhead relative to Kafka for low-to-medium volume workloads
When to Avoid
Avoid when
- ·Event streaming at millions of messages/second: RabbitMQ throughput ceiling is below Kafka's
- ·Consumers need to replay historical messages: RabbitMQ deletes acknowledged messages
- ·Long-term message retention (hours to days) is required: use Kafka's log-based storage
Common misuses
- ·Using RabbitMQ as an event streaming platform requiring message replay: messages are deleted on ack
- ·Publishing without consumer acknowledgments: message loss on consumer crash is silent
- ·Routing high-throughput event streams through RabbitMQ: throughput ceiling is well below Kafka
Consistency & Transactions
Scaling
Read scalability
Multiple consumers per queue distribute work horizontally. Queue mirroring (classic) or quorum queues (current) provide HA across nodes. Consumer throughput scales with consumer count up to queue throughput ceiling.
Write scalability
Single-node throughput is bounded by disk I/O and message durability settings. Publisher confirms add per-message round-trip latency. Lazy queues reduce memory pressure by persisting messages to disk immediately.
Failure Behavior
Known failure modes
- ·Memory alarm: broker suspends publishers when RAM usage exceeds threshold (default 40%)
- ·Queue depth explosion: unprocessed message accumulation exhausts broker RAM and disk
- ·Split-brain in classic mirrored queues during network partition
- ·Publisher confirm timeout cascades: slow confirms trigger application-level retry storms
Bottlenecks
- ·Single-node throughput ceiling for durable messages with publisher confirms
- ·Memory alarm suspension blocks all publishers when queue depth grows too fast
- ·Long-running quorum queue elections during node failure cause brief publish unavailability
Degradation patterns
- ·Queue depth accumulation causes memory alarm and gradual publisher suspension
- ·Slow consumers cause queue depth growth, increasing memory pressure and latency
- ·Classic mirrored queue synchronization after partition recovery is I/O intensive
Recovery considerations
- ·Quorum queues provide crash-safe durability without split-brain risk: prefer over classic mirrored queues
- ·Messages with publisher confirms and consumer acks survive broker restart
- ·Queue recovery after node failure requires quorum (majority of replicas must be available)
Operational Pitfalls
- ·Not using quorum queues for durable, HA workloads: classic mirrored queues have known split-brain behavior
- ·Not setting consumer prefetch count: consumers pull all queued messages, starving other consumers
- ·Publishing without publisher confirms: message loss on broker crash is silent without confirms
- ·Ignoring queue depth metrics: unbounded accumulation causes memory alarm and broker shutdown
Architecture Guidance
Common topology roles
Migration notes
- ·From RabbitMQ to Kafka: fundamental semantic change: persistent ordered log vs transient queue; consumer redesign required
- ·To Amazon SQS: AMQP semantics differ; flexible routing (exchanges/bindings) must be replaced with application-layer routing
- ·Adding dead-letter queues retroactively: configure dead-letter exchange on existing queues without message loss
Advisor Guidance
When: scenario requires event replay or consumer catch-up from historical messages
RabbitMQ deletes acknowledged messages: use Kafka for replay-capable event streaming
When: scenario uses classic mirrored queues for HA
Migrate to quorum queues: classic mirrored queues have known split-brain behavior under network partition
When: scenario has high_throughput_writes exceeding 50k messages/second
RabbitMQ throughput ceiling may be insufficient: evaluate Kafka for sustained high-throughput event streams
Comparison Factors
routing flexibility
High: exchange/binding model supports complex routing topologies natively
throughput
Medium: tens of thousands/second; significantly below Kafka for streaming workloads
message replay
None: acknowledged messages are deleted; no historical replay capability
operational complexity
Medium: lower than Kafka; quorum queues simplify HA management
Managed Cloud Options
Enables Patterns
Basis
Mature technology with well-documented operational behavior; widely deployed in mid-size systems
Learning Modules
Evolution Paths
Simulations
Used In Architecture Scenarios
Marketplace Platform
An e-commerce order lifecycle platform handling cart, checkout, payment, fulfillment, and returns across a mixed read/write workload where product discovery is read-heavy, checkout is write-transactional, and fulfillment is event-driven. The saga pattern orchestrates multi-step checkout: reserve inventory → charge payment → confirm order → notify fulfillment. PostgreSQL owns order records and inventory with row-level locking; Redis holds session state and cart contents with sub-millisecond access; RabbitMQ delivers fulfillment notifications with dead-letter handling; Elasticsearch serves product search and order history with faceted navigation. CQRS separates the write command path from the read model: the order read model is denormalized for fast order history queries without joining across domain tables.
Event-Driven System
A multi-channel notification delivery architecture that accepts upstream business events (order placed, payment received, comment posted, threshold alert triggered) and routes them to per-channel delivery workers (push via FCM/APNs, email via SendGrid, SMS via Twilio, in-app via WebSocket). Kafka carries raw business events from upstream producers. RabbitMQ handles per-channel fan-out with separate exchanges and queues per delivery channel, isolating email queue backlog from push notification delivery. PostgreSQL provides durable notification state tracking (sent, failed, bounced, suppressed). Redis enforces per-user rate limiting (notification frequency caps to prevent fatigue) and stores deduplication tokens to prevent duplicate sends across retry attempts. The inbox pattern on the consumer side ensures idempotent delivery even when Kafka produces duplicate events.
Event-Driven System
A social activity feed architecture where user actions (posts, likes, comments, follows) fan out asynchronously to follower timelines. Redis stores hot feed data as pre-materialized lists per user, enabling O(1) timeline reads for the 99th percentile of users. Kafka carries fan-out work to async workers that write to follower Redis keys. PostgreSQL is the durable store for the social graph, posts, and user content. The system uses a hybrid fan-out model: fan-out-on-write for users with fewer than ~10,000 followers (low fan-out cost), fan-out-on-read for high-follower celebrity accounts where pre-materialized fan-out would saturate workers and Redis write bandwidth.
Marketplace Platform
A two-sided marketplace architecture serving buyers, sellers, listings, transactions, search, and notifications from a shared infrastructure, where multiple independent domains must coordinate without tight coupling. Event sourcing captures every state transition; the saga pattern orchestrates multi-step transactions (create order, reserve inventory, charge payment, notify seller) with compensating transactions for partial failures. Kafka decouples domain event publication from consumption; RabbitMQ handles notification fanout; Elasticsearch serves listing search; Redis caches listing display and session state.