Publisher-Subscriber
matureSummary
Producers publish events to a topic or channel without knowing subscribers. All interested subscribers receive every event, decoupling producers from consumers at both the semantic and temporal level.
Problem
Multiple downstream consumers need to react to the same events, but the producer should not need to know who they are or maintain direct connections to each one. Adding a new consumer must not require modifying the producer.
Description
In pub-sub, the publisher declares that an event occurred (e.g., "order.placed") and broadcasts it to a topic. Any subscriber that has declared interest in that topic receives a copy. Multiple subscribers receive the same event independently : this is the core distinction from competing consumers, where only one consumer processes each message.
Kafka durable pub-sub: subscribers are consumer groups. Each consumer group receives all events independently. Events are retained on disk for a configurable retention period, so a subscriber added after the fact can replay the full history. This is the most operationally robust pub-sub model for event-driven architectures.
Redis pub-sub: ephemeral. No message persistence. If a subscriber is offline when an event is published, that event is permanently lost to that subscriber. Suitable for real-time fanout (live dashboards, notification delivery) where missing an event is acceptable. Not suitable for event sourcing or audit trails.
SNS/SQS fan-out: SNS topic fans out to multiple SQS queues. Each SQS queue is a competing-consumer pool for one subscriber group. Combines pub-sub semantics (multiple independent consumers) with durable queuing per subscriber. The standard AWS pattern for reliable multi-consumer fanout.
Temporal decoupling: a publisher does not need a subscriber to be online to publish. Events accumulate in the topic. This allows subscribers to be deployed, updated, or restarted independently without the publisher being aware.
Tradeoffs
Producer has zero knowledge of subscribers; adding consumers requires no producer change
Kafka enables full event history replay; Redis pub-sub provides none
Kafka requires partition management, consumer group monitoring, and retention tuning
Each subscriber independently tracks offset; each must handle duplicate delivery
Kafka log retention scales with event volume and retention period
When to use
Multiple independent consumers need to react to the same event
Pub-sub delivers each event to all subscribers without producer modification
Producers and consumers evolve independently and must be decoupled
Temporal and semantic decoupling is the primary value of pub-sub
Event replay is needed for new consumers or data recovery
Kafka-backed pub-sub with log retention supports replay; design for this if replay is a requirement
When not to use
Only one consumer should process each message
Use competing consumers if messages must be processed exactly once by one worker
Message delivery must survive broker restart with zero configuration overhead
Ephemeral pub-sub (Redis) loses messages on broker restart; use Kafka or SNS/SQS for durability
Operational Requirements
Configure topic retention period to match the longest expected consumer catchup window
A consumer offline for 48 hours must be able to replay 48 hours of events; retention must cover this
Monitor consumer group lag per topic-partition
Growing lag indicates a consumer cannot keep up with the publish rate
Assign unique consumer group IDs per independent subscriber
Shared group IDs cause competing-consumer semantics instead of pub-sub semantics
Define schema and versioning strategy for events
Multiple consumers on one topic creates coupling through the event schema; use schema registry for Kafka
Characteristics
Technologies
Canonical
Alternatives
Relationships
Evolves to
Complements
Basis
Core event-driven pattern with mature implementations across all major messaging platforms
Related Architecture Knowledge
Inbound: affects this entity
NATS provides sub-millisecond pub/sub messaging with subject hierarchy and wildcard subscriptions, enabling publish-subscribe communication between services.
Full relationship →