DBRaven
Start Review →
Event Streams

Kafka Consumer Lag

Intermediate

How Kafka consumer groups track offsets, how consumer lag accumulates when processing falls behind production, how uneven partition assignment creates lag skew, and how to instrument and remediate consumer lag before it causes data pipeline delays.

Step 1 of 5

Consumer Group: Partition Assignment

Kafka assigns partitions to consumer group members so that each partition is owned by exactly one consumer at a time. With 3 partitions and 3 consumers, each consumer owns 1 partition. With 6 partitions and 3 consumers, each owns 2. The throughput ceiling is bounded by the number of partitions: adding consumers beyond the partition count provides no additional throughput.

Producer

Producer

1,200 msg/s

Topic Partitions

Partition 0

45 msgs

Partition 1

42 msgs

Partition 2

48 msgs

Consumer Groups

order-processor (consumer-0)

1 memberlag: 45

order-processor (consumer-1)

1 memberlag: 42

order-processor (consumer-2)

1 memberlag: 48

3 partitions, 3 consumers: each consumer owns exactly 1 partition

Key Takeaways

  • →Partition count is the hard ceiling on consumer parallelism: more consumers than partitions are idle
  • →Each partition is owned by exactly one consumer in a group at any time
1 / 5