Hot Partition
criticalContainedcapacity failure · isolated propagation
Blast Radius
0%
0/13 nodes
Time to Detect
Seconds to minutes with per-partition throughput and lag monitoring: a single partition's metrics diverge sharply from the fleet. 10 to 60 minutes if noticed only through user-facing latency, because the failure is partial and only hot-partition traffic degrades while averages stay healthy.
Preventive Mitigations
3
Confidence
StrongFailure Cascade
(no topology nodes mapped for this failure mode)
This failure mode is referenced in the scenario knowledge but has no connected topology nodes. Impact scope is scenario-level, all components should be considered potentially affected.
Severity at this step: critical
Detection Signals
Recovery time estimate: Hours to days for structural fixes (partition-count increase, key-schema change). Immediate tactical relief is possible by adding consumers or raising provisioned throughput on the hot partition, but neither removes the underlying skew.
Mitigation Checklist(3 preventive, 2 reactive)
Replace the hot key with a composite (user_id + time_bucket) so one logical entity spreads across partitions over time. It reduces concentration at the cost of ordered-per-entity consumption, since consumers must correlate across buckets.
Append a small random prefix or suffix so one hot key spreads across N partitions: salted_key = original_key + "_" + random(0, N). This is the only fix for a single dominant key. The cost moves to reads, which must fan out to all N salted keys and merge, and it breaks per-key ordering, so it suits write-heavy or independent-event keys more than ordered streams.
Identify high-volume keys in advance (system accounts, top-N products) and route them to a dedicated high-throughput topic or shard instead of the shared pool. The cost is key-classification logic in the producer or router and a separate consumer path.
More partitions spread moderate, many-key skew more evenly. In Kafka partition count can only grow, and existing messages keep their assignment. This helps when several keys are hot; it does nothing for a single dominant key, which still maps to one partition.
Consistent hashing distributes data across nodes using a hash ring, ensuring that load is spread uniformly across all nodes regardless of key distribution. Virtual nodes further smooth out variance, reducing the likelihood of any single node becoming a hot partition.
Affected Systems
Workloads
Blast radius analysis is derived from structured topology and failure mode knowledge. It models structural propagation patterns, not measured production behavior. Actual incident scope depends on runtime conditions, traffic, and recovery actions in place at the time of failure.