sharding introduces risk hot_partition: DBRaven

Source: Pattern·Target: Failure Mode

Summary

Sharding distributes data across partitions, but poor shard key selection concentrates traffic on a small number of shards. A hot partition receives disproportionate load, becomes a bottleneck, and degrades performance for all data on that shard.

Evidence

·Sequential shard keys (timestamps, auto-increment IDs) cause all writes to land on the most recent shard
·Low-cardinality shard keys (country, status) cause uneven distribution when data is skewed
·DynamoDB adaptive capacity documentation explicitly addresses hot partition as a primary failure mode
·Cassandra's virtual nodes partially mitigate hot partitions via consistent hash ring distribution
·Elasticsearch shard hot-spotting from poorly distributed document IDs is documented in production guides

Operational Context

·Monitor per-shard request rates: a hot shard is often 10-100x the average shard load
·High-cardinality shard keys (UUID, hash of user_id) distribute load evenly
·Artificial key padding (appending a random suffix to the key) spreads hot key traffic at the cost of scatter-gather reads

Tradeoffs

·Hash-based shard keys (good distribution) eliminate range query capability: all range queries become scatter-gather
·Detecting hot partitions requires per-shard metrics: aggregate metrics mask the problem
·Resharding to fix a hot partition is expensive: requires data migration across shards

Evidence grounding

Grounded, 5 supporting items

Hot partition is the most documented failure mode of sharding systems. Extensively documented in AWS DynamoDB post-mortems, Cassandra operational guides, and Elasticsearch production guides.