Relationship · Mitigates
Source: Pattern·Target: Failure Mode
Summary
Consistent hashing distributes data across nodes using a hash ring, ensuring that load is spread uniformly across all nodes regardless of key distribution. Virtual nodes further smooth out variance, reducing the likelihood of any single node becoming a hot partition.
Evidence
- ·Murmur3 consistent hash distributes uniformly across the hash space: hot key mitigation vs. range-based sharding
- ·{'Virtual nodes (vnodes) reduce variance': 'with 256 vnodes per node, load imbalance is <5% vs 20-30% with single tokens'}
- ·Amazon Dynamo paper (2007) documents consistent hashing as the core mechanism for avoiding hot partitions
- ·CDN providers use consistent hashing to distribute popular content evenly across edge nodes
- ·Cassandra's vnodes enabled proportional token assignment without manual token management
Operational Context
- ·Even with consistent hashing, access skew (celebrity rows) requires write shardinghash-based distribution only helps with key diversity
- ·Monitor per-node throughput metrics: consistent hashing distributes keys, but access patterns can still concentrate load
- ·Virtual node count must be set at cluster creation: changing it requires data rebalancing
Tradeoffs
- ·Consistent hashing prevents range queries: scan operations require scatter-gather across all nodes
- ·Uniform distribution assumes uniform access patterns: does not help if 90% of requests target the same logical entity
- ·Adding nodes with consistent hashing moves O(K/N) keys (K keys, N nodes): lower disruption than rehashing
Evidence grounding
Grounded, 5 supporting itemsConsistent hashing was specifically designed to solve the uneven distribution problem in distributed systems. Documented in the original Karger et al. (1997) paper and widely deployed in Cassandra, Dynamo, and CDN systems.