Write Amplification Cascade
criticalContainedcapacity failure · linear propagation
Blast Radius
0%
0/19 nodes
Time to Detect
10–30 minutes with I/O utilisation and WAL generation monitoring. The failure degrades gradually rather than spiking instantly; by the time application latency is affected, I/O has been saturated for some time.
Preventive Mitigations
1
Confidence
StrongFailure Cascade
(no topology nodes mapped for this failure mode)
This failure mode is referenced in the scenario knowledge but has no connected topology nodes. Impact scope is scenario-level, all components should be considered potentially affected.
Severity at this step: critical
Detection Signals
Recovery time estimate: Minutes to hours depending on intervention. Dropping unused indexes takes effect immediately. Storage tier upgrades require a brief maintenance window. Structural write amplification reduction (index cleanup, partitioning) takes days to implement and validate.
Mitigation Checklist(1 preventive, 4 reactive)
For high-volume append-only workloads (event logs, analytics), route to ClickHouse or Cassandra, which use LSM trees with sequential I/O and significantly lower write amplification for append patterns (WAF ~ 3–10x vs. PostgreSQL's 20–100x on update-heavy workloads).
PostgreSQL table partitioning by time range or hash distributes write I/O across partition-level pages, potentially benefiting from OS-level I/O parallelism. Each partition has its own set of index pages; hot-partition writes are distributed across the newer partition's pages.
Each secondary index maintained during UPDATE adds one index page write per UPDATE row. Identify indexes with zero or near-zero scans via pg_stat_user_indexes.idx_scan. Drop indexes unused in the last 30 days. Reducing from 8 to 4 indexes can halve write amplification.
Set checkpoint_completion_target = 0.9 and checkpoint_timeout = 15min (default 5min). checkpoint_completion_target defaults to 0.5 on PostgreSQL 13 and earlier but already defaults to 0.9 from PostgreSQL 14 onward, so confirm the running version and current value before assuming this is a needed change. Spreading checkpoint dirty page flushing over a longer period reduces instantaneous I/O peaks. Does not reduce total I/O; trades peaks for a more even sustained rate. See checkpoint_amplification for the full mechanism.
Move from GP3 (3,000–16,000 IOPS) to io2 (up to 64,000 IOPS) or local NVMe. Increases the I/O ceiling, buying headroom. Does not fix the write amplification factor; defers the problem to higher load levels.
Affected Systems
Workloads
Blast radius analysis is derived from structured topology and failure mode knowledge. It models structural propagation patterns, not measured production behavior. Actual incident scope depends on runtime conditions, traffic, and recovery actions in place at the time of failure.