WAL Saturation

PostgreSQL WAL (Write-Ahead Log) generation rate exceeds wal_buffers flush capacity or downstream replica/WAL archive bandwidth, causing write transactions to stall waiting for WAL flush and replication lag to grow unboundedly.

Every write operation in PostgreSQL generates WAL records. WAL is the authoritative record of all changes; a transaction cannot commit until its WAL records are durably flushed to disk. WAL generation saturates when the rate of write operations produces more WAL bytes per second than the WAL disk can absorb at the required latency.

The WAL pipeline has multiple potential saturation points. First, wal_buffers: shared memory buffers that hold WAL records before fsync. The default is -1, which auto-sizes wal_buffers to roughly 1/32 of shared_buffers, bounded between a minimum of 64kB and a maximum of one WAL segment (16MB by default); this auto-sizing default has been stable since PostgreSQL 9.1, and an explicit numeric setting overrides it. If transactions generate WAL faster than the background WAL writer flushes wal_buffers, transactions block waiting to write WAL. Second, WAL disk I/O: the WAL disk must sustain the WAL write bandwidth plus fsync IOPS. For a workload generating 500MB/s WAL (1 million 500-byte updates/second), the WAL device must sustain 500MB/s sequential write throughput: a GP3 EBS volume's max is 1,000 MB/s (125 MB/s baseline), so this workload can run but a GP2 volume would saturate. Third, WAL archiving: pg_wal_archive must copy WAL segments to S3, NFS, or another target faster than segments are generated. If archiving falls behind, pg_wal fills with unarchived segments and can exhaust disk space.

For streaming replication, WAL saturation manifests as replication lag: the replica WAL receiver cannot apply WAL faster than it arrives. At 500MB/s WAL generation and a replica with 100MB/s network bandwidth, replication lag grows at 400MB/s : 1.44TB/hour. The replica cannot catch up until the primary write rate drops below the replica network bandwidth.

The failure is particularly dangerous for CDC systems (Debezium, logical replication slots): a lagging replication slot retains WAL on the primary's pg_wal directory. If the consumer falls behind, pg_wal grows without bound until it fills the primary's disk: at which point the primary will not be able to generate new WAL and will halt all write operations.

PostgreSQL's WAL design is a single stream doing two jobs at once: the same records that let a crashed instance replay to a consistent state are also what streaming replication ships to standbys. MySQL/InnoDB splits this into two independent logs. The InnoDB redo log records physical page changes for crash recovery only, sized via innodb_redo_log_capacity (MySQL 8.0.30 and later; earlier versions size it via innodb_log_file_size times innodb_log_files_in_group). The binary log (binlog) is a separate, logical record of statements or row changes used for replication and point-in-time recovery, controlled independently (max_binlog_size, sync_binlog, binlog_expire_logs_seconds). These are not the same log: they are written by different code paths and each can independently become the bottleneck. A redo log undersized for the write rate forces synchronous checkpointing that stalls commits; a binlog that replicas or CDC consumers (Debezium reading the binlog, not the redo log) cannot keep up with grows lag the same way a lagging PostgreSQL replication slot does, but it is a different file with different retention controls than the redo log.

LSM engines take a narrower approach again. Cassandra's commit log, like RocksDB's write-ahead log, exists only to make the current unflushed memtable durable across a crash; once the memtable is flushed to an SSTable, the corresponding commit log segments are no longer needed for recovery and are recycled. Neither log is what carries data to other replicas. Cassandra replicates at write time through its own replication protocol (the coordinator writes to each replica node responsible for the key, per the keyspace's replication factor), not by shipping the commit log downstream, so commit log pressure and replication lag are separate concerns there in a way they are not under PostgreSQL's single-stream design.