Long-Running Transaction Bloat
criticalSummary
A transaction that holds a snapshot open far longer than it does real work pins the database's xmin horizon, so VACUUM cannot reclaim any dead tuple newer than the snapshot, anywhere in the cluster. Bloat accumulates, freezing stalls (raising transaction-ID wraparound risk), and rows the transaction modified stay locked. One forgotten transaction degrades the whole database, not just the tables it touched.
Description
PostgreSQL transactions are meant to be short. A transaction that stays open for seconds to minutes while holding a snapshot causes several failure modes at once, and the common thread is that it holds back the xmin horizon.
The xmin horizon and bloat. Under MVCC, an UPDATE does not overwrite a row; it writes a new version and marks the old one dead, and a DELETE marks the row dead. A dead tuple can be reclaimed only once no snapshot could still need to see it. The oldest snapshot in the system defines that boundary, the xmin horizon, and VACUUM will not remove any tuple that became dead after it. A transaction that holds one snapshot for its whole life, a long-running query, or a REPEATABLE READ or SERIALIZABLE transaction, pins the horizon at the xmin it began with. From that moment VACUUM can reclaim nothing newer, regardless of which tables the long transaction actually touched, because the guarantee is database-wide. Dead tuples pile up, tables and their indexes grow, and the extra pages mean more I/O per query and worse plans. Its backend_xmin is visible in pg_stat_activity, and the dead tuples it strands show up as n_dead_tup in pg_stat_user_tables.
Transaction-ID wraparound. PostgreSQL's transaction IDs are 32-bit and must be frozen before they age past roughly two billion, or their tuples would appear to come from the future and become invisible. Freezing is VACUUM's job, and VACUUM can only freeze tuples older than the xmin horizon. A long-running transaction that pins the horizon therefore blocks freezing from advancing. If the oldest unfrozen age crosses autovacuum_freeze_max_age, autovacuum launches aggressive anti-wraparound runs; if the horizon stays pinned and the age keeps climbing toward the limit, PostgreSQL stops accepting new write transactions to protect the data ("database is not accepting commands to avoid wraparound data loss"). The transaction contributes to wraparound by blocking freezing, not by using up IDs itself.
Lock queue buildup. If the long transaction also wrote rows, it holds their row locks until it ends. Writers that need those rows queue behind it for its full duration, hold their own connections while they wait, and can exhaust the connection pool. This is the contention path, covered under lock_contention; a long transaction is one of its most common causes.
DDL blocking. An ALTER TABLE or similar DDL needs a table-level lock incompatible with ordinary traffic. A long-running transaction on the table blocks the DDL from acquiring it, and because the lock request is exclusive, every query arriving after it queues behind it. A routine migration turns into a traffic-blocking event, which is the interaction detailed under schema_migration_lock.
An idle-in-transaction session is the usual culprit behind all of these: an application or ORM opens a transaction, runs a statement, then stalls on external work or a code path that never commits, holding its snapshot and any locks it already took while doing no work at all.
Characteristics
Triggers
- ·Application code holds a transaction open while calling an external API or waiting on I/O
- ·ORM begins a transaction automatically and a code path fails to commit or roll back
- ·Batch work runs as one large transaction instead of many small ones
- ·A REPEATABLE READ or SERIALIZABLE transaction, or a long analytical query, holds one snapshot for a long time
- ·An error path leaves a transaction open (idle in transaction) on exception
Detection Signals
Mitigation Strategies
Set idle_in_transaction_session_timeout (PostgreSQL 9.6+) so a connection left idle inside a transaction past the threshold (for example 30s) is terminated automatically. It bounds the most common cause, the forgotten open transaction, without relying on every code path to be correct. The cost is that a genuinely long legitimate transaction must run its work rather than sit idle, or it will be cut off too.
Set lock_timeout before running DDL so that if it cannot take its lock within N seconds it fails fast instead of queuing and blocking every statement behind it, then retry in a quieter window. This bounds the DDL-blocking symptom; it does not shorten the long transaction, so cost moves to retrying the migration rather than absorbing an outage.
Keep a transaction spanning only the database work it must protect: fetch external data before BEGIN, commit, then do external I/O. This shrinks hold time so the transaction stops pinning the horizon and stops queuing writers. The cost is refactoring code that currently wraps a transaction around slow work.
Recovery Steps
- 1.Find the offender: SELECT pid, state, xact_start, backend_xmin, query FROM pg_stat_activity ORDER BY xact_start
- 2.Terminate a stuck transaction: SELECT pg_terminate_backend(pid) for the oldest xact_start / idle-in-transaction session
- 3.Let VACUUM reclaim the stranded tuples once the horizon advances; run VACUUM (ANALYZE) on the worst tables
- 4.Confirm bloat is clearing: check n_dead_tup in pg_stat_user_tables or measure with pgstattuple
- 5.Set idle_in_transaction_session_timeout and add xact_start age alerting to prevent recurrence
Estimated recovery time: Terminating the transaction is immediate and lets the horizon advance at once. Reclaiming accumulated bloat takes minutes to hours of VACUUM depending on volume; after weeks of unreclaimed bloat a VACUUM FULL may be needed, which itself takes an ACCESS EXCLUSIVE lock and blocks the table for its duration.
Affected Systems
Patterns
Technologies
Basis
The xmin horizon, VACUUM reclamation and freezing, idle_in_transaction_session_timeout, and anti-wraparound protection are all documented PostgreSQL behavior; the mechanism linking a pinned horizon to bloat, blocked freezing, and wraparound risk is precise and reproducible.
Related Architecture Knowledge
Inbound: affects this entity
Event-sourced systems that open a database transaction for the full event application cycle create long-running transactions that prevent VACUUM from reclaiming MVCC dead tuples.
Full relationship →PostgreSQL's MVCC model prevents VACUUM from reclaiming dead tuples visible in any open transaction snapshot; long-running transactions cause table bloat and risk transaction ID wraparound.
Full relationship →Write-heavy transactional workloads are vulnerable to transaction bloat when transactions are held open during slow external calls, preventing PostgreSQL VACUUM from reclaiming dead tuples.
Full relationship →