DBRaven
Blast Radius Analysis · Healthcare Records Platform

Deadlock

criticalContained

concurrency failure · isolated propagation

Blast Radius

5%

1/19 nodes

Time to Detect

The database logs a deadlock the moment it resolves one (PostgreSQL: ERROR: deadlock detected, with the conflicting statements listed when log_lock_waits is on; MySQL: the LATEST DETECTED DEADLOCK section of SHOW ENGINE INNODB STATUS). Application monitoring catches the spike within 30 to 60 seconds only if deadlock errors are tracked as a distinct category. Many deadlock spikes go unnoticed because the application retries successfully and only the database log records that anything happened.

Preventive Mitigations

2

Confidence

Strong

Impacted Components

Immediate

PostgreSQL

primary datastore · Directly connected to failure mode 'Deadlock' via risk propagation path in the topology.

Failure Cascade

1

PostgreSQL

Deadlock (isolated propagation) directly affects these components.

Severity at this step: critical

Detection Signals

Error Rate SpikeAlert

Recovery time estimate: Each individual deadlock resolves automatically: PostgreSQL within one deadlock_timeout, InnoDB effectively immediately. Application retry adds roughly 100 to 500ms per attempt. Bringing a sustained deadlock rate down is a code change on a release cycle, and the rate typically drops as soon as consistent lock ordering ships.

Mitigation Checklist(2 preventive, 2 reactive)

SELECT FOR UPDATE SKIP LOCKED for queue workloadspreventsmedium

For work-queue access where any available row will do, SKIP LOCKED skips rows already locked by other workers instead of waiting, so competing workers never form a wait cycle. It changes semantics rather than adding cost: a skipped row is simply handled by someone else, so it fits "claim any pending row" but not "this specific row must be processed here", and it does nothing for multi-row transactional updates.

Consistent lock orderingpreventslow

Establish a canonical order for acquiring locks on multiple rows, for example sorting by primary key before locking. If every path acquires locks in the same order, a cycle cannot form. The cost is discipline that lives in application code and code review, and the limit is coverage: it prevents lock-order-inversion deadlocks but not gap-lock or foreign-key-cascade cycles, and it is unworkable when the lock set is discovered mid-transaction rather than known up front.

Reduce transaction scope and lock hold timemedium

Shorter transactions hold locks for less time, shrinking the window a cycle can form in. Move external calls and computation outside the transaction, and acquire locks as late as possible. The cost is refactoring, and the limit is that a business operation genuinely needing several locks held together cannot shed them.

Retry with exponential backoff and jitterlow

On a deadlock error, roll back and retry after a short randomized delay (roughly 100 to 500ms with jitter), bounded to 3 to 5 attempts before surfacing the error. Backoff spaces the contenders out so they stop reforming the same cycle. This does not reduce the deadlock rate, it absorbs it: cost moves into added tail latency, and the transaction must be safe to replay, since a retried write runs again from the start.

Affected Systems

Workloads

Event StreamingMixed OLTP (SaaS Core)Write-Heavy Transactional

Technologies

PostgreSQL

Blast radius analysis is derived from structured topology and failure mode knowledge. It models structural propagation patterns, not measured production behavior. Actual incident scope depends on runtime conditions, traffic, and recovery actions in place at the time of failure.