Post-Mortem Framework · Operational: GC Pressure

GC Pressure

SEV-3, Limited Impact

Isolated propagation · operational · Affects 0 scenario(s)

Severity Classification

Classified as PARTIAL based on failure mode severity.

Propagation Chain

Origin component

GC Pressure begins at the source component. Trigger: Heap utilization exceeds 80% causing G1GC to run Full GC.

Immediate (T+0) · Signal: Latency Spike

Blast Radius

GC pressure is isolated to a single JVM instance. However, stop-the-world pauses can cause the instance to appear unavailable to load balancers or cluster coordinators, triggering failover or shard rebalancing that increases load on remaining healthy instances. On Elasticsearch or Cassandra clusters, GC-induced apparent unavailability of one node can trigger secondary effects across the cluster.

Contributing Factors

Trigger Condition: Heap utilization exceeds 80% causing G1GC to run Full GCoperational

This operational trigger enables GC Pressure: Heap utilization exceeds 80% causing G1GC to run Full GC

Trigger Condition: Object allocation rate exceeds GC collection throughput rateoperational

This operational trigger enables GC Pressure: Object allocation rate exceeds GC collection throughput rate

Trigger Condition: Large query result sets loaded into memory simultaneouslyoperational

This operational trigger enables GC Pressure: Large query result sets loaded into memory simultaneously

Remediation Plan

ImmediateCheck JVM GC logs for pause duration and frequency: grep 'GC pause' gc.log | tai

Check JVM GC logs for pause duration and frequency: grep 'GC pause' gc.log | tail -50

Effort: Minutes to hours (on-call response)

ImmediateCheck heap utilization: JMX, Prometheus JVM metrics, or jstat -gcutil <pid>

Check heap utilization: JMX, Prometheus JVM metrics, or jstat -gcutil <pid>

Effort: Minutes to hours (on-call response)

ImmediateIf heap is > 80% utilized under load, increase Xmx or add replicas to reduce per

If heap is > 80% utilized under load, increase Xmx or add replicas to reduce per-instance load

Effort: Minutes to hours (on-call response)

Short-TermUpgrade to ZGC or Shenandoah for heap sizes above 4GB

ZGC and Shenandoah collectors perform most work concurrently with application threads, achieving sub-millisecond pauses at heap sizes of 8–32GB. Enable with -XX:+UseZGC (JDK 15+ for production readiness). Requires JDK 11+ (Shenandoah) or JDK 15+ (ZGC production).

Effort: 1 day to 1 week

Short-TermTune heap size to maintain allocation headroom

Set -Xmx to leave sufficient headroom for live set + allocation buffer. A heap at 85% utilization during normal operation has no headroom for spikes. Rule of thumb: live set should not exceed 50–60% of Xmx.

Effort: 1 day to 1 week

Short-TermEnable GC logging and set up pause-duration alerting

-Xlog:gc*:file=gc.log:time,level,tags captures all GC events with timestamps. Parse logs for pause durations exceeding your SLO (e.g., alert on pauses > 200ms). GC log analysis is the primary diagnostic tool for GC pressure investigation.

Effort: 1 day to 1 week

Short-TermAdd alerting for documented detection signals

Configure alerts for: latency spike, cpu saturation. Set thresholds to fire at 70% of critical level to allow response before full failure.

Effort: 1–3 days

Long-TermReduce object allocation rate at application level

Profile with async-profiler or JFR to identify allocation hotspots. Common fixes: streaming large result sets instead of buffering, object pooling for frequent allocations, reducing intermediate collection copies in hot paths.

Effort: 1–4 sprints

This post-mortem framework is derived from structured architecture knowledge. It provides an evidence-grounded starting point, not a substitute for a live incident review conducted by the team closest to the system. Adjust remediation priorities based on actual runtime observations.