Noisy Neighbor
degradedContainedcapacity failure · fan out propagation
Blast Radius
0%
0/22 nodes
Time to Detect
Latency spikes are visible within seconds. Identifying the noisy neighbor as the root cause requires correlating victim latency spikes with a co-tenant's resource spike: typically requires infrastructure-level metrics (node CPU, disk I/O) correlated with per-pod or per-tenant metrics. This investigation may take minutes to hours.
Preventive Mitigations
3
Confidence
StrongFailure Cascade
(no topology nodes mapped for this failure mode)
This failure mode is referenced in the scenario knowledge but has no connected topology nodes. Impact scope is scenario-level, all components should be considered potentially affected.
Severity at this step: degraded
Detection Signals
Recovery time estimate: Immediate once the noisy workload is terminated or throttled: seconds. Permanent isolation requires infrastructure changes: hours to days.
Mitigation Checklist(3 preventive, 2 reactive)
Move high-value or latency-sensitive tenants to dedicated database instances. Accepts higher operational cost for isolation guarantees. The standard progression for SaaS multi-tenancy as usage grows.
Move analytical workloads, batch jobs, or high-CPU services to dedicated node pools using Kubernetes node selectors and taints/tolerations. This eliminates co-tenancy between workloads with different resource profiles.
CPU limits cap a pod's burst capacity at the node level. Memory limits cause OOM kill instead of unbounded memory growth. Setting requests = limits creates a Guaranteed QoS class: the pod receives guaranteed CPU and is not throttled by noisy neighbors.
Tenant isolation enforces resource boundaries that prevent a single tenant's workload from impacting shared infrastructure used by other tenants.
PostgreSQL statement_timeout kills queries that exceed a time limit. This prevents a single expensive query from holding shared I/O for an extended period. Set conservatively per workload type (e.g., 5s for OLTP, 300s for analytics connections).
Affected Systems
Workloads
Blast radius analysis is derived from structured topology and failure mode knowledge. It models structural propagation patterns, not measured production behavior. Actual incident scope depends on runtime conditions, traffic, and recovery actions in place at the time of failure.