Tenant Noisy Neighbor
criticalSummary
In a multi-tenant system, one tenant's high resource consumption: query load, connection count, write rate, or storage I/O: degrades database or service performance for all other tenants sharing the same infrastructure, violating the implicit isolation guarantee that a shared-infrastructure SaaS product implies.
Description
Multi-tenant SaaS systems run many customers on shared infrastructure to achieve cost efficiency. Shared resources: database connection pools, WAL/binlog bandwidth, buffer cache, CPU, disk I/O: are finite. When one tenant consumes a disproportionate share, others are starved.
Common noisy neighbor scenarios:
1. Query blast: a tenant runs a poorly-optimized report query that performs a full
table scan on a large tenant-data table. The scan fills the buffer cache with
its own data, evicting frequently-accessed rows for all other tenants. Other
tenants' queries suddenly see high latency as their data must be re-read from disk.
2. Write burst: a tenant imports a million records via bulk upload, saturating WAL
bandwidth and causing replication lag across all other tenants sharing the same
primary.
3. Connection exhaustion: a tenant's application has a connection leak or aggressive
pool sizing. Their connections consume the shared pool, blocking connection
acquisition for other tenants' requests.
4. Lock contention: a tenant's migration or batch job holds locks on shared tables
(in a shared-schema model) or creates long-running transactions that hold
shared resources.
Row-level-security shared-schema architectures are most susceptible: all tenants are in the same tables, so a full-table scan by one tenant is a full-scan of all tenants' data, loading the entire buffer cache with one tenant's dataset.
Characteristics
Triggers
- ·Large tenant bulk data import or batch migration without rate limiting
- ·Analytics or reporting query on production database without query timeout
- ·Application bug causing connection leak in one tenant's integration
- ·Runaway process in a tenant's automation or API client
Detection Signals
Mitigation Strategies
Apply per-tenant connection limits and statement_timeout (PostgreSQL: SET statement_timeout = '30s') on each tenant's connection. Long-running queries are automatically terminated before they saturate resources. Combine with per-tenant query QPS limits enforced at the connection proxy layer.
Identify tenants above a size or activity threshold and migrate them to dedicated database instances. This is the tiered isolation model: most tenants on shared, power users on dedicated. Eliminates the noisy neighbor problem for the most impactful tenants.
Route reporting and analytics queries to a read replica dedicated to analytics workloads. The primary and other replicas serving production traffic are isolated from the buffer cache pollution caused by large analytical scans.
Recovery Steps
- 1.Identify the noisy tenant using per-tenant query logs, pg_stat_activity, or tenant-tagged metrics
- 2.Terminate or rate-limit the noisy tenant's queries immediately
- 3.Apply statement_timeout and connection limits to the identified tenant
- 4.Assess whether the tenant requires migration to dedicated infrastructure
- 5.Implement per-tenant resource monitoring to detect future violations before they impact others
Estimated recovery time: Immediate relief after terminating the noisy query (seconds). Resource recovery (buffer cache, connection pool) normalizes within 1–5 minutes. Long-term isolation via dedicated infrastructure takes days to weeks depending on migration complexity.
Affected Systems
Patterns
Technologies
Basis
Noisy neighbor in multi-tenant SaaS systems is well-documented in Shopify, Notion, Salesforce, and Heroku engineering blog posts; per-tenant resource isolation approaches are described in PostgreSQL RLS documentation and database proxy (PgBouncer, ProxySQL) configuration guides
Run This Failure
Blast radius analysis for this failure mode within each scenario that carries it.
Related Architecture Knowledge
Inbound: affects this entity
Marketplace mixed workloads are vulnerable to tenant noisy neighbor when high-volume sellers concentrate write activity on shared infrastructure, degrading performance for other sellers.
Full relationship →Tenant isolation partitions resources between tenants so that one tenant's workload cannot consume resources allocated to others, eliminating the noisy neighbor problem.
Full relationship →Used In Architecture Scenarios
Multi-Tenant SaaS
A multi-tenant API gateway providing authentication, distributed rate limiting, request routing, payload transformation, and per-tenant usage analytics for API publishers. The hot path: authentication check, rate limit evaluation, and routing decision: must complete in under 1ms using Redis-only data structures to avoid proxying latency dominating upstream service response time. PostgreSQL stores tenant configuration, subscription plans, and API key definitions. Kafka receives API usage events for downstream billing and analytics. Configuration changes (rate limit updates, routing rule edits) must propagate to all gateway replicas without restart.
Multi-Tenant SaaS
A multi-tenant developer tooling platform providing CI/CD pipeline execution, log aggregation, code analysis, and dependency scanning across isolated tenant organizations. Tenant isolation is the primary correctness constraint: a security boundary violation between tenants is a critical incident, not a performance event. PostgreSQL row-level security enforces data isolation; Redis manages job queues and distributed locks; Elasticsearch indexes pipeline log output for search; Kafka delivers webhook events to tenant-registered endpoints; MinIO stores pipeline artifacts. Resource quota enforcement prevents any single tenant's burst from affecting others.
Multi-Tenant SaaS
A multi-tenant SaaS architecture where multiple customers are served from a shared deployment, with PostgreSQL row-level security providing logical tenant isolation, Redis delivering per-tenant caching, and connection pooling managing the aggregate connection demand across tenant workloads. Tenant isolation, resource fairness, and operational simplicity are the three competing forces this architecture must balance.