Compare Scenarios

Migration Step 1

Both scenarios define a migration step at this stage. Developer Tools Platform: triggered by 'First noisy neighbor incident where one tenant's CI burst de'. Audit and Compliance Platform: triggered by 'Compliance audit finding that audit records were modified af'.

Migration Step 2

Both scenarios define a migration step at this stage. Developer Tools Platform: triggered by 'Kafka publish failures rolling back pipeline completion tran'. Audit and Compliance Platform: triggered by 'Compliance report generation taking > 5 minutes against Post'.

Migration Step 3

Both scenarios define a migration step at this stage. Developer Tools Platform: triggered by 'Log search returning results from other tenants' pipelines d'. Audit and Compliance Platform: triggered by 'SIEM consumer lag causing it to fall behind retention window'.

Redis job queue depth > 1000 correlated with a single tenant identifier; other tenants reporting p99 job start time > 60 seconds; tenant-level queue metrics showing one tenant holding > 50% of in-flight worker slots

Tier 1: Job Queue Tenant Noisy Neighbor: Shared Redis queue with shared worker pool allowing one tenant to monopolize available capacity. Recommended evolution: Implement per-tenant queue lanes in Redis (separate key namespaces per tenant, e.g., jobs:{tenant_id}:{priority}); implement a weighted fair scheduler at the worker dispatch layer that reads from tenant queues in round-robin order with priority weighting; cap the number of concurrently executing jobs per tenant to the tenant's quota, not to the total available worker count .

DDL migration duration > 10s on pipeline_runs, jobs, or artifacts tables; migration deployment causing timeout errors for active CI pipeline API calls during the deployment window; pg_locks showing AccessExclusiveLock held by ALTER TABLE statement

Tier 2: PostgreSQL Schema Migration Lock: High-volume tables requiring locking DDL changes during deployments with concurrent tenant activity. Recommended evolution: Adopt zero-downtime migration patterns exclusively: add columns with nullable defaults first (no table lock in PostgreSQL 11+), then backfill, then add constraints via NOT VALID followed by VALIDATE CONSTRAINT in a separate transaction; use pg_repack for table rewrites; never run concurrent index creation without CONCURRENTLY on any table with > 1M rows .

PostgreSQL write p99 > 20ms with low connection count; pg_stat_activity showing transactions serialized on the same partition's chain-tip read; ingestion throughput plateauing well below hardware limits; auto_explain showing sequential scan on audit_events for "SELECT hash FROM audit_events ORDER BY id DESC LIMIT 1"

Tier 1: Integrity Chain Write Serialization: Per-partition chain-tip read before each insert serializing concurrent audit writers. Recommended evolution: Introduce partition-level chain sequence tables: a single row per partition tracking the current chain tip with an advisory lock, eliminating the full table read. Alternatively, shard the integrity chain by source system or tenant, accepting per-shard chains rather than a single global chain. Use PostgreSQL INSERT ... RETURNING with sequence-assigned IDs to eliminate the pre-insert read entirely, deferring chain hash computation to an async integrity sealer that appends hashes in order without blocking the write path. .

Compliance investigator queries returning in > 30s; PostgreSQL showing high sequential scan counts on audit_events partitions; investigator-facing API p99 > 10s; pg_stat_statements showing actor_id-scoped queries without partition pruning in the query plan

Tier 2: Actor Query Full-Partition Scan: Missing secondary index table for actor_id and resource_id lookup paths across time-partitioned audit data. Recommended evolution: Build a secondary index table audit_events_by_actor(actor_id, event_time, event_id) populated synchronously on insert. Accept the additional write per event as the cost of O(log n) actor-scoped queries. Alternatively, route actor-scoped queries to ClickHouse where columnar storage makes actor_id filters efficient without a secondary B-tree index. .