Compare Scenarios

Migration Step 1

Both scenarios define a migration step at this stage. IoT Telemetry Ingestion Platform: triggered by 'PostgreSQL write p99 > 100ms at sustained device fleet load;'. Write-Heavy Transactional Platform: triggered by 'Dual-write inconsistency events observed in production: DB w'.

Migration Step 2

Both scenarios define a migration step at this stage. IoT Telemetry Ingestion Platform: triggered by 'Alerting system query latency > 500ms due to TimescaleDB que'. Write-Heavy Transactional Platform: triggered by 'Primary write CPU > 70% sustained during peak windows; WAL v'.

Migration Step 3

Both scenarios define a migration step at this stage. IoT Telemetry Ingestion Platform: triggered by 'Multi-device aggregate queries (fleet-wide max/min/avg over '. Write-Heavy Transactional Platform: triggered by 'Audit completeness requirements grow beyond point-in-time ba'.

TimescaleDB write latency p99 > 50ms for batch INSERT operations; pg_stat_activity showing wait events on WAL flush; TimescaleDB active chunk autovacuum running continuously; Kafka consumer group lag for storage writers growing steadily at baseline (non-storm) load

Tier 1: TimescaleDB Write Throughput Ceiling: TimescaleDB single-node write throughput ceiling (~50k–100k rows/second depending on row width and chunk size configuration). Recommended evolution: Tune TimescaleDB chunk_time_interval to match write cadence (smaller chunks = faster compression, lower WAL amplification per chunk); enable native compression on chunks older than 1 hour to reduce on-disk footprint; add a dedicated NVMe volume for WAL separate from data directory; consider TimescaleDB multi-node for horizontal write distribution across data nodes .

Kafka consumer group lag jumping from baseline (<100k) to >10M messages within minutes; Kafka broker disk write rate elevated; TimescaleDB write thread pool fully saturated; Redis last-known-value update latency acceptable but historical storage significantly behind real-time; device reconnect event visible in device authentication logs correlating with lag spike

Tier 2: Kafka Consumer Lag from Reconnect Storm: Kafka consumer pool sized for steady-state throughput, not burst from device reconnect storm; insufficient storage writer parallelism for burst absorption. Recommended evolution: Pre-scale storage writer consumer replicas before anticipated high-risk windows (maintenance events, regional failovers); implement burst-aware consumer scaling using consumer group lag as the autoscale signal; tune Kafka consumer max.poll.records to batch storage INSERTs into TimescaleDB for higher per-consumer throughput (target 500–1000 rows per INSERT batch rather than single-row inserts) .

PgBouncer wait_queue > 0 sustained; application p99 write latency rising faster than PostgreSQL p99; pool_mode=transaction showing >80% utilization

Tier 1: Connection Pool Saturation: PgBouncer pool_size too small for write concurrency profile. Recommended evolution: Increase PgBouncer pool_size incrementally; profile transaction duration to right-size pool; consider separate pools for write-heavy and read-only workloads .

PostgreSQL checkpoint_completion_target warnings in logs; wal_buffers flushing more than once per second; pg_stat_bgwriter shows checkpoints_req rising; write p99 > 20ms without query explanation

Tier 2: WAL and Checkpoint Pressure: Write rate exceeding PostgreSQL's WAL flush and checkpoint throughput. Recommended evolution: Tune checkpoint_completion_target to 0.9; increase wal_buffers to 64MB; move PostgreSQL WAL to a dedicated NVMe volume separate from data directory .