Compare Scenarios

Migration Step 1

Both scenarios define a migration step at this stage. IoT Telemetry Ingestion Platform: triggered by 'PostgreSQL write p99 > 100ms at sustained device fleet load;'. Two-Sided Marketplace Platform: triggered by 'Checkout failures from payment provider unavailability causi'.

Migration Step 2

Both scenarios define a migration step at this stage. IoT Telemetry Ingestion Platform: triggered by 'Alerting system query latency > 500ms due to TimescaleDB que'. Two-Sided Marketplace Platform: triggered by 'Listing search p99 > 1s; faceted navigation (category + pric'.

Migration Step 3

Both scenarios define a migration step at this stage. IoT Telemetry Ingestion Platform: triggered by 'Multi-device aggregate queries (fleet-wide max/min/avg over '. Two-Sided Marketplace Platform: triggered by 'Domain teams stepping on each other's schema migrations; dat'.

TimescaleDB write latency p99 > 50ms for batch INSERT operations; pg_stat_activity showing wait events on WAL flush; TimescaleDB active chunk autovacuum running continuously; Kafka consumer group lag for storage writers growing steadily at baseline (non-storm) load

Tier 1: TimescaleDB Write Throughput Ceiling: TimescaleDB single-node write throughput ceiling (~50k–100k rows/second depending on row width and chunk size configuration). Recommended evolution: Tune TimescaleDB chunk_time_interval to match write cadence (smaller chunks = faster compression, lower WAL amplification per chunk); enable native compression on chunks older than 1 hour to reduce on-disk footprint; add a dedicated NVMe volume for WAL separate from data directory; consider TimescaleDB multi-node for horizontal write distribution across data nodes .

Kafka consumer group lag jumping from baseline (<100k) to >10M messages within minutes; Kafka broker disk write rate elevated; TimescaleDB write thread pool fully saturated; Redis last-known-value update latency acceptable but historical storage significantly behind real-time; device reconnect event visible in device authentication logs correlating with lag spike

Tier 2: Kafka Consumer Lag from Reconnect Storm: Kafka consumer pool sized for steady-state throughput, not burst from device reconnect storm; insufficient storage writer parallelism for burst absorption. Recommended evolution: Pre-scale storage writer consumer replicas before anticipated high-risk windows (maintenance events, regional failovers); implement burst-aware consumer scaling using consumer group lag as the autoscale signal; tune Kafka consumer max.poll.records to batch storage INSERTs into TimescaleDB for higher per-consumer throughput (target 500–1000 rows per INSERT batch rather than single-row inserts) .

Redis cache miss spike visible in monitoring; PostgreSQL query rate spiking for listing reads despite stable write volume; p99 listing API latency > 500ms during traffic spike events

Tier 1: Viral Listing Thundering Herd: Cache TTL expiry on hot listings during peak traffic: all concurrent requests bypass cache simultaneously. Recommended evolution: Implement staggered TTL jitter on listing cache entries; use probabilistic early refresh (refresh before TTL expiry when remaining TTL < 20% and request rate is high); implement single-flight/request coalescing at the application layer to collapse concurrent cache misses into a single database read .

Saga compensation events appearing in order event log; checkout p99 > 2s; pg_locks showing contended rows on inventory_reservations table; idempotency key conflicts increasing in payment service logs

Tier 2: Checkout Saga Contention: Concurrent checkout transactions competing for the same inventory rows; saga timeout thresholds too aggressive. Recommended evolution: Increase inventory reservation table partition count; tune saga step timeout to 2x the observed p99 for each step under load; implement a per-listing checkout serialization queue to prevent N concurrent sagas competing for the same inventory .