Compare Scenarios

Migration Step 1

Both scenarios define a migration step at this stage. Geospatial Tracking Platform: triggered by 'PostGIS proximity query p99 > 100ms under concurrent fleet t'. Observability Platform: triggered by 'Prometheus storage limits reached at production metric volum'.

Migration Step 2

Both scenarios define a migration step at this stage. Geospatial Tracking Platform: triggered by 'Location update API p99 > 200ms correlated with geofence cou'. Observability Platform: triggered by 'Elasticsearch indexing pressure causing log rejection (HTTP '.

Migration Step 3

Both scenarios define a migration step at this stage. Geospatial Tracking Platform: triggered by 'PostgreSQL location_history table exceeding 100GB with query'. Observability Platform: triggered by 'Alert evaluation latency > 1s causing missed alert firing wi'.

Redis used_memory > 75% of maxmemory; Redis evictions appearing in INFO stats; GEOSEARCH returning stale or missing entity positions; Redis OOM errors in application logs during fleet expansion events; proximity query latency increasing above 10ms baseline

Tier 1: Redis Geospatial Memory Pressure: Redis memory exhausted by unbounded geospatial entity growth without entity expiry or cleanup. Recommended evolution: Implement entity-scoped Redis key TTL tied to the last received update timestamp. Entities that have not sent a position update in > 5 minutes are expired from Redis automatically (Redis EXPIRE on the ZSET entry using a per-entity auxiliary key pattern, since ZSET members do not support per-member TTL natively). Alternatively, introduce a background reconciliation job that removes entities from the live position surface after an inactivity threshold. Shard the geospatial index across multiple Redis instances by geographic region using consistent hashing on the region key. .

TimescaleDB write p99 > 50ms; WAL volume > 200MB/minute sustained; disk I/O utilization > 80% on TimescaleDB data volume; chunk creation log entries during fleet expansion events correlated with write latency spikes; TimescaleDB worker queue depth growing during ingestion bursts

Tier 2: TimescaleDB Write Throughput Ceiling: TimescaleDB hypertable chunk write throughput saturated by high-frequency location update volume; chunk creation DDL causing write stalls during expansion. Recommended evolution: Tune TimescaleDB chunk_time_interval to match the ingestion rate: smaller chunks (1-hour intervals instead of 1-day) reduce per-chunk write volume but increase chunk creation frequency. Use timescaledb-parallel-copy for bulk historical ingestion. Move the TimescaleDB WAL to a dedicated NVMe volume. Introduce write batching at the application layer: buffer 500ms of position updates per entity and write as a single multi-row INSERT, reducing the per-update overhead from N single-row INSERTs to N/batch_size batch INSERTs. .

ClickHouse part merge frequency increasing; dashboard queries timing out on metrics with high label cardinality; ClickHouse system.metrics showing active_parts count elevated; new metric instrumentation causing sudden storage growth disproportionate to fleet size; query_log showing metrics queries scanning full column segments without pruning

Tier 1: Metric Cardinality Budget Exceeded: Unbounded label cardinality generating millions of distinct time series that exceed ClickHouse part merge capacity and query planner pruning effectiveness. Recommended evolution: Enforce a cardinality budget at ingestion: before a metric is accepted, evaluate the distinct value count of each label dimension against a per-dimension limit (e.g., max 100 distinct values for any single label key). Reject or rewrite metrics that exceed the budget: rewrite user_id labels to user_cohort or drop them entirely. Implement a cardinality analysis dashboard showing the top 10 highest-cardinality metric series sorted by storage cost. ClickHouse distributed table partitioning by metric name reduces the impact of a single high-cardinality metric on global query performance. .

Kafka log topic consumer lag growing > 1 million messages during incident periods; Elasticsearch indexing throughput metrics showing queue buildup; incident post-mortems noting that relevant log records were not available in the search interface during the incident; log consumer memory pressure from unbounded batch accumulation

Tier 2: Log Volume Spike Exceeding Consumer Throughput: Log Kafka consumer sized for normal throughput; unable to drain the spike volume produced during incident-driven log floods. Recommended evolution: Size the log consumer for 10x normal throughput, not 1x: observability platform capacity must be planned for the incident scenario, not the steady state. Implement consumer autoscaling triggered by consumer lag metric: when Kafka consumer lag exceeds a threshold, add consumer instances automatically. Implement log sampling at the producer side for DEBUG and INFO level messages during identified spike periods : preserve all ERROR and WARN messages, sample INFO at 10%, sample DEBUG at 1%. This bounds the worst-case log volume without sacrificing diagnostic signal. .