Compare Scenarios

Migration Step 1

Both scenarios define a migration step at this stage. API Gateway Platform: triggered by 'PostgreSQL hot path query p99 > 2ms under sustained request '. ML Feature Serving Platform: triggered by 'Feature computation logic duplicated across 3+ model serving'.

Migration Step 2

Both scenarios define a migration step at this stage. API Gateway Platform: triggered by 'Rate limit enforcement allowing requests above the configure'. ML Feature Serving Platform: triggered by 'Regulatory or reproducibility requirement to re-run model pr'.

Migration Step 3

Both scenarios define a migration step at this stage. API Gateway Platform: triggered by 'First Redis instance crash causing 100% gateway error rate f'. ML Feature Serving Platform: triggered by 'Semantic similarity search recall insufficient from BM25 ful'.

Redis command latency p99 > 0.5ms; gateway hot path p99 exceeding 2ms with Redis as the bottleneck (not upstream service); Redis CPU > 60% sustained; Lua script execution visible in SLOWLOG at > 0.1ms frequency

Tier 1: Redis Rate Limit Throughput: Single Redis instance processing all rate limit Lua scripts serially for all tenants across all gateway replicas. Recommended evolution: Shard rate limit counters across Redis Cluster nodes by hashing tenant_id to a cluster slot; this distributes Lua script execution across nodes proportional to tenant count; ensure tenant_id-keyed counters use hash tags ({tenant_id}) so all keys for a tenant land on the same slot and Lua scripts can operate on them atomically; do not use Redis Cluster without testing Lua script compatibility against your cluster topology first .

Tenant reports that rate limit increase takes > 30 seconds to take effect across all gateway replicas; configuration change audit log shows primary PostgreSQL write completing, but gateway replicas still routing to old backend endpoints beyond the expected cache TTL window

Tier 2: Configuration Propagation Latency: Local in-process cache TTL too long, or cache invalidation signal (Redis pub/sub or Kafka) not reaching all replicas. Recommended evolution: Implement configuration change notification via Redis pub/sub: PostgreSQL configuration writes also publish a config_invalidated event to a Redis channel; each gateway replica subscribes to this channel and flushes the affected local cache key on receipt; this reduces propagation latency from TTL duration to sub-second pub/sub delivery without eliminating the local cache that protects Redis from per-request configuration lookups .

Feature store p99 rising from < 5ms to > 50ms immediately following a model deployment or pod scaling event; Cassandra or PostgreSQL feature store read QPS spiking 10–100x simultaneously with serving fleet restart; feature store connection pool exhaustion visible in application logs during the cold start window; Redis cache hit rate dropping to < 10% for the first 60 seconds after deployment

Tier 1: Cache Cold Start Latency Spike: Simultaneous cache cold start across all serving pods after deployment, converting sequential feature requests into a thundering herd against the backing feature store. Recommended evolution: Implement pre-warming: before a new model version receives traffic, a warm-up job runs inference requests for a representative sample of entity IDs, populating the Redis cache before the pod enters the serving fleet. Use probabilistic early cache refresh (fetch from backing store before TTL expiry when remaining TTL < 20% under high request rate) to prevent simultaneous TTL expiry on hot features. Stagger deployment rollout: deploy 10% of pods, wait for cache warm, then proceed to the next 10%. .

Model performance metrics (precision, recall, AUC) degrading without a corresponding input distribution shift; ClickHouse drift dashboard showing feature value distributions served online diverging from training population distributions; explicit skew audit (comparing offline training feature values against replayed online feature values for the same entity at the same timestamp) showing systematic differences in specific features

Tier 2: Training-Serving Skew Detection: Feature computation logic divergence between the offline training pipeline and the online serving pipeline: a feature transformation applied in training is not applied identically in serving. Recommended evolution: Enforce a single feature computation function registered per feature in a shared feature registry, executed by both the online pipeline and the offline training pipeline. The two paths must use the same code, not independently maintained implementations. Implement a skew monitoring job that computes a random sample of features using both paths and alerts on distribution divergence above a threshold. .