Compare Scenarios

Migration Step 1

Both scenarios define a migration step at this stage. AI Retrieval-Augmented Generation Platform: triggered by 'LLM responses requiring more factual accuracy or domain-spec'. ML Feature Serving Platform: triggered by 'Feature computation logic duplicated across 3+ model serving'.

Migration Step 2

Both scenarios define a migration step at this stage. AI Retrieval-Augmented Generation Platform: triggered by 'Document ingestion p99 > 500ms due to embedding API call in '. ML Feature Serving Platform: triggered by 'Regulatory or reproducibility requirement to re-run model pr'.

Migration Step 3

Both scenarios define a migration step at this stage. AI Retrieval-Augmented Generation Platform: triggered by 'Index scan range too large for per-query latency targets; te'. ML Feature Serving Platform: triggered by 'Semantic similarity search recall insufficient from BM25 ful'.

Retrieval quality metrics (MRR, NDCG) declining despite stable query volume; users reporting irrelevant context being surfaced; pgvector IVFFlat probes set below recommended value for current document count

Tier 1: Vector Index Recall Degradation: IVFFlat index not rebuilt after significant document additions; or probes too low for current index size. Recommended evolution: Schedule periodic index rebuilds triggered by document count growth (e.g., rebuild at 2x the document count present at last index build); increase ivfflat.probes to improve recall at cost of query latency; evaluate HNSW for recall-critical workloads .

PostgreSQL process memory > 8GB; OOM killer events on the database host; vector query p99 latency increasing as shared_buffers evicts vector index pages; pg_stat_bgwriter showing high buffers_clean rate

Tier 2: PostgreSQL Memory Pressure from Vector Operations: Vector index (HNSW or large IVFFlat) and embedding storage competing with relational data for shared_buffers. Recommended evolution: Increase PostgreSQL shared_buffers to 40% of available RAM; move vector tables to a dedicated tablespace on NVMe; partition large vector tables by document category to reduce per-query index scan range; evaluate dedicated pgvector replica for query isolation .

Feature store p99 rising from < 5ms to > 50ms immediately following a model deployment or pod scaling event; Cassandra or PostgreSQL feature store read QPS spiking 10–100x simultaneously with serving fleet restart; feature store connection pool exhaustion visible in application logs during the cold start window; Redis cache hit rate dropping to < 10% for the first 60 seconds after deployment

Tier 1: Cache Cold Start Latency Spike: Simultaneous cache cold start across all serving pods after deployment, converting sequential feature requests into a thundering herd against the backing feature store. Recommended evolution: Implement pre-warming: before a new model version receives traffic, a warm-up job runs inference requests for a representative sample of entity IDs, populating the Redis cache before the pod enters the serving fleet. Use probabilistic early cache refresh (fetch from backing store before TTL expiry when remaining TTL < 20% under high request rate) to prevent simultaneous TTL expiry on hot features. Stagger deployment rollout: deploy 10% of pods, wait for cache warm, then proceed to the next 10%. .

Model performance metrics (precision, recall, AUC) degrading without a corresponding input distribution shift; ClickHouse drift dashboard showing feature value distributions served online diverging from training population distributions; explicit skew audit (comparing offline training feature values against replayed online feature values for the same entity at the same timestamp) showing systematic differences in specific features

Tier 2: Training-Serving Skew Detection: Feature computation logic divergence between the offline training pipeline and the online serving pipeline: a feature transformation applied in training is not applied identically in serving. Recommended evolution: Enforce a single feature computation function registered per feature in a shared feature registry, executed by both the online pipeline and the offline training pipeline. The two paths must use the same code, not independently maintained implementations. Implement a skew monitoring job that computes a random sample of features using both paths and alerts on distribution divergence above a threshold. .