Compare Scenarios

Migration Step 1

Both scenarios define a migration step at this stage. Social Feed Platform: triggered by 'PostgreSQL timeline read query p99 > 500ms; query plan for "'. AI Retrieval-Augmented Generation Platform: triggered by 'LLM responses requiring more factual accuracy or domain-spec'.

Migration Step 2

Both scenarios define a migration step at this stage. Social Feed Platform: triggered by 'Fan-out worker queue lag during celebrity post events > 5 mi'. AI Retrieval-Augmented Generation Platform: triggered by 'Document ingestion p99 > 500ms due to embedding API call in '.

Migration Step 3

Both scenarios define a migration step at this stage. Social Feed Platform: triggered by 'Redis memory utilization approaching 80% of a single node; R'. AI Retrieval-Augmented Generation Platform: triggered by 'Index scan range too large for per-query latency targets; te'.

Kafka consumer group lag for fan-out worker group growing steadily; feed propagation latency (time from post write to follower feed update) exceeding 30s p95; Redis write rate on feed keys elevated but not saturated; post activity rate normal

Tier 1: Fan-Out Worker Queue Backlog: Fan-out worker pool undersized for burst post activity or a celebrity post creating a sustained high-fan-out event. Recommended evolution: Add fan-out worker replicas; implement fan-out cost routing: route high-follower-count fan-out events to a dedicated high-cost worker pool with separate Kafka consumer group and Redis write quota; use follower count threshold (e.g., >50k followers) as the routing decision. Monitor fan-out cost per post as a first-class metric. .

Redis memory utilization > 75%; eviction rate rising; cache miss rate on feed reads increasing; cold feed read fallback queries appearing in PostgreSQL slow query log; feed read p99 > 100ms despite Redis being online

Tier 2: Redis Feed Memory Ceiling: Redis feed list storage approaching memory limit; feed items being evicted before TTL; or feed list cap set too high for available memory. Recommended evolution: Reduce feed list cap from current value toward 100–150 items; increase Redis cluster capacity or shard feed keys by user_id range across multiple Redis primaries; implement tiered feed storage: hot recent items in Redis, older items fetched from PostgreSQL on demand with explicit product UX affordance .

Retrieval quality metrics (MRR, NDCG) declining despite stable query volume; users reporting irrelevant context being surfaced; pgvector IVFFlat probes set below recommended value for current document count

Tier 1: Vector Index Recall Degradation: IVFFlat index not rebuilt after significant document additions; or probes too low for current index size. Recommended evolution: Schedule periodic index rebuilds triggered by document count growth (e.g., rebuild at 2x the document count present at last index build); increase ivfflat.probes to improve recall at cost of query latency; evaluate HNSW for recall-critical workloads .

PostgreSQL process memory > 8GB; OOM killer events on the database host; vector query p99 latency increasing as shared_buffers evicts vector index pages; pg_stat_bgwriter showing high buffers_clean rate

Tier 2: PostgreSQL Memory Pressure from Vector Operations: Vector index (HNSW or large IVFFlat) and embedding storage competing with relational data for shared_buffers. Recommended evolution: Increase PostgreSQL shared_buffers to 40% of available RAM; move vector tables to a dedicated tablespace on NVMe; partition large vector tables by document category to reduce per-query index scan range; evaluate dedicated pgvector replica for query isolation .