Compare Scenarios

Migration Step 1

Both scenarios define a migration step at this stage. AI Retrieval-Augmented Generation Platform: triggered by 'LLM responses requiring more factual accuracy or domain-spec'. API Gateway Platform: triggered by 'PostgreSQL hot path query p99 > 2ms under sustained request '.

Migration Step 2

Both scenarios define a migration step at this stage. AI Retrieval-Augmented Generation Platform: triggered by 'Document ingestion p99 > 500ms due to embedding API call in '. API Gateway Platform: triggered by 'Rate limit enforcement allowing requests above the configure'.

Migration Step 3

Both scenarios define a migration step at this stage. AI Retrieval-Augmented Generation Platform: triggered by 'Index scan range too large for per-query latency targets; te'. API Gateway Platform: triggered by 'First Redis instance crash causing 100% gateway error rate f'.

Retrieval quality metrics (MRR, NDCG) declining despite stable query volume; users reporting irrelevant context being surfaced; pgvector IVFFlat probes set below recommended value for current document count

Tier 1: Vector Index Recall Degradation: IVFFlat index not rebuilt after significant document additions; or probes too low for current index size. Recommended evolution: Schedule periodic index rebuilds triggered by document count growth (e.g., rebuild at 2x the document count present at last index build); increase ivfflat.probes to improve recall at cost of query latency; evaluate HNSW for recall-critical workloads .

PostgreSQL process memory > 8GB; OOM killer events on the database host; vector query p99 latency increasing as shared_buffers evicts vector index pages; pg_stat_bgwriter showing high buffers_clean rate

Tier 2: PostgreSQL Memory Pressure from Vector Operations: Vector index (HNSW or large IVFFlat) and embedding storage competing with relational data for shared_buffers. Recommended evolution: Increase PostgreSQL shared_buffers to 40% of available RAM; move vector tables to a dedicated tablespace on NVMe; partition large vector tables by document category to reduce per-query index scan range; evaluate dedicated pgvector replica for query isolation .

Redis command latency p99 > 0.5ms; gateway hot path p99 exceeding 2ms with Redis as the bottleneck (not upstream service); Redis CPU > 60% sustained; Lua script execution visible in SLOWLOG at > 0.1ms frequency

Tier 1: Redis Rate Limit Throughput: Single Redis instance processing all rate limit Lua scripts serially for all tenants across all gateway replicas. Recommended evolution: Shard rate limit counters across Redis Cluster nodes by hashing tenant_id to a cluster slot; this distributes Lua script execution across nodes proportional to tenant count; ensure tenant_id-keyed counters use hash tags ({tenant_id}) so all keys for a tenant land on the same slot and Lua scripts can operate on them atomically; do not use Redis Cluster without testing Lua script compatibility against your cluster topology first .

Tenant reports that rate limit increase takes > 30 seconds to take effect across all gateway replicas; configuration change audit log shows primary PostgreSQL write completing, but gateway replicas still routing to old backend endpoints beyond the expected cache TTL window

Tier 2: Configuration Propagation Latency: Local in-process cache TTL too long, or cache invalidation signal (Redis pub/sub or Kafka) not reaching all replicas. Recommended evolution: Implement configuration change notification via Redis pub/sub: PostgreSQL configuration writes also publish a config_invalidated event to a Redis channel; each gateway replica subscribes to this channel and flushes the affected local cache key on receipt; this reduces propagation latency from TTL duration to sub-second pub/sub delivery without eliminating the local cache that protects Redis from per-request configuration lookups .