ai_embedding_lookup vulnerable to memory_pressure

ai embedding lookupVulnerable Tomemory pressure oom

Source: Workload·Target: Failure Mode

Summary

AI embedding lookup workloads require the entire vector index to reside in RAM for acceptable latency. When the index size grows beyond available memory, the OS begins paging the HNSW graph to disk, causing query latency to degrade from milliseconds to seconds and eventually OOM-killing the process.

Evidence

·HNSW index for 1M 1536-dim float32 vectors requires 6GB for raw vectors + 3-5GB for graph structure = 9-11GB total
·When HNSW graph edges exceed RAM, random graph traversal triggers page faults : query latency spikes 10-100x
·{'Faiss documentation explicitly states': 'IVFFlat requires the centroid index + one cluster in RAM minimum'}
·pgvector HNSW index is loaded into PostgreSQL's shared_buffers: insufficient shared_buffers causes heavy disk I/O
·Weaviate's operational guide warns against deploying vector collections whose index exceeds 80% of available RAM

Operational Context

·Size instances for 2x the vector index size to leave headroom for OS, application, and query buffers
·Monitor RSS (resident set size) of the vector store process: steady growth indicates index expansion toward the RAM ceiling
·Use quantization (PQ, SQ8) to reduce vector size by 4-8x at the cost of 1-2% recall degradation

Tradeoffs

·Quantization reduces memory by 4-8x but degrades recall slightly: evaluate recall impact before deploying
·IVFFlat has lower memory than HNSW but requires training (cluster computation) when adding new vectors
·Horizontal scaling (shard vectors across instances) reduces per-node memory at the cost of scatter-gather query overhead

Evidence grounding

Grounded, 5 supporting items

Vector index memory requirements are documented in pgvector, Faiss, and Pinecone engineering guides. The memory cliff (index exceeds RAM) is a well-known failure mode for growing vector stores.