Vector Databases and Embeddings
AdvancedHow vector embeddings enable semantic search, how HNSW and IVFFlat indexes make approximate nearest neighbor search practical at scale, and when to use pgvector versus a dedicated vector database.
Step 1 of 5
Vector Embeddings: Meaning in High-Dimensional Space
A vector embedding is a dense array of floating-point numbers that encodes the semantic meaning of a piece of content. A sentence embedding model (e.g., text-embedding-3-small from OpenAI) converts "database connection pooling" into a 1,536-dimensional vector like [0.023, -0.141, 0.872, ...]. Semantically similar sentences produce vectors that are close together in this high-dimensional space: measured by cosine similarity or dot product.
Why this matters: traditional B-tree indexes find exact matches. A B-tree on document text cannot find "all documents about connection management": it can only find exact string matches. A vector index finds documents whose embedding vectors are close to the query embedding vector, which corresponds to semantic similarity.
The practical use cases: RAG (Retrieval-Augmented Generation) pipelines retrieve relevant context from a knowledge base by finding document embeddings close to the query embedding. Recommendation engines find similar items by finding item embedding vectors close to the user's preference vector.
The storage cost: a 1,536-dimensional float32 vector is 6,144 bytes per document. One million documents = 6GB of vector data. 100 million documents = 600GB. The entire vector corpus must fit in memory for acceptable ANN performance: disk-based ANN is 100x slower than in-memory ANN.
Vector storage requirements at different corpus sizes
Key Takeaways
- →A 1536-dim float32 embedding is 6KB: 1M documents = 6GB of vector data before index overhead
- →Vector indexes must reside in RAM for sub-100ms queries: plan memory budget before choosing vector store
- →Semantic similarity ≠ keyword match: vectors find conceptually related content, not just string-matching content