Search-Heavy Content Platform

Summary

A content platform architecture centered on Elasticsearch for full-text search, faceted navigation, and ranked results, with PostgreSQL as the transactional source of truth and Redis for session management and hot content caching. WAL-based CDC maintains index freshness by streaming PostgreSQL changes into Elasticsearch asynchronously. The core tension is between search index freshness, query performance, and index maintenance cost under high write volume.

Problem Statement

Content platforms, job boards, e-commerce catalogs, and media libraries require full-text search with faceted navigation that PostgreSQL full-text search cannot serve at scale. Elasticsearch provides the query capabilities: inverted indexes, relevance scoring, aggregation buckets: but adds a second system that must stay consistent with the PostgreSQL source of truth. A stale index serves outdated results; an over-indexed catalog degrades write throughput and index segment count. The indexing pipeline must be decoupled from the write path to prevent search indexing latency from degrading transactional writes.