Use Two-Sided Marketplace Platform as the Foundational Architecture Pattern
Deterministic ADR derived from topology, simulation, and advisor intelligence for Two-Sided Marketplace Platform. Traceable to YAML knowledge entities.
Context
Marketplace platforms combine multiple distinct operational domains: catalog/listings, search/discovery, order/transaction, payments, notifications, seller operations: that must stay loosely coupled but operationally coordinated. A listing update must propagate to search without blocking the write path. A payment failure must trigger compensating inventory release without requiring synchronous rollback across services. A trending listing must serve thousands of concurrent reads without overloading the transactional store. The architecture must handle unpredictable traffic spikes (sale events, viral listings) while maintaining transaction consistency for financial operations. Primary operational risks include: Saga compensation cascade: a partial failure in the payment step leaves inventory reserved and order in intermediate state; if the compensation saga fails too, the system has inconsistent state with no automated resolution; Hot listing thundering herd: a viral listing receiving simultaneous traffic from a social media spike causes Redis cache miss on first eviction, and all concurrent requests hit PostgreSQL for listing data simultaneously; RabbitMQ notification queue backlog: a notification service outage causes the RabbitMQ queue to accumulate unbounded messages; when the service recovers, it replays the full backlog against a potentially rate-limited email/push provider.
Decision
We will adopt the **Two-Sided Marketplace Platform** architecture pattern. This is a expert-complexity architecture appropriate for teams at platform engineering team level or above. The advisor rates this pattern as 'advanced' operational maturity.
Rationale
A two-sided marketplace architecture serving buyers, sellers, listings, transactions, search, and notifications from a shared infrastructure, where multiple independent domains must coordinate without tight coupling. Event sourcing captures every state transition; the saga pattern orchestrates multi-step transactions (create order, reserve inventory, charge payment, notify seller) with compensating transactions for partial failures. Kafka decouples domain event publication from consumption; RabbitMQ handles notification fanout; Elasticsearch serves listing search; Redis caches listing display and session state. Core technology stack: postgresql, elasticsearch, redis, kafka, rabbitmq.
Accepted Tradeoffs
- ⚠Saga pattern adds ordering system resilience but requires designing compensating transactions for every failure scenario: the compensation logic must be as robust as the forward path
- ⚠Event sourcing provides complete order history and auditability but doubles write amplification for every order state transition; at high order volume this saturates WAL throughput
- ⚠Elasticsearch listing index is eventually consistent with PostgreSQL: price changes and availability updates have a propagation latency; this must be communicated in search UX (e.g., "prices may vary")
- ⚠RabbitMQ for notifications decouples delivery from business logic but requires queue depth monitoring and dead-letter configuration: unbounded queue growth is an operational incident
- ⚠Redis listing cache dramatically reduces read load during viral traffic spikes but requires TTL and invalidation strategy that keeps listing data fresh without cache poisoning
Risks
One partition (a database shard, a Kafka topic partition, a Redis hash slot) receives traffic so far above its peers that it saturates while the others sit idle. Aggregate capacity looks healthy, but the hot partition throttles or lags, and everything routed to it degrades. The cause is skew in how keys map to partitions, and the fix depends on whether the skew is spread across many keys or concentrated in one.
A failure or degradation in one service causes increased load, held resources, or error propagation in its callers, which in turn degrade their callers, until the failure front propagates through the entire dependency graph and brings down services with no direct dependency on the original failure point.
Concurrent writers to the same rows serialize behind each other's row locks, so latency is set not by the work a transaction does but by how long it waits for the writers ahead of it. On a hot row the queue depth, and therefore the tail latency, grows with concurrency while throughput flattens. Blocked writers hold connections open, so a single contended row can drain the connection pool as a secondary failure.
When a popular cached key expires or a service recovers from downtime, all requests that were waiting or arrive simultaneously miss the cache and hit the origin database concurrently, producing a request spike that can overwhelm the database within seconds.
Message queue or event stream consumer processing rate falls below producer write rate, causing consumer lag to grow unboundedly: eventually leading to increased end-to-end latency, producer backpressure, data expiry, or queue resource exhaustion.
Alternatives Considered
AI Retrieval-Augmented Generation Platform shares core technology (kafka, postgresql) with the chosen architecture but applies different structural patterns; Two-Sided Marketplace Platform is a better fit for the identified workload profile.
Analytics Data Platform shares core technology (kafka, postgresql) with the chosen architecture but applies different structural patterns; Two-Sided Marketplace Platform is a better fit for the identified workload profile.
API Gateway Platform shares core technology (kafka, postgresql) with the chosen architecture but applies different structural patterns; Two-Sided Marketplace Platform is a better fit for the identified workload profile.
Audit and Compliance Platform shares core technology (kafka, postgresql) with the chosen architecture but applies different structural patterns; Two-Sided Marketplace Platform is a better fit for the identified workload profile.
Scaling Thresholds
Signals indicating the architecture is approaching its scaling limits:
Tier 1: Viral Listing Thundering Herd
Signal: Redis cache miss spike visible in monitoring; PostgreSQL query rate spiking for listing reads despite stable write volume; p99 listing API latency > 500ms during traffic spike events
Evolution: Implement staggered TTL jitter on listing cache entries; use probabilistic early refresh (refresh before TTL expiry when remaining TTL < 20% and request rate is high); implement single-flight/request coalescing at the application layer to collapse concurrent cache misses into a single database read
Tier 2: Checkout Saga Contention
Signal: Saga compensation events appearing in order event log; checkout p99 > 2s; pg_locks showing contended rows on inventory_reservations table; idempotency key conflicts increasing in payment service logs
Evolution: Increase inventory reservation table partition count; tune saga step timeout to 2x the observed p99 for each step under load; implement a per-listing checkout serialization queue to prevent N concurrent sagas competing for the same inventory
Tier 3: Notification Queue Backlog
Signal: RabbitMQ queue depth > 100k messages; notification delivery latency > 5 minutes; downstream notification provider (SendGrid, FCM) rate limit errors in consumer logs; dead-letter queue receiving messages from retry exhaustion
Evolution: Add notification consumer replicas; implement consumer-side rate limiting against downstream provider quotas; tune RabbitMQ prefetch count to prevent consumer overload on recovery; implement dead-letter queue with manual review tooling
Tier 4: Domain Service Decomposition Pressure
Signal: Database connection pool exhausted by combination of checkout + search + listing writes all competing for the same PostgreSQL pool; single PostgreSQL primary CPU > 80% sustained; domain boundaries in code becoming unclear as direct table access crosses service lines
Evolution: Decompose into separate PostgreSQL databases per domain (listings, orders, payments, users) using the database-per-service pattern; each domain has its own connection pool; cross-domain data access goes through events, not direct database queries
Migration Path
Monolithic marketplace application with single database → Event-driven marketplace with Kafka + saga-based checkout flow
Checkout failures from payment provider unavailability causing full transaction rollback and user-facing errors; need for asynchronous notification delivery; listing search performance insufficient from PostgreSQL full-text search
PostgreSQL full-text search for listing discovery → Elasticsearch for listing search with CDC-based indexing
Listing search p99 > 1s; faceted navigation (category + price + location + rating) not supportable in PostgreSQL without full-table scans; ranking algorithm requiring feature vectors that PostgreSQL cannot efficiently support
Monolithic PostgreSQL serving all domain writes → Domain-separated databases with event-based cross-domain data propagation
Domain teams stepping on each other's schema migrations; database resource contention across domains (listing writes vs checkout transactions vs analytics); need to independently scale checkout volume without scaling listing read capacity
Operational Requirements
- Minimum team maturity: Platform Engineering Team: This scenario has expert operational complexity. It is recommended for Platform Engineering Team teams or higher.
- Runbooks and alerting for high-severity risks: 5 high-severity risks identified. Each requires a documented runbook, alerting threshold, and on-call response procedure before running in production.
- Event stream operations expertise: This architecture includes event stream infrastructure (Kafka, Kinesis, or similar). Operations requires consumer group management, partition assignment, dead-letter handling, and lag monitoring.
- Cache sizing and eviction policy configuration: Redis or equivalent cache requires correct maxmemory configuration, eviction policy selection (allkeys-lru is common), and cold-start warming strategy after restarts.