Use Financial Ledger Platform as the Foundational Architecture Pattern
Deterministic ADR derived from topology, simulation, and advisor intelligence for Financial Ledger Platform. Traceable to YAML knowledge entities.
Context
Financial systems cannot tolerate lost writes, phantom reads, or inconsistent account balances. A payment that debits one account must credit another within the same atomic transaction. An audit trail must capture every state transition, not just the current state. Concurrent balance updates must not interleave partial writes. The architecture must guarantee that every ledger entry is durable before acknowledging success to the caller, and that all downstream systems (notifications, analytics, reconciliation) receive every event at least once, deduplicating on a stable event ID. Primary operational risks include: Split-brain balance inconsistency: if the primary PostgreSQL fails during a transaction and a replica is promoted, uncommitted transactions may be partially applied on the replica if synchronous_commit is not set to remote_apply or higher; Schema migration lock on hot tables: a table-level lock on the accounts or ledger_entries table during a DDL migration blocks all concurrent transactions, causing a write queue buildup that can last minutes; Write amplification from event sourcing append: appending an event row for every state transition plus updating the materialized balance view doubles the I/O per transaction; under burst load this saturates WAL throughput.
Decision
We will adopt the **Financial Ledger Platform** architecture pattern. This is a expert-complexity architecture appropriate for teams at platform engineering team level or above. The advisor rates this pattern as 'advanced' operational maturity.
Rationale
A strict consistency financial architecture for double-entry accounting, payment processing, and audit trail management where every debit must have a corresponding credit, every state transition must be ordered and durable, and the complete history must be reconstructable without gaps. Event sourcing provides an append-only audit log; the outbox pattern guarantees at-least-once downstream event delivery without distributed two-phase commit (idempotent consumers deduplicate on event ID for effectively-once processing); PostgreSQL provides ACID guarantees for ledger entry atomicity. The primary architectural strength is: The outbox pattern eliminates split-brain between a database write and a message broker publish by writing both the domain record…. The outbox pattern eliminates split-brain between a database write and a message broker publish by writing both the domain record and the outbox event in a single ACID transaction, ensuring events are published if and only if the database write committed. Key trade-off: Adds ~1ms write overhead per transaction for the outbox INSERT. Operational note: Outbox table requires a relay process: this is an additional operational component to monitor. Evidence: Atomic write to both business table and outbox table in one transaction: no window for inconsistency. Core technology stack: postgresql, kafka.
Architectural Strengths
- ✓The outbox pattern eliminates split-brain between a database write and a message broker publish by writing both the domain record…
- ✓Financial transaction workloads benefit from event sourcing because the event log provides an immutable audit trail, enables…
- ✓Write-heavy transactional workloads that emit downstream events (order placed, payment captured) benefit from the outbox pattern…
- ✓PostgreSQL serves as a capable event store for moderate event volumes, leveraging JSONB payloads, UNIQUE constraints for…
- ✓Kafka's durable, ordered, append-only log is the canonical infrastructure for an event store at scale
- ✓Event sourcing naturally produces a normalized write model (the event log) that CQRS separates from purpose-built read models (projections)
Accepted Tradeoffs
- ⚠Event sourcing provides perfect audit completeness but doubles write amplification per transaction: every state change writes both an event row and updates the projected balance state
- ⚠Synchronous replication (synchronous_commit = remote_apply) eliminates split-brain risk but doubles write latency (each commit waits for replica acknowledgment)
- ⚠The outbox pattern guarantees at-least-once event delivery (consumers must dedupe on event ID for effectively-once) but adds a synchronous outbox INSERT to every financial transaction, increasing per-transaction write cost
- ⚠Two-phase commit enables cross-service atomicity but introduces coordinator failure risk and hold-open transactions that block lock release during coordinator unavailability
- ⚠CQRS separation improves read scalability but means the query model is eventually consistent with the command model: financial dashboards may show slightly stale balances
Risks
Concurrent writers to the same rows serialize behind each other's row locks, so latency is set not by the work a transaction does but by how long it waits for the writers ahead of it. On a hot row the queue depth, and therefore the tail latency, grows with concurrency while throughput flattens. Blocked writers hold connections open, so a single contended row can drain the connection pool as a secondary failure.
A failover mechanism promotes a new leader without confirming the old one has stopped, so two nodes simultaneously believe they hold the primary role and both accept writes. The two histories diverge, and when the partition that triggered the failover heals, one set of committed transactions must be discarded.
Each logical application write triggers multiple physical writes through index maintenance, WAL generation, MVCC versioning, and replication, causing actual disk IOPS to exceed the provisioned I/O ceiling while the logical write rate appears modest.
An ALTER TABLE or other DDL statement takes an ACCESS EXCLUSIVE lock that conflicts with every other lock type, including a plain SELECT's ACCESS SHARE. Once that lock request is waiting, every later query on the table queues behind it too, so a DDL statement that is merely waiting, not yet running, is enough to take the table's entire traffic down.
Alternatives Considered
AI Retrieval-Augmented Generation Platform shares core technology (kafka, postgresql) with the chosen architecture but applies different structural patterns; Financial Ledger Platform is a better fit for the identified workload profile.
Analytics Data Platform shares core technology (kafka, postgresql) with the chosen architecture but applies different structural patterns; Financial Ledger Platform is a better fit for the identified workload profile.
API Gateway Platform shares core technology (kafka, postgresql) with the chosen architecture but applies different structural patterns; Financial Ledger Platform is a better fit for the identified workload profile.
Audit and Compliance Platform shares core technology (kafka, postgresql) with the chosen architecture but applies different structural patterns; Financial Ledger Platform is a better fit for the identified workload profile.
Scaling Thresholds
Signals indicating the architecture is approaching its scaling limits:
Tier 1: Hot Account Lock Contention
Signal: pg_locks shows contended rows on accounts table; write p99 > 50ms; deadlock errors in application logs; pg_stat_activity showing many transactions waiting for RowExclusiveLock on the same account rows
Evolution: Implement optimistic locking with version column and retry; or queue concurrent updates for the same account entity through an account-scoped serialization queue at the application layer; or partition the accounts table by account range
Tier 2: Synchronous Replication Write Latency
Signal: Write p99 > 100ms with synchronous_commit = remote_apply; replica WAL apply lag visible in pg_stat_replication; network jitter between primary and replica causing write latency spikes correlating with replication ACK delays
Evolution: Co-locate primary and replica in the same availability zone for lowest replication RTT; tune wal_sender_timeout and recovery_min_apply_delay; evaluate whether synchronous_commit = on (durable to primary WAL only) is acceptable for your regulatory risk model
Tier 3: Event Log Volume and WAL Saturation
Signal: PostgreSQL WAL volume > 500MB/minute sustained; event sourcing table growing faster than VACUUM can reclaim; wal_buffers flushing > 2x per second; I/O utilization on WAL volume > 80%
Evolution: Move WAL to a dedicated NVMe volume; tune checkpoint_completion_target to 0.9; partition the events table by time range (monthly partitions) to bound per-partition VACUUM scope; evaluate whether the balance projection can be computed lazily (on read) rather than maintained eagerly (on write)
Tier 4: Single-Primary Throughput Ceiling
Signal: PostgreSQL primary sustaining > 3000 TPS on financial transactions; write p99 > 200ms despite I/O and pool optimization; audit event table exceeding 1 billion rows
Evolution: Evaluate domain partitioning by currency, region, or account range across multiple PostgreSQL primaries with saga-based cross-shard coordination; or evaluate CockroachDB for distributed ACID writes with global consistency guarantees
Migration Path
Mutable account balance table with no event history → Event sourced ledger with append-only events and projected balance view
Audit requirement to reconstruct account state at any historical point in time; compliance requirement for complete transaction history; inability to explain why a balance is what it is from current state alone
Synchronous Kafka publish in transaction (dual-write pattern) → Outbox pattern with CDC relay to Kafka
Kafka publish failures causing financial transaction rollbacks; or Kafka publish succeeding but transaction rolling back, causing phantom events downstream
Single PostgreSQL primary serving all reads and writes → CQRS with separate read model and write model
Financial dashboard query p99 > 500ms causing dashboard-driven I/O competing with write transactions; reporting queries running against the primary during month-end close causing write latency spikes
Operational Requirements
- Minimum team maturity: Platform Engineering Team: This scenario has expert operational complexity. It is recommended for Platform Engineering Team teams or higher.
- Runbooks and alerting for high-severity risks: 4 high-severity risks identified. Each requires a documented runbook, alerting threshold, and on-call response procedure before running in production.
- Event stream operations expertise: This architecture includes event stream infrastructure (Kafka, Kinesis, or similar). Operations requires consumer group management, partition assignment, dead-letter handling, and lag monitoring.
- Replica lag monitoring and lag-aware routing: Read replicas must be monitored for replication lag. The application router must include a max_lag_ms threshold; queries above that threshold must be redirected to the primary.