Analytics Data Platform
Analytics Pipelinehigh complexity
Deterministic topology derived from YAML knowledge entities. Nodes represent workloads, datastores, patterns, and risk components. Edges show typed relationships with propagation direction.
11
Components
5
Connections
3
Failure Modes
1
Propagation Paths
Max exposure: high· 1 high-risk node in this topology
Topology Graph11 nodes · 5 edges
1 high-risk nodeClick a failure mode below to trace propagation
Workload
Datastore
Cache
Event stream
Pattern
Risk node
Risk path
Failure Propagation Trace
Topology Notes
- ·PostgreSQL logical replication slot feeds the CDC connector (e.g., Debezium). The connector publishes per-table change events to Kafka topics partitioned by primary key hash. Replication slot lag is a first-class operational SLA.
- ·Kafka topics act as the durable event log between the OLTP source and the analytics store. Event retention should be set to at least 7 days to allow consumer replay after failures without requiring OLTP backfill.
- ·ClickHouse consumers read from Kafka using the Kafka table engine or an intermediate consumer service. Inserts are batched to reduce part fragmentation. The analytics store must never be queried directly from the OLTP application.
- ·Materialized views in ClickHouse are maintained incrementally as new parts arrive. Summary tables for common aggregations (daily active users, revenue rollups) are pre-computed and served to dashboards without full-scan queries.
- ·The OLTP write path (application → PgBouncer → PostgreSQL) is fully isolated from the analytics read path (ClickHouse). CDC adds WAL overhead to the OLTP primary but does not introduce synchronous latency to application writes.