Architecture Anti-Patterns

10 anti-patterns

Structural anti-patterns detected through deterministic keyword and topology signal analysis. Each entry documents symptoms, operational consequences, and structured mitigation paths.

Distribution1

Data1

Scaling2

Operational2

Consistency1

Complexity1

Reliability2

Async Consistency Ambiguity

Consistency

High

Event-driven or eventually consistent architectures without explicit consistency contracts expose the application to unpredictable read behaviors. When consumers don't know the maximum expected lag window, and the product doesn't communicate eventual consistency to users, every stale read becomes a potential bug report. Consistency guarantees must be explicit, per-operation: not system-wide assumptions.

Symptoms

!Users report that their own writes disappear or revert
!Read-after-write consistency violations reported as bugs
!Different users see different states of the same entity simultaneously

Operational Consequences

·Debugging consistency violations requires understanding the full async pipeline
·Inconsistent state visible to users damages trust and triggers support burden

Mitigation Patterns

→Define explicit consistency contracts per endpoint: strong, bounded-staleness, or eventual
→Implement read-after-write routing: post-write reads go to the write path, not projection

Topology Signals

Event-driven read paths without explicit lag SLA documentationCQRS projections without maximum projection lag alertsRead replicas without replication lag monitoringAsync event handlers that write to read models without ordering guarantees

Scaling Implications

↑Consistency degradation worsens under load as async pipeline latency increases
↑More consumer groups means more potential consistency lag points

◈ eventual consistency◈ cqrs operational◈ replication lageventual_consistencyread_after_write

Cache Stampede Architecture

Reliability

High

An architecture that relies on a caching layer without stampede protection creates a structural fragility: whenever the cache is cold (restart, flush, or mass-expiry), the full request load falls simultaneously to the database. This can generate 10-100x the normal database QPS in seconds, saturating connection pools and triggering cascading failures.

Symptoms

!Database QPS spikes 10-100x after Redis restart or cache flush
!Connection pool exhaustion during cache cold-start
!P99 latency spikes correlated with cache TTL mass-expiry windows

Operational Consequences

·Cache flush becomes a high-risk operation that can cascade to database outage
·Redis rolling restart is dangerous without warm-up procedures

Mitigation Patterns

→Implement lock-based refill (SETNX mutex) to serialize cache population per hot key
→Use stale-while-revalidate: serve slightly stale data while refreshing in background

Topology Signals

Redis with no stampede protection mechanism documentedHigh-traffic cache keys with no stale-while-revalidate or lock-based refillCache layer sits directly in front of the primary databaseMass-expiry possible due to synchronized TTL population

Scaling Implications

↑Adding cache nodes does not prevent stampede: all nodes miss simultaneously
↑Database must be sized for stampede load, not just normal cache-warm load

◈ cache invalidationcache_stampedethundering_herd

Connection Pool Starvation

Operational

High

Database connection pools configured without proper sizing, timeout settings, or isolation boundaries become a hard throughput ceiling under load. When the pool is exhausted, all new requests queue: or fail immediately. Without connection pool instrumentation, starvation is invisible until requests timeout. This is one of the most common causes of unexpected production capacity ceilings.

Symptoms

!Request failures or timeouts under load that correlate with connection pool exhaustion
!Query latency spike during traffic bursts that doesn't correlate with database CPU
!Pool wait time visible in connection pool metrics during peak traffic

Operational Consequences

·Pool exhaustion causes request failures that look like application errors, not infrastructure
·Adding more application instances increases connection pool demand proportionally

Mitigation Patterns

→Size connection pool to 10-20% of database max_connections per application instance
→Set pool connection timeout to fail fast rather than queue indefinitely

Topology Signals

Single connection pool shared by OLTP and analytical queriesNo pool size monitoring or alerting configuredConnection pool size not documented relative to database max_connectionsLong-running queries not isolated from short-lived OLTP queries

Scaling Implications

↑Horizontal scaling of application servers increases total connection pool demand
↑At scale, connection pool sizing must account for all instances simultaneously

◈ replication lagconnection_poolingdatabase_saturation

Distributed Monolith

Distribution

Critical

A distributed monolith is a system that is physically distributed (multiple services, network calls) but logically coupled (shared databases, synchronous call chains, shared schema, coordinated deployments). It combines the worst of both worlds: monolith coordination overhead plus distributed systems failure modes.

Symptoms

!Deploying service A requires deploying service B and C first
!Schema migrations require coordinating all service deployments
!Service A's performance degrades when service B increases query load

Operational Consequences

·Deployment velocity is bounded by the slowest service in the dependency chain
·Incidents cascade across all services sharing the database

Mitigation Patterns

→Establish explicit data ownership boundaries before creating service boundaries
→Introduce anti-corruption layers between services to prevent schema coupling

Topology Signals

Multiple services share a single database instanceDeep synchronous call chains with no async boundariesCoordinated deployment windows required across servicesSingle failure causes cascading failures across all services

Scaling Implications

↑Horizontal scaling of individual services provides no benefit if the shared database is the bottleneck
↑Read scale requires routing decisions that must account for cross-service query patterns

◈ eventual consistency◈ cqrs operationalservice_boundariesdata_ownership

Event Replay Amplification Trap

Reliability

High

Event-driven architectures that support consumer replay but lack protection against replay amplification create operational landmines. Replaying a large event backlog through downstream services can generate orders-of-magnitude more write traffic than the original event production rate, overwhelming databases, caches, and downstream API integrations simultaneously.

Symptoms

!Consumer group reset causes database write spike orders-of-magnitude above normal
!Cache stampede triggered by large-scale consumer replay
!External webhook or notification system overwhelmed during replay

Operational Consequences

·Production replay of large backlog can trigger cascading failures downstream
·Replay rate cannot be controlled without consuming application changes

Mitigation Patterns

→Implement idempotency keys for all consumer-driven database writes
→Add replay rate limiting: process events at controlled throughput, not full consumer speed

Topology Signals

Kafka-backed architecture with stateful consumers writing to databasesNo replay rate limiting or batching controls in consumer implementationMultiple downstream effects per event (DB write + cache + notification)No dead letter queue for replay-triggered downstream failures

Scaling Implications

↑Consumer replay throughput is bounded by the slowest downstream system
↑Parallel replay on multiple consumer groups amplifies downstream load multiplicatively

◈ kafka consumer lag◈ eventual consistencyevent_replayidempotency

Hot Partition Concentration

Scaling

High

When a partition key design concentrates traffic on a small subset of partitions, horizontal scaling provides no relief. The bottleneck is a coordination problem : too much work assigned to too few partitions: not a resource problem. Adding more nodes doesn't help if 80% of traffic routes to the same 2 partitions.

Symptoms

!Specific Kafka partitions have disproportionate consumer lag
!Database index page contention on high-write-frequency ranges
!Throughput ceiling reached despite adding nodes to the cluster

Operational Consequences

·Throughput ceiling from partition saturation cannot be resolved by adding nodes
·Hot partition consumer experiences higher latency than cold partition consumers

Mitigation Patterns

→Add random suffix salt to hot partition keys to distribute across N partitions
→Analyze percentile distribution of access patterns before choosing partition key

Topology Signals

Time-based or sequential partition key in high-write systemPower-law user activity distribution without hash-based key saltingSingle event type dominates Kafka topic partition trafficB-tree index sequential hot page in PostgreSQL high-write tables

Scaling Implications

↑Horizontal scaling improves capacity for cold partitions but not the hot partition
↑The hot partition becomes the single throughput bottleneck for the entire system

◈ partition hotspots◈ kafka consumer lagpartition_hotspothorizontal_scaling

Over-Centralized Event Broker Dependency

Operational

High

When every service communication flows through a single event broker (Kafka cluster), the broker becomes a central point of failure for the entire system. Kafka is operationally complex; a misconfiguration, disk saturation, or network partition affecting the broker affects every producer and consumer simultaneously. Centralization creates operational leverage but also centralizes failure blast radius.

Symptoms

!Kafka cluster maintenance window requires coordinating all service deployments
!Any Kafka availability issue causes cascading degradation across all services
!Critical synchronous paths (user login, payment) flow through Kafka unnecessarily

Operational Consequences

·Kafka cluster health is on the critical path for all services simultaneously
·Broker partition rebalancing causes consumer lag spikes across all consumer groups

Mitigation Patterns

→Isolate critical synchronous paths from Kafka: use direct API calls for latency-sensitive ops
→Separate Kafka clusters by domain or criticality to limit blast radius

Topology Signals

All inter-service communication flows through a single Kafka clusterSynchronous user-facing paths use Kafka as transportNo fallback path for Kafka unavailability on critical flowsMultiple unrelated business domains share the same Kafka cluster

Scaling Implications

↑All services scale their producer/consumer capacity with the same central broker
↑Broker scaling requires careful partition assignment without consumer disruption

◈ kafka consumer lagsingle_point_of_failureevent_stream

Premature Distributed Complexity

Complexity

Moderate

Adopting distributed systems patterns (event sourcing, CQRS, saga orchestration, distributed tracing, service mesh) before the organization's scale demands them creates operational overhead without proportional benefit. The team spends more time managing distributed infrastructure than building product. Simple, well-understood architectures outperform complex architectures operated poorly.

Symptoms

!Operational runbooks are longer than business logic documentation
!Debugging a single user-facing bug requires understanding 4+ services
!Incident resolution requires coordinating knowledge from multiple service owners

Operational Consequences

·Distributed failure modes require expertise that the team hasn't yet developed
·On-call burden increases proportionally to distributed system complexity

Mitigation Patterns

→Apply the 'rule of three': wait until a pattern is needed three times before distributing
→Start with a well-structured monolith: it is faster to build, easier to debug

Topology Signals

Deep event sourcing or CQRS for simple read-heavy use casesSaga orchestration for single-entity workflowsService mesh deployed for an architecture with <5 servicesDistributed tracing infrastructure cost exceeds product infrastructure cost

Scaling Implications

↑Distributed architecture may actually reduce throughput vs monolith for current scale
↑Premature distribution adds network latency that a monolith wouldn't have

◈ cqrs operational◈ eventual consistencyoperational_maturitycomplexity_budget

Shared Database Anti-Pattern

Data

Critical

Multiple services sharing direct read/write access to the same database creates invisible coupling that compounds over time. Every schema change becomes a coordination problem. Every performance issue becomes a shared incident. Data ownership becomes ambiguous, making it impossible to reason about invariants for any single service.

Symptoms

!Service deployments require database migration coordination
!Service A incidents frequently involve Service B's tables being locked
!No clear documentation of which service owns which tables

Operational Consequences

·Schema migrations require coordinated service deployments across all services
·One service's inefficient queries degrade all other services

Mitigation Patterns

→Establish schema ownership: each table declared as owned by exactly one service
→Introduce read-only cross-service views as an intermediate step

Topology Signals

Multiple distinct services connect to the same database instanceCross-service foreign key relationships in the database schemaShared connection pool configuration for multiple service typesDatabase schema has no service-boundary namespacing

Scaling Implications

↑Database cannot be sharded per-service because all services share the schema
↑Caching strategy must account for all services' access patterns

◈ cqrs operational◈ eventual consistencydata_ownershipservice_boundaries

Write Amplification Cascade

Scaling

High

Architectures with multiple redundant write paths: CDC pipelines, projection updates, cache invalidations, audit writes, and search index updates: can amplify a single application write into many downstream writes. Under write-heavy load, this amplification consumes disproportionate I/O and creates write bottlenecks in components that individually seem under capacity.

Symptoms

!I/O saturation in downstream systems that have low direct query traffic
!Replication lag grows under write-heavy workloads disproportionate to primary load
!Search index reindexing lags significantly behind primary database writes

Operational Consequences

·Total write I/O is amplification-factor times higher than primary database write rate
·Write amplification compounds during spikes: peak write load amplifies worst at peak

Mitigation Patterns

→Audit total write amplification factor: count all downstream writes per primary write
→Batch downstream writes where staleness tolerance permits to reduce per-event I/O

Topology Signals

Multiple downstream systems triggered by each primary database writeCDC (Change Data Capture) fan-out to multiple consumersWrite-through cache in addition to primary database writesEvent publication on every database write for projection maintenance

Scaling Implications

↑Horizontal scaling of primary write tier does not scale downstream write capacity
↑Write amplification means scaling one component shifts bottleneck to downstream

◈ write amplification◈ kafka consumer lagwrite_amplificationcdc