Compare Scenarios

Migration Step 1

Both scenarios define a migration step at this stage. Distributed Job Queue Platform: triggered by 'Background jobs competing with user-facing API requests for '. Social Feed Platform: triggered by 'PostgreSQL timeline read query p99 > 500ms; query plan for "'.

Migration Step 2

Both scenarios define a migration step at this stage. Distributed Job Queue Platform: triggered by 'High-priority transactional jobs (e.g., payment processing, '. Social Feed Platform: triggered by 'Fan-out worker queue lag during celebrity post events > 5 mi'.

Migration Step 3

Both scenarios define a migration step at this stage. Distributed Job Queue Platform: triggered by 'Multi-step jobs (e.g., ingest file → validate → transform → '. Social Feed Platform: triggered by 'Redis memory utilization approaching 80% of a single node; R'.

Worker idle rate > 20% despite queue depth > 10k pending jobs; PostgreSQL pg_locks showing wait events on job table index; worker job claim p99 latency > 50ms (claim should be sub-10ms with correct indexing); CPU on PostgreSQL elevated from index scan overhead on job claim queries

Tier 1: Job Claim Lock Contention: Missing or misconfigured partial index on the job claim query; high worker concurrency driving SELECT FOR UPDATE SKIP LOCKED contention on a narrow hot page. Recommended evolution: Add a partial index on (priority DESC, created_at ASC) WHERE status = 'pending' AND run_at <= NOW(): the WHERE clause reduces the index to only claimable jobs, dramatically reducing index scan range; if contention persists, implement a job dispatch service (single dispatcher process) that batches claim queries and distributes job IDs to workers via an in-memory channel, removing per-worker database claims; tune FILLFACTOR on the job table to 70% to reduce hot page contention on SKIP LOCKED .

High-priority job queue depth growing despite workers available; low-priority batch jobs showing high throughput while transactional job latency (time from enqueue to execution start) p95 > 30s; worker pool metrics showing workers claiming jobs uniformly across priority levels rather than draining the high- priority queue first

Tier 2: Priority Inversion Under Load: Single worker pool consuming from all priority queues with equal weight; no priority-weighted polling implementation. Recommended evolution: Separate worker pools per priority tier (e.g., dedicated transactional workers for high-priority jobs, shared workers for low-priority batch); or implement priority-weighted polling in a unified worker pool (poll high-priority queue N times before polling low-priority queue once, where N is the priority weight ratio); add high-priority job execution latency as a first-class SLA metric with alerting threshold separate from batch job latency .

Kafka consumer group lag for fan-out worker group growing steadily; feed propagation latency (time from post write to follower feed update) exceeding 30s p95; Redis write rate on feed keys elevated but not saturated; post activity rate normal

Tier 1: Fan-Out Worker Queue Backlog: Fan-out worker pool undersized for burst post activity or a celebrity post creating a sustained high-fan-out event. Recommended evolution: Add fan-out worker replicas; implement fan-out cost routing: route high-follower-count fan-out events to a dedicated high-cost worker pool with separate Kafka consumer group and Redis write quota; use follower count threshold (e.g., >50k followers) as the routing decision. Monitor fan-out cost per post as a first-class metric. .

Redis memory utilization > 75%; eviction rate rising; cache miss rate on feed reads increasing; cold feed read fallback queries appearing in PostgreSQL slow query log; feed read p99 > 100ms despite Redis being online

Tier 2: Redis Feed Memory Ceiling: Redis feed list storage approaching memory limit; feed items being evicted before TTL; or feed list cap set too high for available memory. Recommended evolution: Reduce feed list cap from current value toward 100–150 items; increase Redis cluster capacity or shard feed keys by user_id range across multiple Redis primaries; implement tiered feed storage: hot recent items in Redis, older items fetched from PostgreSQL on demand with explicit product UX affordance .