Developer Tools Platform

Topology Notes

• ·API gateway enforces tenant identification on every inbound request (JWT or API key containing tenant_id). The tenant_id is injected into the PostgreSQL session context (SET app.current_tenant = :tenant_id) for RLS policy evaluation. The gateway also enforces per-tenant rate limits before requests reach the application layer : quota enforcement at the gateway is the first line of defense against resource abuse.
• ·Redis hosts two logical functions: job dispatch queues (per-tenant queue lanes, key namespace jobs:{tenant_id}:*) and distributed locks for pipeline state machines (lock namespace pipeline_lock:{pipeline_id}). These two use cases must be on the same Redis instance for operational simplicity at small scale, but should be separated onto dedicated Redis instances if either workload approaches 80% of Redis memory or CPU independently.
• ·Elasticsearch receives log output from pipeline workers via a log shipping agent (e.g., Filebeat or a direct bulk API client). Log documents must include tenant_id as a top-level indexed field, not nested, to enable efficient filtering at query time. The Kibana equivalent (or custom search UI) must enforce tenant_id = current_tenant as a non-removable query clause in every search request.
• ·MinIO stores pipeline artifacts (build outputs, test reports, coverage data) under per-tenant bucket prefixes (/{tenant_id}/pipelines/{pipeline_id}/). Presigned URL generation for artifact download must enforce that the requesting tenant matches the artifact's tenant_id prefix: do not issue open presigned URLs that can be guessed across tenant boundaries.
• ·Kafka receives pipeline completion events via the outbox relay. Webhook delivery consumers read from Kafka and dispatch to tenant-registered endpoints. Per-tenant Kafka consumer instances (or per-tenant partitioned consumers) prevent a slow tenant endpoint from stalling the delivery loop for other tenants.