ML Feature Serving Platform

Topology Notes

• ·The online serving path has a strict latency budget: Redis cache lookup (p99 < 3ms) or Cassandra feature lookup (p99 < 20ms) must complete within the total inference budget. Multi-feature lookups for a single inference request must be parallelized using async batch fetch: sequential per-feature lookups multiply latency by feature count and will always violate the serving SLA.
• ·Redis feature cache keys are structured as feature:{feature_name}:{entity_id}:{pipeline_version}. The pipeline_version component is critical: it prevents a cache hit on a feature computed by an old pipeline version from being served to a model expecting the new version. TTLs are set per feature type based on expected freshness requirements: user behavioral features (5 minutes), item popularity features (1 hour), static entity attributes (24 hours).
• ·Cassandra stores feature values as feature_name + entity_id primary key with pipeline_version + computed_at as clustering columns. This enables point-in-time queries (SELECT ... WHERE computed_at <= label_timestamp ORDER BY computed_at DESC LIMIT 1) for offline training. Cassandra token-aware routing must be configured in the client to avoid coordinator hops for feature lookups by entity_id.
• ·Qdrant collections are organized by embedding model version and feature type. Each collection stores pre-computed embeddings with payload metadata (entity_id, computed_at, pipeline_version). HNSW index parameters are tuned per collection based on the recall/latency tradeoff for the specific model consuming them. Collection aliases are used to enable zero-downtime index swaps during embedding model upgrades.
• ·ClickHouse receives feature serving logs (entity_id, feature_name, served_value, pipeline_version, served_at) via Kafka for drift monitoring. Drift dashboards compute per-feature distribution statistics (mean, p5, p95, null rate) over rolling 24-hour windows and compare against training population baselines. Alert thresholds are set per-feature based on the model's known sensitivity to that feature's distribution.