Overview

This system distributes realtime events from producers to many consumers while maintaining low latency under burst traffic.

Core Architecture

• Used Redis/Valkey pub/sub as the distribution backbone.
• Added worker coordination to prevent duplicate delivery paths.
• Isolated producer and consumer concerns for simpler scaling.

Scaling Decisions

• Introduced queue boundaries for pressure control.
• Tuned worker concurrency with measured throughput targets.
• Added visibility into lag, drop risk, and recovery time.

Result

The architecture remained stable under spike conditions and supported cleaner horizontal scaling for new consumers.