Backpressure and Load Shedding
Backpressure makes producers respect downstream capacity; load shedding rejects work that cannot be served within its value or deadline.
Backpressure and Load Shedding
Backpressure makes producers respect downstream capacity; load shedding rejects work that cannot be served within its value or deadline. A queue is a buffer, not extra processing capacity. If arrival rate λ exceeds service rate μ, backlog grows at λ−μ. Admission control caps accepted work; backpressure slows producers; load shedding rejects or degrades low-value work. Bounded queues turn memory exhaustion into explicit overload behavior.
Mechanism and worked example
At 1,200 jobs/s arriving and 800 jobs/s completing, backlog grows by 400/s. A 60-second burst adds 24,000 jobs. If arrivals later fall to 500/s and service stays at 800/s, backlog drains at 300/s and needs 80 seconds to clear.
| Condition | Backlog | Action |
|---|---|---|
| λ < μ | Can drain | Keep headroom |
| λ = μ | No recovery headroom | Avoid saturation |
| λ > μ | Grows at λ−μ | Throttle, shed or add real capacity |
| Recovery μ > λ | Drains at μ−λ | Measure age and drain time |
Failure modes and operations
Unbounded queues convert overload into latency and memory pressure. Autoscaling workers can worsen a database bottleneck. Retry storms add arrivals when service is degraded. Priority queues can starve low-priority work without aging or fairness.
Trade-offs and design boundary
Rejecting work harms immediate availability but preserves accepted work. Stale reads may be acceptable for recommendations but not financial mutations. Per-tenant quotas protect noisy-neighbor fairness at the cost of unused reserved capacity.
Interview transfer exercise
Protect checkout during an analytics burst. Bound queue and database concurrency, reserve capacity for order commits, and shed analytics/recommendation refresh first.
Read the diagram
Measure queue age and downstream capacity → Admission applies tenant and priority budgets → Producer slows or low-value work is shed → Workers drain within database concurrency limit