- Author
- Erik Bernhardsson (at the mortgage startup Better when writing)
- Published
- 27 March 2018
- Kind
- Personal blog post, with a short simulation and a plotted result
- Original
- erikbern.com/2018/03/27/waiting-time-load-factor-and-queueing-theory.html
- Subject
- Mathematics and optimization
- Themes
- Feedback loops
What it is
Erik Bernhardsson writes about operations research. The interest began at a mortgage startup. The post asks how load affects cycle time. It lists examples such as database latency, email replies and meeting booking.
What it says
- The post says running a system at lower throughput can cut waiting time dramatically.
- It says variability explains this, because random bursts of arrivals form a queue.
- It says a simulated queue has twice the latency at 50% utilization as at 0%.
- It says latency climbs fast past about 80% load, and higher percentiles suffer more.
- It gives 1 / (1 - f) as the M/M/1 mean, with f the load.
- The author proposes delegating or automating urgent work until its load is below 50%.
- It says one machine twice as fast beats two slower machines with separate queues.
What evidence it gives
The post gives a short Python simulation of a queue. It plots latency against load. It shows the exact M/M/1 formula matching a simulation. The email example is described as anecdotal. No data from real systems appears.
Interests and limits
The author worked at a mortgage startup and gives personal opinion on workload. The simulation uses chosen distributions, and the author says the exact choice is not important. Real systems were not measured. The advice on delegating is a personal strategy, not tested.
Connections
These links are this wiki's own reading, not claims the sources make.
- Embracing the parallel coding agent lifestyle. Willison calls review the bottleneck when agents run in parallel. Bernhardsson's queue shows what that costs. Latency climbs fast past about 80% load. So a fully busy reviewer may be the slowest part. Perhaps reviewers should stay half idle on purpose.