The encyclopedia · Engineering & Operations · Technical decision · 2013
Google hedged the slowest request to fix tail latency
At scale the slowest of many parallel requests decides response time, so Google drops the straggler by duplicating it and taking the first to finish.
the move
In a single server, latency is measured by its average. In a warehouse-scale system that divides a single user request among thousands of machines, the user experiences the slowest one, so the more machines involved, the closer the response time creeps to the extreme tail.
The naive fix is to avoid variance: buy faster, more uniform hardware. Dean and Barroso argued instead for redundancy at the level of the call. Hedged requests issue a backup after a delay and take the first answer, and duplicate requests are another variant that immediately sends copies.
This turns a worst-of-n problem into a best-of-n one, and it is inexpensive because the duplicated call is only useful in the uncommon case where the original happens to be slow.
why it works
- User latency is decided by the slowest machine, not the average.
- A backup only costs when the original is the straggler.
- Taking the first response pulls the tail toward the median.
what transfers
When one outlier among many units decides the outcome, spend a little redundancy to take the best result instead of waiting for the worst.
what came after
Hedging and request duplication became standard techniques in large-scale systems, and the paper reshaped how engineers reason about latency instead of throughput at scale.
references
spotted an error? The archive wants to know.