The encyclopedia · Engineering & Operations · Technical decision · 2013–2015
Backblaze stored 17 data shards plus 3 parity to cut redundancy cost
Backblaze splits a file into 17 shards plus 3 parity, so any 17 of 20 rebuild it - durability at about 18% overhead, not 200%.
Backblaze
the move
The usual way to survive a disk or server failure is to keep multiple copies, which means paying three times the storage for conventionally 3x-replicated data. For a backup provider this is the dominant cost.
Reed-Solomon erasure coding offers a cheaper trade. Backblaze splits a file into 17 equal shards, computes 3 parity shards, and stores all 20 across separate drives. Any 17 of the 20 pieces restore the original, so a file survives the loss of up to 3 shards.
That is the point of the design: instead of three full copies it keeps one copy plus a small amount of parity, getting near the same durability while spending a fraction of the storage.
why it works
- Parity shards cost far less than full duplicate copies.
- Any 17 of 20 pieces reconstruct the file, tolerating multi-drive loss.
- Lower storage overhead directly lowers the provider's unit cost.
what transfers
When you pay for reliability by copying data, you are buying it in expensive whole units; coding mathematics lets you buy a precise amount for a fixed overhead instead.
what came after
Backblaze open-sourced its Java Reed-Solomon library, and erasure coding became a common way both cloud providers and backup services trade a little CPU for much cheaper durability.
references
spotted an error? The archive wants to know.