The usual way to survive a disk or server failure is to keep multiple copies, which means paying three times the storage for conventionally 3x-replicated data. For a backup provider this is the dominant cost.

Reed-Solomon erasure coding offers a cheaper trade. Backblaze splits a file into 17 equal shards, computes 3 parity shards, and stores all 20 across separate drives. Any 17 of the 20 pieces restore the original, so a file survives the loss of up to 3 shards.

That is the point of the design: instead of three full copies it keeps one copy plus a small amount of parity, getting near the same durability while spending a fraction of the storage.

Buy durability with parity math, not with duplicate copies

Parity shards cost far less than full duplicate copies.

Any 17 of 20 pieces reconstruct the file, tolerating multi-drive loss.

Lower storage overhead directly lowers the provider's unit cost.

When you pay for reliability by copying data, you are buying it in expensive whole units; coding mathematics lets you buy a precise amount for a fixed overhead instead.

Backblaze open-sourced its Java Reed-Solomon library, and erasure coding became a common way both cloud providers and backup services trade a little CPU for much cheaper durability.

FOLLOW THE EVIDENCE

The sources

  1. Backblaze open-sources Reed-Solomon erasure coding source code Backblaze · backblaze.com ·
  2. Erasure coding OpenStack Wiki · wiki.openstack.org