Ceph CRUSH found storage by calculating it, not looking it up

Large distributed file systems traditionally needed metadata that said where each block or object lived. At petabyte scale, those allocation maps became coordination, recovery and rebalancing problems.

Ceph's answer was CRUSH: a pseudo-random placement function. Clients and storage nodes could calculate an object's location from the object name and cluster map, instead of asking a central table.

The OSDI paper reported Ceph supporting more than 250,000 metadata operations per second, and the CRUSH publication framed the algorithm for tens or hundreds of thousands of storage devices while avoiding a central directory.

Turn placement into a function

A deterministic placement function lets every participant independently find the same object.

Weights and failure-domain rules preserve real-world constraints such as racks and device sizes.

Small topology changes move only the data that must move, making growth and failure less disruptive.

Removing allocation lists shrinks the metadata server's job to namespace, not every byte's address.

At extreme scale, deleting coordination can matter more than optimizing it; a shared deterministic function can replace a shared mutable directory.

CRUSH became one of Ceph's signature ideas and helped make the project a durable open-source storage system. Its beauty is architectural: the best directory at petabyte scale was no directory at all.