Google made massive data processing into a two-function map-and-reduce chore.

Google's crawl and log data grew past what any single server could hold or process, and the natural answer was a larger, more expensive server or a bespoke parallel program no one wanted to write.

Dean and Ghemawat's library, first written in February 2003, let a programmer supply just two functions; the runtime split the input, shipped it out, sorted by key, and retried anything that failed.

By late 2004 Google was running hundreds of distinct MapReduce programs on up to tens of thousands of machines for tasks from Google News and Zeitgeist to large graph and machine-learning jobs.

The paper, published in OSDI 2004 and later in Communications of the ACM, became the template for big-data processing and was re-implemented as the open-source Hadoop, which made the model an industry default.

Move compute to data and split it into map+reduce

Splitting a job into map and reduce lets ordinary engineers parallelize without writing threads or message passing.

Keeping data on the same machine as its computation avoids shipping terabytes across the network.

Retrying failed map tasks makes a farm of unreliable PCs behave like one dependable machine.

When data is too large for one box, decompose the job into embarrassingly parallel pieces and let a scheduler hide hardware failures, instead of buying a bigger, costlier machine.

MapReduce became Google's standard batch model and, through Hadoop, the engine that established the big-data era across industry. Google later added higher-level systems (FlumeJava, Dataflow) as interactive SQL replaced the two-step batch model, and MapReduce itself was eclipsed, but its core idea that a large job can be two functions became permanent.