WARC saved the web by storing crawl history as records

Early web archives could store fetched pages, but replay and long-term management needed more than payload bytes. Crawls also needed request headers, response headers, timestamps, IP addresses, digests, metadata and links between records.

WARC's design made that context first-class. The format is a simple concatenation of WARC records, each declaring its type and headers before the content block. It can carry ordinary HTTP material and also arbitrary metadata linked to stored data.

The WARC 1.1 specification says the format was designed to structure, manage and store web archives, support compression and integrity, store control information from protocols such as HTTP and DNS, and record all control information from the harvesting protocol.

Its adoption was helped by backward compatibility with the older ARC world: archives could move to richer records without abandoning existing crawls.

Archive the transaction, not just the file

It kept payload and capture context together, making later replay and audit possible.

Record types let one file mix responses, requests, metadata and deduplication references.

Digest fields gave preservation systems a built-in integrity handle.

The appendable stream model matched how crawlers actually collect the web.

When the object is contextual, preserve the transaction around it. The metadata path can matter as much as the bytes captured.

WARC became the working file format of web archiving, used by national libraries, research crawlers and preservation tools. It turned a crawl from a pile of pages into a replayable evidence trail.