#1374 2008 · Billion Prices Project (MIT/Harvard; PriceStats) · Economic measurement / data
Economists measured inflation daily by scraping prices retailers already post
问题
Official inflation takes months and field agents — and in some countries it is lied about
背景
Consumer price indices are built the way they were built a century ago: statistical offices send agents to stores to record prices monthly, then publish with a lag. The method is slow and, where a government manipulates the statistic, unverifiable from outside — Argentina's official index after 2007 famously understated inflation, and publishing alternatives risked fines.
In 2008 Alberto Cavallo and Roberto Rigobon of MIT founded the Billion Prices Project: software collecting the posted prices of large online retailers daily — hundreds of retailers in more than 50 countries — with the same data commercialized through PriceStats (now part of State Street) as real-time inflation indicators.
换别人会怎么做
Commission a larger survey or model the official series harder — still slow, still dependent on the very agency whose numbers you need to check.
他们看到了什么
Inflation statistics needed months and a government's honesty. The prices were already online, posted daily by retailers — scrape them, and the index becomes both faster and independent of whoever it measures.
那一手
The price data already existed; retailers post every price online for their own sales. BPP replaced clipboards with scrapers: nightly reads of entire catalogs, every variety, every day. The project published its first daily index for Argentina in 2008 (where the gap against the official series became itself measurable), a daily US index in 2010, and from 2011 PriceStats has published daily inflation indices for 22 countries on a three-day lag.
为什么管用
Scraping removes the two binding constraints of statistical collection: labor (agents visiting stores) and time (monthly cycles). A bot reads entire catalogs nightly at near-zero marginal cost, so the index can be daily and cover every variety sold. Because posted online prices co-move with offline levels in most sectors, the series tracks official CPI where that is honest — and where it is manipulated, the divergence becomes visible in real time rather than argued about annually. Commercialization through PriceStats funds the collection, and academic publication keeps the methodology open.
值了多少
Daily prices from hundreds of retailers in 50+ countries since 2008; PriceStats publishes real-time inflation for 22 countries at 3-day lag
什么时候会失灵
Online prices skew toward sectors that sell online, so the basket differs from a national CPI; quality adjustment is cruder than statistical offices' hedonic methods; retailers redesign sites and scrapers break, so maintenance never ends; and in economies with thin e-commerce or large informal sectors, scraping sees almost nothing. It also measures posted prices, not the negotiated and couponed prices people actually pay.
后来呢
Scraped-price indices now supplement central-bank statistics; the Argentina daily series (Inflacion Verdadera, 2007-15) became the standard exposure of official understatement; the work seeded big-data measurement across economics.
资料来源
- [1]The Billion Prices Project: Using Online Prices for Measurement and Research (NBER Working Paper 22111; JEP 30(2))National Bureau of Economic Research, 2016nber.org
- [2]The Billion Prices Project — data and design (Cavallo, Deutsche Bundesbank Statistics Forum slides)Deutsche Bundesbank, 2015bundesbank.de