Data Colada caught Ariely's faked honesty data with font and rounding forensics

A 2012 PNAS study by Shu, Mazar, Gino, Ariely and Bazerman reported a strikingly cheap fix for dishonesty: move the honesty pledge to the top of a form and people lie less. In its field experiment, nearly 13,500 car-insurance customers who signed at the top reported about 2,400 more miles — 10% — than those who signed at the bottom. The finding was cited more than 400 times, taught in business schools, adopted by insurer Lemonade, tested by governments in the UK, Guatemala and Canada, and credited by the IRS with helping collect an extra $1.6 million from vendors in a single quarter.

The trouble began when the effect refused to replicate. Original authors joined outsiders to rerun the signature experiment and found no difference, and in 2020 they published that failure in PNAS — posting the original 2012 data for the first time, a transparency step that was still rare. Joe Simmons, Leif Nelson and Uri Simonsohn, the academics behind the Data Colada blog, downloaded the spreadsheet with a team of anonymous researchers and started asking not whether the statistics were right but whether the file looked like something people actually produced.

Their audit found four anomalies. Reported miles driven were uniformly spread from 0 to 50,000 — indistinguishable from an Excel random-number generator. Reported odometer readings had no round numbers, although people round when they estimate. Half the baseline rows were typed in Calibri and half in Cambria, each Cambria row a near-twin of a Calibri row always higher by under 1,000 miles; a million simulations never produced it by chance. Baseline rows had been duplicated and jittered; the experimental data came from a random number generator — fabricated 'beyond any shadow of a doubt'.

The evidence forced a reckoning. All five original authors agreed the data appeared fraudulent and asked PNAS to retract the paper; four authors said they played no part in collecting the data, leaving Ariely — the only author in contact with the insurer — as the open question. Ariely denied wrongdoing and blamed his hands-off approach. BuzzFeed News published the investigation on August 21, 2021; the Hartford confirmed only that a small 2007–2008 project with Ariely existed, and that it could find no data or deliverables from it.

Instead of rerunning the statistics, audit the raw file's mundane layer — fonts, rounding, duplicate rows and distributions — where fabricated data betray themselves

Fraud is easiest to hide in the statistics and hardest in the everyday texture of data — fonts, rounding and duplicates survive peer review and expose the generator.

The 2020 paper's decision to post raw data turned an unverifiable claim into an auditable file; without the posting, the fabrication could not have been found.

The tests were cheap and public — distribution fits, digit counts, font splits and simulations any reader could rerun — so the verdict rested on evidence, not authority.

The finding's own success created the incentive to check: an intervention adopted by insurers and tax agencies was worth verifying once it stopped replicating.

When a result matters, trust ends at the raw data: require the file, then check rounding, duplicates, fonts and distributions — fabricating numbers is easy; making them look human is not.

Within days of the Data Colada post in August 2021, all five authors agreed the data appeared fraudulent and asked PNAS to retract it; BuzzFeed News' investigation, updated August 26 with the Hartford's statement, made the case a symbol of the replication crisis. The fraud's decade of influence was already spent: products built on the signature effect, from Lemonade's honesty pledge to government tax-form changes, had to be reassessed. The authors' systemic lesson stood: journals should require raw data, because posting is what made the fabrication catchable at all.