#133 2013 · Center for Open Science / Cortex journal (Chris Chambers) · Academic publishing / research integrityincentive-flip
A journal stopped judging papers by their results and started approving them before anyone knew what the results would be
the problem
Evaluators judging work by its outcome quietly rewarded people for engineering outcomes, not for doing the work well
background
Academic journals traditionally decide whether to publish a study after seeing its results, rewarding novel, statistically significant findings and routinely rejecting rigorous studies that found nothing notable. This created a quiet but powerful incentive for researchers to engage in p-hacking (running many analyses until one crosses a significance threshold), selectively reporting only favorable comparisons, or even reframing an exploratory finding as though it had been the original hypothesis all along (HARKing) — because a null result, however carefully obtained, was far less publishable than a flashy one.
Everyone in the field knew this incentive existed and that it was distorting the published scientific record, but the standard fixes — replication requirements, statistical training, ethics guidelines — all operated after the fact, on research that had already been shaped by the pressure to produce a publishable result before a single data point was collected.
the move
In 2013, neuroscientist Chris Chambers introduced Registered Reports at the journal Cortex: a two-stage review format where researchers submit their research question, hypotheses and full methodology for peer review before collecting any data, and the journal grants 'in-principle acceptance' based purely on the importance of the question and the rigor of the design. Stage two, after the study runs, checks only that the pre-registered protocol was actually followed — the paper cannot be rejected for producing a null or unexciting result.
the payoff
More than 300 journals have since adopted the Registered Reports format, and the mechanism visibly changed what gets published: roughly 96% of conventionally published psychology papers report a statistically significant result, compared to only about 44% of Registered Reports' pre-registered hypotheses — evidence that the traditional format wasn't just capturing more true positives, it was filtering out a large share of the negative results actually being found.
what came after
Registered Reports are now widely cited in the open-science and replication-crisis literature as the structural fix that direct appeals to researcher integrity couldn't achieve, and major journals including Nature expanded their own Registered Reports programs in the years since, treating pre-commitment to method over outcome as a durable publishing standard rather than a niche experiment.
references
- [1]Registered reportWikipedia, 2026en.wikipedia.org
- [2]Registered ReportsCenter for Open Science, 2025cos.io