SNSF ran a lottery for grant proposals tied by uncertainty

Funding agencies rank grant proposals by peer-review score and fund everything above a cutoff line, treating that line as a meaningful boundary between funded and rejected quality. But studies of inter-rater reliability keep finding that reviewer agreement on the same proposal is weak: kappa statistics are typically well below 0.5, and a Finnish Academy study comparing two independent expert panels reviewing the same grant proposals found a kappa of just 0.23 -- with panel discussion producing no better consistency than simply averaging the original individual scores. That means a proposal jus

The standard response to that uncertainty is to trust the ranking anyway and defend the cutoff as rigorous: more panel discussion, additional tie-break criteria, re-scoring disputed proposals. Fang and Casadevall argued in 2016 that grant funding was already, in effect, 'a lottery without the benefits of being random' -- reviewer disagreement was already determining many outcomes near the funding line, just disguised as considered judgment rather than acknowledged as chance.