#1531 2004 · Slashdot · online communities
Let Readers Rate Readers, and Rate the Raters
the problem
A huge open comment forum drowns in noise; hired moderators don't scale and staff censorship breeds revolt.
background
Slashdot, a news and commentary site for open-source technology attracting about a third of a million unique users each day, opened its threads to everyone: the site's creators mandated anonymous posting, and posts are never deleted from the database, only shown to fewer readers.
Moderation was distributed to the readers themselves. In the system documented by the CHI 2004 study, eligible users are randomly given five moderation points for a limited time and spend them scoring comments up or down; accumulated standing, karma, qualifies users and shifts with how they are rated. A second layer, meta-moderation, lets other users review whether moderations were fair, policing the police.
what everyone would do
Hire moderators or ban anonymous posting.
what they saw
Don't hire judges, ration judgment: give everyone a turn with five points that expire, let reputation earn privilege, let the community rate the raters. Scarcity plus rotation keeps any faction from capturing the forum.
the move
Filtering becomes a micro-economy of rationed judgment. Scarce, expiring moderation points ration the power to judge; karma rations who gets the privilege; comment scores set each reader's visibility threshold, so everyone self-selects their noise level; and meta-moderation aligns moderators with community standards by making their judgments themselves rated. No one can dominate: points expire, quotas cap participation, randomness rotates the bench.
why it works
Random selection with expiring points prevents organized capture; karma ties the privilege to demonstrated contribution; visibility thresholds let each reader tune noise; and meta-moderation audits the judges at negligible cost.
the payoff
A CHI 2004 study documented distributed moderation keeping a huge daily-user site readable without staff censors.
where it breaks
It fails when moderator culture homogenizes, since fair-unfair reviews inherit the same biases; when karma becomes the goal rather than the signal; and for late or low-karma comments, which the study found systematically under-moderated, the crowd's blind spots stay blind.
what came after
The karma-and-threshold architecture seeded Reddit and every member-maintained community; meta-moderation remains the classic answer to who guards the guardians.
references
- [1]Slash(dot) and Burn: Distributed Moderation in a Large Online Conversation SpaceCHI 2004 (Lampe & Resnick, University of Michigan), 2004pensivepuffin.com
- [2]The Virtues of ModerationYale Journal of Law & Technology (James Grimmelmann), 2015james.grimmelmann.net