Crowd-based peer review passes test

The organic synthesis journal Synlett has been conducting an editorial experiment called “intelligent crowd reviewing” that could shake up the traditional peer review process. The results “are stunning,” says Benjamin List, the journal’s editor-in-chief and director of homogeneous catalysis at the Max Planck Institute for Kohlenforschung. Peer review for scientific journals typically involves three anonymous referees from the same research area who judge the quality of a manuscript sent to them by a journal editor. List and his graduate student and editorial assistant Denis Hofler instead worked with an information technology company to create a protected online forum for reviewers. With authors’ permission, they posted 10 submitted papers on the forum and gave a stable of about 100 of the journal’s reviewers 72 hours to respond anonymously to papers of their choosing as well as to respond to fellow reviewers’ comments. In parallel, some of the manuscripts were also examined