Crowdsourcing in peer review: estimating the effects of crowd peer review on academic impact and social media impact
Lixue Lin et al.
What the paper says
Purpose Traditional peer review systems face growing criticism for their inefficiency and bias, prompting the exploration of innovative alternatives such as online crowdsourcing-based peer review models. This study aimed to investigate the effects of crowd peer review—specifically, open and closed variants—on both academic impact and social media impact, addressing the need for empirical evidence on these emerging digital practices in scholarly communication. Design/methodology/approach Articles published by Synlett that implemented closed crowd peer review (n = 917) and articles published by Atmospheric Chemistry and Physics (ACP) that implemented open crowd peer review (n = 12,827) were selected as samples. Causal inference was used to examine the data. Alternative outcome variables and different datasets were used to check the robustness of the results. Findings The results revealed that open crowd peer review significantly enhanced both academic impact and social media impact, whereas closed crowd peer review had no significant effect on either metric. These findings suggest that openness and transparency mechanisms are necessary rather than optional features of an effective crowdsourcing-based peer review system. Originality/value This study provides novel empirical evidence on the quantifiable benefits of open crowd peer review, advancing the understanding of digital crowdsourcing mechanisms in academic communication. By highlighting the differential impacts of open and closed models, the results offer actionable insights for publishers and platforms seeking to innovate peer review processes in alignment with digital transformation and online community engagement. Peer review The peer review history for this article is available at: https://publons.com/publon/10.1108/OIR-06-2025-0426.
Evidence weight
Balanced mode · F 0.40 / M 0.15 / V 0.05 / R 0.40
| F · citation impact | 0.50 × 0.4 = 0.20 |
| M · momentum | 0.50 × 0.15 = 0.07 |
| V · venue signal | 0.50 × 0.05 = 0.03 |
| R · text relevance † | 0.50 × 0.4 = 0.20 |
† Text relevance is estimated at 0.50 on the detail page — for your query’s actual relevance score, open this paper from a search result.