ReviewGPT: Reducing subjectivity in the review process using AI

Abstract

There is widespread, multidisciplinary evidence that the peer review process often reflects the idiosyncratic perceptions of reviewers, resulting in low interrater reliability and concern regarding the extent to which reviews and journal decisions accurately reflect the scientific contribution of a manuscript.These issues emerge because the peer review process is highly subjective, which makes it rife with opportunities for the expression of various human biases and errors.For example, reviewers may evaluate manuscripts more harshly when they disagree with their own views or preferred theoretical or methodological approaches, place disproportionate emphasis on a single aspect of the manuscript rather than evaluating it in its entirety, or increase the critical tone of their review in an effort to impress the editor.In addition to various cognitive biases, reviewers are also subject to human error and gaps in knowledge.As a result, reviewers may misread or misunderstand authors' work, make inaccurate statements, or suggest improper practices.Because of these biases and errors, relying solely on subjective reviewer comments in editorial decisionmaking is likely to introduce bias and error into the review process, resulting in the rejection of meritorious scientific contributions, unnecessary revisions, and the preservation of the status quo.As a result, academic careers are derailed, research is slow to disseminate, and scientific innovation is stalled.We fully agree with Allen et al. ( 2026) that the subjectivity of reviewers creates significant problems for science that must be addressed.However, in our view, subjectivity is an issue not only for reviewers, but also for authors and editors.As a result, changes to the peer review process that increase author and editorial discretion may inadvertently exacerbate the impact of human error and bias in the peer review process.In this commentary, we describe our concerns with increasing author and editorial discretion, and propose increasing objectivity as a solution to the problem of error and bias in peer review.To facilitate the adoption of this more objective path and address Allen et al.'s recommendation to explore how AI can be used to improve the peer review process, we provide access to two custom GPTs aimed at improving manuscript and review quality-Friendly Reviewer and Review Checker.