Reviewer self‐reported confidence and evaluation breadth/depth in open peer review

What the paper says

Understanding how reviewer self‐reported confidence shapes evaluation behavior is vital, as it influences editorial decisions, the identification of methodological flaws, and which research directions gain prominence. However, prior studies have rarely examined how self‐reported confidence affects both the breadth and depth of review content across distinct evaluative dimensions. This study addresses this gap using large‐scale data from an open peer review platform to analyze the relationship between reviewer self‐reported confidence and evaluation breadth and depth. Results show that self‐reported confidence is negatively associated with breadth, meaning reviewers with higher confidence focus on fewer dimensions. However, it is positively associated with depth, as reviewers with higher confidence provide more detailed, rigorous feedback within chosen dimensions. Dimension‐specific analyses show heterogeneous focus patterns. For soundness, reviewers with higher confidence attend to fewer aspects without adding depth; for contribution, they narrow focus but deepen feedback; for novelty, they broaden focus and increase depth. These findings align with attention allocation theory, suggesting that reviewers with higher confidence selectively allocate attention to dimensions they deem most relevant or demanding.