ABOVE: © ISTOCK.COM, LAZY_BEAR
A cornerstone of scientific publishing is peer review, where journal editors seek the opinions of one or more experts before accepting or rejecting a paper. Ideally, the process offers an unbiased assessment of a study’s merits and suggestions for making it better. An emerging form of peer review that de-anonymizes reviewers has the intent of increasing transparency and accountability, but a new study on the circumstances under which reviewers choose to reveal their identities suggests it could instead encourage biased critiques.
The norm for peer-reviewed papers has long been an anonymous process where the identities of those reviewing are protected with the aim of encouraging candid reviews. Over the past decade, some open science advocates, who favor a move toward greater transparency and accessibility of scientific knowledge through means such as open data tools and open access journals, have also lobbied for open peer review (OPR), which involves making peer review reports public and identifying reviewers. However, little research has examined whether OPR would achieve advocates’ goals of increasing accountability, fairness, and quality in evaluations.
See “How to Make Scientists into Better Peer Reviewers”
A study published today (October 27) in the Proceedings of the Royal Society B analyzed more than 4,000 peer review reports for the journal Functional Ecology to assess the outcome of eliminating confidentiality from the peer review process. Charles Fox, a University of Kentucky entomology professor and author of the paper, examined the reviews of papers submitted to this journal from 2003 to 2005 and a decade later, from 2013 to 2015. Peer reviewers for Functional Ecology choose whether to sign their comments, which identifies them to the authors of the paper.
Fox’s analysis found that overall, 5.6 percent of reviewers signed their comments. The results showed “a pretty strong gender divide,” with men almost twice as likely to sign their comments as women, Fox tells Times Higher Education. Additionally, he found that the signed reviews tended to be more positive and less critical of the reviewed papers and their writers. Reviewers were also three times more likely to identify themselves when they had been suggested to editors by manuscript authors. Reviewers who selected the salutation “professor,” typically a more senior position, were 1.6 times more likely to sign their comments than those who identified themselves as “Dr.,” which, according to the study, indicates some hesitancy to be identified on the part of more junior reviewers.
The results, particularly the gender divide, “strongly suggest” that a fully de-anonymized peer-review model “will discourage some scholars, those more likely to experience (or expect to experience) discrimination in academia and scholarly publishing, from participating in the review process,” Fox writes in the paper. He also postulates that “reviewers will be less critical of papers, and/or they will be less willing to submit reviews when their assessment is negative,” which could diminish the value of the peer review provided to authors.
Fox acknowledges this analysis is unable to identify any direct causes of changes in behavior in different peer review models, and that few assessments have been conducted on the effects of the OPR model.
