The Peer Reviewers’ Openness Initiative, Year 1
The Peer Reviewer’s Openness (PRO) initiative is a grass-roots movement in which reviewers agitate for new standards of transparency at the journals who depend on them. I’m a signatory to this initiative and a co-author on the paper announcing it, and since 1 January 2017 I have been enforcing PRO as a reviewer. I’ll reflect on how that is going.
First, what does a PRO reviewer do? This depends a bit on interpretation. PRO guidance indicates that the reviewer should withhold providing their full review to the editor until the manuscript has met the minimum standards for transparency that PRO endorses: all papers should state whether the data are publicly available or not and include a link to a stable online repository if the data are available. If the data are not available, a reason (any reason at all) should be explicitly given. Most of the manuscripts that I’m sent to evaluate do not meet these standards. That’s why we’re doing this: we want a new normal that is much more transparent than the current status quo. However, no one wants to needlessly slow down peer review by waiting two months to announce to the editor that all we’re going to say about the manuscript is that the data transparency is inadequate. To avoid this, I adopt the proactive approach of contacting the action editor as soon as I discover that the manuscript doesn’t meet the PRO standard, and ask the editor to request an agreement from the authors that they will address this in review. Editors could also promise to enforce my request themselves at this stage. Either way works: if resubmission is encouraged, information about how to access the data and materials will be there when the paper is published. One more paper will provide access to data, and a few more authors will learn that this is becoming the new normal.
I’ve had mixed reactions in my interactions with editors of various journals. Many have relayed my query to the authors. Those that did got a response quickly. In every case, the authors were happy to provide the information I wanted, and there was no delay whatsoever in the peer review process. A few others have dropped me as a reviewer. These editors gave several reasons:
- One called my request “ahistoric” – that’s right! We are trying to change how things are done.
- Others argued that they could not spare the time to relay a query to the manuscript author. Fine: in that case, I don’t have time to voluntarily write a review for you.
- At APA journals, I meet with the consistent argument that they will never ask one author to do something that isn’t asked of all authors. This is an absurd policy. Since every paper is unique and reviewers may already ask for anything at all – they may ask you to remove or add analyses, to change or omit arguments, really, anything they think matters – I don’t see how editors could ever enforce this counterproductive view of fairness. I’ll venture to say that actually, they don’t.
It is satisfying to know that the papers I endorse for publication now all meet at least the PRO standard of transparency. It has also been satisfying to see that many of the journals I work most closely with have now signed on to the Transparency and Openness Promotion guidelines. Level 1 of these guidelines is all the PRO initiative requires. The best outcome those of who signed the PRO initiative could hope for is that in another couple of years, describing data accessibility is so commonplace that we no longer need to request it.