The Peer Reviewers’ Openness Initiative, Year 1

The Peer Reviewer’s Openness (PRO) initiative is a grass-roots movement in which reviewers agitate for new standards of transparency at the journals who depend on them. I’m a signatory to this initiative and a co-author on the paper announcing it, and since 1 January 2017 I have been enforcing PRO as a reviewer. I’ll reflect on how that is going.


First, what does a PRO reviewer do? This depends a bit on interpretation. PRO guidance indicates that the reviewer should withhold providing their full review to the editor until the manuscript has met the minimum standards for transparency that PRO endorses: all papers should state whether the data are publicly available or not and include a link to a stable online repository if the data are available. If the data are not available, a reason (any reason at all) should be explicitly given. Most of the manuscripts that I’m sent to evaluate do not meet these standards. That’s why we’re doing this: we want a new normal that is much more transparent than the current status quo. However, no one wants to needlessly slow down peer review by waiting two months to announce to the editor that all we’re going to say about the manuscript is that the data transparency is inadequate. To avoid this, I adopt the proactive approach of contacting the action editor as soon as I discover that the manuscript doesn’t meet the PRO standard, and ask the editor to request an agreement from the authors that they will address this in review. Editors could also promise to enforce my request themselves at this stage. Either way works: if resubmission is encouraged, information about how to access the data and materials will be there when the paper is published. One more paper will provide access to data, and a few more authors will learn that this is becoming the new normal.


I’ve had mixed reactions in my interactions with editors of various journals. Many have relayed my query to the authors. Those that did got a response quickly. In every case, the authors were happy to provide the information I wanted, and there was no delay whatsoever in the peer review process. A few others have dropped me as a reviewer. These editors gave several reasons:
  • One called my request “ahistoric” – that’s right! We are trying to change how things are done.
  • Others argued that they could not spare the time to relay a query to the manuscript author. Fine: in that case, I don’t have time to voluntarily write a review for you.
  • At APA journals, I meet with the consistent argument that they will never ask one author to do something that isn’t asked of all authors. This is an absurd policy. Since every paper is unique and reviewers may already ask for anything at all –  they may ask you to remove or add analyses, to change or omit arguments, really, anything they think matters – I don’t see how editors could ever enforce this counterproductive view of fairness. I’ll venture to say that actually, they don’t.
It is satisfying to know that the papers I endorse for publication now all meet at least the PRO standard of transparency. It has also been satisfying to see that many of the journals I work most closely with have now signed on to the Transparency and Openness Promotion guidelines. Level 1 of these guidelines is all the PRO initiative requires. The best outcome those of who signed the PRO initiative could hope for is that in another couple of years, describing data accessibility is so commonplace that we no longer need to request it.
Share this post:

PhD positions at Cardiff University School of Psychology!

There are several opportunities to undertake PhD study at the School of Psychology at Cardiff University. The deadline is 23 February 2018, to begin in October 2018. One of the positions is in my lab!

I’m really looking forward to working with a new student. I loved being a PhD student myself, despite the widely-noted downsides of working in academia (which you should be aware of going in!). I want my PhD students to look back fondly on their training too, and not to regret their choice. Here’s what went into my decision to become a PhD student:

What career are you aiming for, and do you need a PhD to get it? You need a PhD to enter tenure-track academia, but in other research careers, you may not need a PhD, and having one might not increase your income. If you want to try for an academic career, that’s great! Be aware that getting the PhD is just necessary for that, not sufficient. But though the academic job market is weak, the market for competent researchers is not. If you would be happy to work outside a university, the skills you acquire as a PhD student in cognitive psychology (writing, critical thinking, organisation, statistical analysis, programming) will make you more employable (though a PhD is not the only way to acquire these skills). Your prospects in academia are also better if you can afford not to limit yourself to a particular geographic region. I moved ~1000 miles for my PhD position. Since then, I’ve worked in three countries, but that wasn’t a drawback – I enjoy living abroad (and so does my partner, which is also something you may have to consider). So, keep in mind that getting your PhD is just one of the challenges you will face pursuing a career in academia.

Studying for a PhD is worth your time and effort only if you enjoy the process. You want to be confident about this – your happiness for the next few years depends on it. What makes you think you will enjoy research?  Did you enjoy doing research during your bachelor’s or master’s study? I’m not asking whether you excelled at it – did you actually enjoy thinking about the research idea, reading about it, analysing the data? If you did not actually enjoy it, you will struggle with your PhD no matter how smart or motivated you are. Your undergraduate research occupied just a portion of your working time. You got plenty of breaks from it to work on other things. PhD work won’t be like that: it will be your full-time job for three years. Let’s be honest: it will probably be even more work than a full-time job. If you have spent time working in a lab, voluntarily while you were a student or as a job, that will give you a better basis for deciding whether research work is for you. When I decided to join Nelson Cowan’s lab as a PhD student, I had worked part-time 1) at a library helping people with research for 3 years, 2) in a clinical developmental lab for undergraduate course credit for 2 years, 3) in a cognitive lab for undergraduate course credit for 1 year, and 4) as a part-time research assistant on a project investigating the effectiveness of preschool literacy programs for 2 years. These experiences were really important. Because of them, I knew I enjoyed working in labs and doing research. Even the work I enjoyed least was alright.

I had also tried out a few topics, and knew before I began looking for a PhD position which of these topics appealed most to me. I cannot overemphasise the importance of this. Your PhD will be a deep dive into a narrow topic. You are going to examine a very specific problem in more depth than anyone has ever examined it before. If you are not already interested in this problem, you will not enjoy this process. Don’t start down a three-year journey to expertise in one topic of psychology thinking that when it is over, you can easily pivot to something very different. The opportunities available to you in my lab will be focused on working memory and the methods we can use to study it that you can master in <3 years. The extended network you will join as a junior member will be full of people obsessed with working memory. If my PhD student is thriving and wants to pursue an academic career, s/he would be well-positioned to get a postdoctoral research job in a lab investigating working memory, executive functioning, multi-tasking, or something in that ballpark. If my PhD student wants a post-doc job in a different ballpark, s/he will be competing with other thriving PhDs who are already playing over there. You don’t want to dive into a PhD project before you are sure it is something you can get happily obsessed with.

When it works right, PhD study is brilliant. It works right when the supervisor and student share a vision and excitement about the project. I had that as a PhD student and I want you to have it too. Check out our opportunities and see if something we are offering has that potential for you.


Share this post: