The Peer Reviewers’ Openness Initiative, Year 1

The Peer Reviewer’s Openness (PRO) initiative is a grass-roots movement in which reviewers agitate for new standards of transparency at the journals who depend on them. I’m a signatory to this initiative and a co-author on the paper announcing it, and since 1 January 2017 I have been enforcing PRO as a reviewer. I’ll reflect on how that is going.

 

First, what does a PRO reviewer do? This depends a bit on interpretation. PRO guidance indicates that the reviewer should withhold providing their full review to the editor until the manuscript has met the minimum standards for transparency that PRO endorses: all papers should state whether the data are publicly available or not and include a link to a stable online repository if the data are available. If the data are not available, a reason (any reason at all) should be explicitly given. Most of the manuscripts that I’m sent to evaluate do not meet these standards. That’s why we’re doing this: we want a new normal that is much more transparent than the current status quo. However, no one wants to needlessly slow down peer review by waiting two months to announce to the editor that all we’re going to say about the manuscript is that the data transparency is inadequate. To avoid this, I adopt the proactive approach of contacting the action editor as soon as I discover that the manuscript doesn’t meet the PRO standard, and ask the editor to request an agreement from the authors that they will address this in review. Editors could also promise to enforce my request themselves at this stage. Either way works: if resubmission is encouraged, information about how to access the data and materials will be there when the paper is published. One more paper will provide access to data, and a few more authors will learn that this is becoming the new normal.

 

I’ve had mixed reactions in my interactions with editors of various journals. Many have relayed my query to the authors. Those that did got a response quickly. In every case, the authors were happy to provide the information I wanted, and there was no delay whatsoever in the peer review process. A few others have dropped me as a reviewer. These editors gave several reasons:
  • One called my request “ahistoric” – that’s right! We are trying to change how things are done.
  • Others argued that they could not spare the time to relay a query to the manuscript author. Fine: in that case, I don’t have time to voluntarily write a review for you.
  • At APA journals, I meet with the consistent argument that they will never ask one author to do something that isn’t asked of all authors. This is an absurd policy. Since every paper is unique and reviewers may already ask for anything at all –  they may ask you to remove or add analyses, to change or omit arguments, really, anything they think matters – I don’t see how editors could ever enforce this counterproductive view of fairness. I’ll venture to say that actually, they don’t.
It is satisfying to know that the papers I endorse for publication now all meet at least the PRO standard of transparency. It has also been satisfying to see that many of the journals I work most closely with have now signed on to the Transparency and Openness Promotion guidelines. Level 1 of these guidelines is all the PRO initiative requires. The best outcome those of who signed the PRO initiative could hope for is that in another couple of years, describing data accessibility is so commonplace that we no longer need to request it.
Share this post:

ESCoP preliminary satellite event: The new Journal of Cognition

In December 2016, ESCoP made the momentous decision to stop working on behalf of a publishing company and instead control our own journal. This presents the opportunity for us to create the communication platform for cognitive psychology that researchers want. We envision a journal that is free for all to access, with high-quality content that has been curated by experts and improved by constructive peer review. We are among the first learned societies in psychology to lead in publishing in the fair open-access model, and  I will be speaking and leading discussion about the new Journal of Cognition at the upcoming ESCoP meeting in Potsdam. I will be hosting a discussion about the new journal just before the conference begins on Sunday 3 September from 13:00-14:00. I plan to give an introduction to the innovative content we seek (including Registered Reports and data reports) and the benefits of the fair open access model for researchers and their universities. I will also be seeking feedback from delegates on our plans, including our plans for promoting the journals content and increasing the impact of cognitive psychology generally. All delegates are invited to attend this forum. I hope to see you there!     

Share this post:

Who benefits from for-profit academic publishing?

If you’ve never considered how academic publishing works, start here. If you’re interested in learning more about the forces at work, a group of scholars, librarians, and publishing experts have written a white paper detailing where the money in publishing goes, especially in open-access publishing. We can’t for a minute forget that traditional publishers’ primary aim is to sell content freely provided by scholars for a profit. At the moment, publishers make money selling journal subscriptions to libraries while at the same time charging fees for making select content publicly available. Because widespread open access would threaten their ability to sell subscriptions, the fees charged for open access at for-profit journals do not reflect only the work needed to publish an OA article, but also the anticipated loss involved in not keeping that article behind the paywall. But while most content remains behind the paywall, libraries cannot stop paying for those subscriptions. Publishers collect money for the subscriptions, while also collecting gold-OA fees. Universities are stuck paying both.

Publishing needn’t work this way; there are journals owned and controlled by non-profit groups, and more content should be. When the European Society for Cognitive Psychology recently decided to break with its publisher, we were motivated by the realization that we could not do much to influence our journal’s policies, because we didn’t own and control the journal’s title. We decided to remedy this by launching a new journal adhering to fair OA principles. A mass shift away from traditional publishing to a fair OA model would save universities an enormous sum without reducing the quality of scholarship at all.

Though the biggest traditional publishers talk about embracing the principles of OA, they have little motivation to work on an OA journal if they do not own it. When ESCoP invited publishers to help us launch our new OA journal, we were greeted enthusiastically until we made clear that ESCoP would retain ownership of the journal. Only one of the top-five publishers was interested in working on a society-owned open access journal, and only if ESCoP would financially contribute a substantial amount to the production costs. Fortunately we also received competitive bids from publishers specializing in online OA publishing. None of these publishers requested that ESCoP pay money to mitigate their risks. They provided straightforward estimates of what costs they incurred from taking on the work of publishing our journal, along with a plan for per-article fees to meet those costs. We ultimately chose Ubiquity Press from these options (more about that next week).

The big publishers all have official positions about open access, describing how they want to “empower” scholars or help us “innovate”. But their exorbitant gold OA fees cannot be justified by any of these principles. I suspect we are never going to get around the fact that the traditional publishers are only committed to offering OA if they can do so while protecting their lucrative subscription model. 

Share this post:

II. What’s in a fair article processing charge?

A companion post explaining why I think scholars should switch to publishing via non-profit open-access is here.

Ideally, a scholar would know before choosing an open-access journal how the article processing charge (APC) is reckoned. It should always be clear what we are paying for. APCs for open-access publishing vary enormously. A few non-profit open-access journals are able to offer their authors gratis publishing, while other journals charge thousands of £/€/$ for ostensibly the same service. Why the differences, and what services are worth paying for?

The most expensive APCs I’ve heard of are charged by traditional, for-profit journals to make individual articles available on a case-by-case basis. Scholars supported by universities or funding agencies that insist on open access might pay these fees to comply with institutional requirements. The APC does not reflect any superior service. It reflects the loss to the publisher incurred when the paper is not exclusively available by subscription. The gratis options I’ve heard of are from non-profit open access journals who have either 1) obtained financial backing from non-profit institutions (such as national science organizations or scholarly societies) that pay the essential fees incurred in publishing for the authors or 2) recruited disinterested, idealistic editors who volunteer their services, and additionally perform the essential professional services usually provided by a publisher (e.g., hosting a journal’s website, maintaining databases of reviewers, sending reminder emails, ensuring articles are securely archived). When open-access publishing is free for the author, that doesn’t imply that there are not costs involved. It means that someone else is paying them. 

Can we decompose an APC to evaluate its fairness? Some of the costs involved in publishing an article are really incurred when the article is accepted and complete when the processing is finished. For instance, we can calculate the exact cost of the hours of expert labor needed to copy-edit and typeset a manuscript. However, many other expenses are less straightforward. Some costs may be bulk-distributed across all the content a publisher handles, and may scale depending on how much content the publisher distributes. These include things like costs of dedicated staff, and server space for housing the journal content. A small journal independently publishing 20 papers a year might approach these costs differently than a larger journal publishing 200 papers per year. Whatever the scale, these costs may still be estimated per paper accepted. You want a professional to handle these issues. There are not going to be lots of academics willing to honorarily proofread content or take responsibility for the integrity of a server (and do a solid job of it). Handling these tasks requires professional skill and we should value people with those skills enough to reward their good work.

Editorial evaluation of submitted content cannot be valued on the basis of accepted articles, because offering differential rewards for accepted papers would create a perverse incentive. Often, editorial work goes uncompensated but I think this is a tactical mistake that we should strive to avoid. Evaluating a manuscript requires skill and effort, and we want the people doing this to be motivated to do this thoroughly and quickly. At the journals I am familiar with where editors are paid (note that frequently they are not), they are either paid an honorary annual lump sum or a handling fee per paper. This sort of compensation does not introduce bias because editor income is not contingent on the decision. So even non-profit article processing charges are not a simple break down of editorial fees plus production fees: the processing charge must be larger than the real cost in order to account for the editorial labor involved in rejecting papers. Assuming that editors are paid a handling fee of €40 per paper and the journal rejects on average 50% of submissions, the APC due to content evaluation is €160: With a 50% rejection rate, each accepted article covers the €40 fee for editor-in-chief and action editor for the accepted article and one rejected paper. Obviously, if the rejection rate is higher, this changes. Obviously, this is in addition to the other costs incurred when a paper is accepted. It also assumes that that peer reviewers work for free; in my ideal world, they would be compensated too. Thinking about this, it is becoming easier for me to suppose that a fair APC will be many hundreds of €.         

Where could costs be cut? If enough scholars and their universities fully endorsed the non-profit open-access publishing model, then one excellent step would be for universities to factor staff’s editorial duties into their workload models. Though academics are currently expected to accept editorial responsibilities and peer review regularly, these activities are not (usually? ever? I would love to hear if any of you are given time each week at work for this) explicitly included in a workload model. If editorial work were not an extra task but replaced other work, then the real cost of a fair APC would be appropriately subsidized by universities, who have a stake in this, and the charged APC would decrease. Rejection rates are another big factor. Reconsidering policies contributing to high rejection rates (e.g., closed editorial decision systems that discourage sharing opinions between journals, placing arbitrary limits on journal size to create scarcity, poor self-evaluation of manuscripts and feedback leading to resubmission of the same rejected content to journal after journal, etc.) would also reduce the need to budget as much for editorial evaluation.

To summarize, decomposing the costs in a fair APC depends on a lot of factors: the size of the journal, its rejection rate, whether editorial work is compensated, whether there are funding sources absorbing some of the costs, etc. The real costs in labor of evaluating and producing an article and ensuring that it remains accessible easily reach many hundreds of €s. We should expect non-profit open access journals to periodically justify its APC to authors, proving that it is a fair rate that allows the journal to properly support its mission.

Share this post:

I. Why non-profit, open-access publishing is better for scholars

Scholars, who do the bulk of the work involved in academic publishing, are faced with a choice: 1) give away your work to a for-profit journal to publish, or 2) pay a non-profit journal an article processing charge. Let’s look at the costs of publishing, both in terms of money and effort, incurred at every stage from the start of a research project until it finally appears in a journal, to discover where the costs come from, and who benefits most. 

The initial costs of producing a scholarly article are borne by scholars and the universities or funding agencies supporting them. Scholars conceived the idea for the project, made sure that the work necessary for writing the article was accomplished, which can involve hours of management, pain-staking data coding, costly data collection, and/or many hours of data analysis, all on top of actually writing the article. On the day you finally submit your article for publication, you and the institutions supporting you have incurred all the costs for producing that work. What next? Well, we all know that we can’t sell scholarly writing. If we charged for it, few people would ever read it. But scholars and their universities want the work to be available to interested readers in perpetuity, and they also want some indication about its quality. So we give the article to a journal capable of determining its quality, and if they accept it, the journal handles its distribution. Ideally, scholars get what they want from this arrangement: the assurance that their work will be forever available to a wide audience, and that its quality will be certified by the editorial process.

Whether we chose a for-profit publisher or a non-profit publisher, the article must now be evaluated by one or two of the journal’s editors plus an additional 2-4 peer reviewers. Assuming it is eventually accepted, staff at the journal copy edit it, check that all the references are right, apply their house style so that it doesn’t look like a term paper, and distribute it. If it is rejected, the author will submit it to another journal, where the evaluation process recommences. So, in order for the scholar to get the credit and exposure they hope for, hours of skilled labor are required from at least 4 additional people. How is that labor paid for? At a traditional for-profit journal, this comes from the earnings the publisher makes from selling subscriptions to the journal. The publisher makes as much as 37% profit, which you might naturally assume enables them to pay a fair rate for the skilled labor that made the whole process possible. But, just as authors give their work to a journal for the honor of letting that journal publish it, scholars evaluate articles for the journal for the honor of doing so, or possibly for an honorarium that does not amount to a fair rate for the labor needed to do a thorough job. For the effort of recruiting some honorary scholar labor (a task that is also usually farmed out to scholars), proofreading, and type-setting your paper, the publisher takes control of your work so that they can charge your university money so that your students have the opportunity to read it. Honorary scholarly labor is the backbone of this system, and for providing it, your university has to pay for access to the final product. 

The emerging alternative is to instead choose an open-access journal that does not operate by selling subscriptions. In this model, authors (or preferably their universities) typically pay an article processing fee when their paper is accepted for publication. Paying to give something away may seem suspicious, but the value of this fee depends on what the fee covers and what you get in exchange. Suppose you just want your paper to be publicly available, perhaps because it is a stipulation of your funding source or a rule of your institution. For-profit publishers charge huge fees to for a la carte open access to offset the decrease in income they would see from paid subscriptions if many author chose open access. This justification of the fee does not benefit the author at all. Non-profit open access journals charge lower rates designed to reflect the actual expenses incurred from evaluation and production of the articles. The published article is made freely available, so the university will never need to pay anything further for its students and staff to access the final product.

In terms of giving scholars what they want (i.e., credit and exposure), how do these publishing models compare? First, let’s agree that it is not really the publisher who provides the lovely feeling of esteem we get from having a paper accepted: that’s the editor and peer reviewers, who are respected colleagues. The process of editorial review is identical at for-profit and non-profit journals. The for-profit model adds nothing essential here. In terms of distribution of your article, there is no comparison between models: the non-profit open-access model makes your paper freely available to anyone, forever. The for-profit publishers will try their hardest to ensure that only the interested readers with a current university ID can possibly access your paper without paying for it. Anyone else will be charged a ludicrous fee, a fee which probably exceeds what your editor received for handling your paper! 

So to avoid paying for the necessary services to evaluate and distribute your article represented by a non-profit article processing charge, you (with your effort) and your university (with their money) are willing to perform most of the work needed to produce an article, sign away control of it, and then buy the copy-edited and typeset product at a marked-up rate. Universities are arguably better off paying a non-profit publisher an article processing charge to procure the services they want to ensure access to an unrestricted finished product. This financial benefit for the university increases as more scholars and their universities choose the open-access model and less new content is subscription-restricted. 

Next up: What’s in a fair article processing charge?

Share this post: