If we were to create a way to share scientific research progress today, what would it look like? In this age of immediate digital publishing, it probably wouldn’t take months. In these times of user-generated content, it would probably be author-controlled. In an era where we post comments about everything from our lunch to the news, we would likely find a way to make it transparent and open. And in this world in which content is so easily shared, curation would be separate from publishing.
Problems with Publishing
We have all heard the complaints about the current publishing system—it is too slow; it is fraught with occasional poor-quality, arbitrary review; it is dominated by a small number of highly selective journals, and it puts pressures on scientists that may contribute to the reproducibility crisis. I recently had a chance to contemplate what the future of scientific communication might look like during a meeting convened by ASAPBio and HHMI entitled Transparency, Recognition, and Innovation in Peer Review in the Life Sciences, which brought together societies, publishers, editors, researchers, and other interested parties to discuss what can be done to improve peer review and publishing.
Open peer review is designed to provide greater transparency of a journal’s editorial process….
The journal review process currently has two functions—assessment of the technical quality of the work, and a gatekeeper role that determines the fit with the journal’s editorial lens. In highly selective, high-profile journals these two functions are combined at times to the detriment of the quality assessment function. If reviewers focus too much on the gatekeeper role and attempt to assess the impact rather than the technical quality of the work, this can result in poor quality reviews that
Erika C. Shugart
focus on the fit of the article rather than improvement of the quality. Reviews of this type don’t help the authors improve their science, which is truly the purpose of peer review.
Another common problem that can result from reviewers being overly concerned about impact is that to make the paper “worthy” of the journal authors are asked to do additional experiments that don’t improve the quality of the science or the validity of the conclusions. This can result in months of publishing delays as the authors attempt to add material to the manuscript. In the end, the manuscript may still be rejected, which leads the authors to start the process again with a new journal, reconsider the structure of their manuscript and rewrite it, or both.
New Approaches
The peer review problems described above do not afflict all publishing experiences. A number of publishing outlets are taking new approaches. One of the newest is bioRxiv, modeled after arXiv, which has served the physical sciences for over 25 years. bioRxiv publishes preprints and gives authors control of when their results go public. It allows commenting, but not the peer review function. Another venture, F1000Research, is a publishing platform that makes both the article and post-publication reviews public.Closer to the traditional publishing model, a number of journals are making the reviews public to a variety of degrees, including the EMBO journals, the British Medical Journal, and several Royal Society journals. Open peer
review is designed to provide greater transparency of a journal’s editorial process in the hopes that this will raise the quality and civility of the reviewer comments. It encompasses a wide range of options from open identities (the author and reviewers know each other’s identities), open reports (the reviews are made public but the reviewer identities are masked), open interaction (authors and reviewers communicate directly), and the eLife model in which reviewers are known to each other and work together to provide clear instructions to authors, to many options for post-publication commenting.
Concerns about open peer review include the possibility of reviewers writing falsely positive reviews for powerful authors and the related concern of authors taking revenge on reviewers due to bad reviews. Based on the discussions that took place during the Transparency, Recognition, and Innovation in Peer Review in the Life Sciences meeting, many of these concerns may be unfounded, as the journals that already offer this have had good experiences and positive feedback from authors.
In the near future, authors and reviewers should expect to see an expansion in the number of journals offering a variety of open peer review, as a number of significant titles have indicated that they plan to increase transparency by offering the option to share peer reviews.
Journals Run by Scientists for Scientists
Another alternative outlet is society journals, which are by run scientists for scientists, often have editorial policies that emphasize the technical quality over the unexpectedness of the science, and take pride in constructive peer review. For example, Molecular Biology of the Cell has an editorial policy that was expressed by its founding editor, David Botstein: “Is it new and is it true?” Additionally, any revenue generated by the journals goes back to the societies and is used to create programs that help further the scientific profession.
If society journals and other avenues are such a publishing nirvana, then why does anyone go anywhere else? The answer is, of course, the impact factors of the highly selective journals and the associated prestige that the names of such journals bring to their offshoots. Scientists could have a different publishing experience if they choose, but the pressures, particularly for early career scientists, to get publications in high-profile journals on their CV keep them trying for a limited number of journals.
The perniciousness of the improper use of the journal impact factor was highlighted in 2013 with the publication of the San Francisco Declaration on Research Assessment (DORA), which was formulated at the 2012 ASCB Annual Meeting. However, not much has changed in the past five years. As the cartoon character Pogo said, “We have met the enemy and he is us.” Scientists have the power to change the assessment system, since they are often asked to assess each other in grant reviews, hiring decisions, and promotion and tenure decisions, but this will require cultural change. DORA has now been revitalized by new funding and in-kind support from ASCB, Cancer Research UK, the Company of Biologists, eLife, EMBO,
F1000Research, Hindawi, PLOS, and Wellcome. The effort, which is housed at ASCB, is taking the lead on documenting and spreading best practices. We hope that by documenting best practices and sharing them with researchers, funders, and university administers we will be able to lower the use of the impact factor as a shortcut to assessment of scientific quality and improve the outcomes for all scientists.
Scientific publishing has changed radically in the last 20 years with digital publishing, open access, and other innovations, but expect the disruption in publishing to continue and even accelerate as new tools and platforms become available. Some trends that I expect to see expand in the near future are open peer review and separate curation services for preprints and perhaps even journal articles. It is anyone’s guess what scientific publishing will look like 20 years from now, but it is in our power to create a system that is efficient, rewards good science over flashy results, and invests back into the scientific community.
About the Author:
Erika Shugart is the Chief Executive Officer of ASCB.
