Open and Ongoing Peer Review: A New Model for Scientific Research Peer review: time for a makeover?

Peer review is slow, expensive, profligate of academic time, highly subjective, prone to bias, easily abused, poor at detecting gross defects, and almost useless for detecting fraud.
Richard Smith, Editor, British Medical Journal (1)

As a researcher in Emergency Medicine, peer review is something I get to see both sides of as a routine part of my job. So many evenings are spent reviewing journal articles and grant applications! And so much of my time is spent revising articles and applications based on feedback from other reviewers.

Peer review is a key part of scientific research, helping to make our work more reliable and trustworthy. Essentially, peer review involves experts in the field evaluating research before it gets published. This process is meant to ensure that the research is solid, the methods used are appropriate, and the conclusions are backed by evidence.

In our world, peer review is often seen as a safeguard, catching errors, biases, or unfounded claims that the original authors might have missed. By having others take a critical look at the work, we aim to make sure that the research meets the high standards expected in our field. This helps build trust in the findings, making them more likely to be accepted by the scientific community and the public.

But peer review isn’t without its challenges. One of the biggest issues is that it can be a lengthy and sometimes overwhelming process. For example, in our field, research undergoes peer review at multiple stages: first by supervisors or within the research team, then by funders and their reviewers, regulatory authorities like the Health Research Authority, Research Ethics Committees, research sponsors, information governance teams, and patient/public representatives. On top of that, local research sites may have their own reviews. And finally, once the research is complete, the findings go through yet another round of peer review before they can be published. If the reviewers don’t like what they see, they can block the research at any one of these stages.

This constant cycle of reviews can delay research, sometimes significantly. What’s more, if any of these reviews result in rejection or major revisions, it can prevent the research from ever seeing the light of day. This is especially frustrating when reviewers ask for changes to the interpretation of the findings or tweaks to the methodology—especially when those methods have already been reviewed and approved at earlier stages.

So, how can we make the peer review process better? One idea is to move towards open and ongoing peer review, a model that’s already being used in some contexts, like the NIHR journals. In this approach, research methods that have been peer-reviewed by funders wouldn’t need to be critiqued again at the publication stage. Instead, the focus could be more on how the results are presented and interpreted.

I’ve recently had experience of this, when we published two articles that relate to an NIHR grant to develop a platform trial for prehospital troponin testing. (The grant for the large trial ultimately failed to get past peer review, so we’re still working on this!). You can see the two articles in question at the links below. Have a look at the right side bar, and you’ll see how open peer review works. Our papers were published very quickly, without any peer review. The journal then invited reviewers, and the status of the paper was updated once the reviews were returned. We were given an opportunity to revise the manuscripts once we’d received feedback from the reviewers. The peer review for the qualitative paper is still ongoing at the time of writing.

Our first such paper is here – a multiple criteria decision analysis to make the process of deciding which tests to evaluate robust and transparent.

The second is here – a qualitative study to evaluate facilitators and barriers to prehospital point of care testing in the opinions of practising paramedics.

Opening up the peer review process and making it continuous could also reduce the barriers to publication. Instead of requiring that all reviewers be completely satisfied before anything gets published, research could be shared with ongoing dialogue and debate around the interpretation of the results. This would not only speed up the process but also allow for a more dynamic and inclusive conversation about the research.

As we think about the future, it’s important to consider how peer review should evolve. Should it continue as a gatekeeper, blocking publication until all reviewers are happy? Or should it become a more open, ongoing process that encourages dialogue and debate?

We’d love to hear what you think. How do you see peer review changing as we move forward? Share your thoughts in the comments below!