COMMENTARY

Peer review - challenges, pitfalls and trust

Bridget Farham^{I, II}

^IEditor, South African Medical Journal, Cape Town, South Africa
^IIConsultant, WHO African Regional Office, Emergency Preparedness and Response, Health Information and Risk Assessment Programme, Brazaville, Republic of the Congo

Correspondence

Keywords: COVID-19, preprint, medical research

Just this morning I had reason to think about the whole issue of peer review in our local academic journals. South Africa is in the enviable position on the continent of having a large body of highly regarded and internationally recognised academic institutions. As editor of the South African Medical Journal (SAMJ) I have a long tradition of excellence to uphold - sometimes a daunting task. Over the years the SAMJ has evolved and is now a well-respected repository of research and policy papers that are read and cited around the world. During the COVID-19 outbreak in the country, the SAMJ has attracted a large number of submissions from our body of clinicians and researchers, all of whom are happy to submit to a local journal, where I have expedited review and online publication in the interests of keeping people abreast of all the various issues around the current pandemic.

So why was I thinking about peer review specifically? A paper dealing with a particularly important aspect of COVID-19 in our context had been submitted in late June. One of the authors contacted me to enquire about its status, concerned that this potentially important piece of research was taking such a long time to be considered. The reason - waiting for peer review. The SAMJ gives its peer reviewers a limited time in which to review, after which we invite new reviewers. This particular paper had been declined, or simply ignored, by six reviewers. This is not an uncommon situation, unfortunately.

We have approximately 1200 reviewers registered on our website, of whom we use around 200, among whom 150 are regulars and complete reviews within the allocated time or ask for more time if necessary. Wikipedia's definition of peer review is that it is 'the evaluation of work by one or more people with similar competencies as the producers of the work. It functions as a form of self-regulation by qualified members of a profession within the relevant field ... and in academia, scholarly peer review is often used to determine an academic paper's suitability for publication'¹.

The pitfalls of peer review are many. One of the biggest is personal bias among others in a field, which can lead to a research submission being rejected simply because the findings do not fall within an accepted paradigm. For those of us who are professional editors rather than academics or, in my case, clinicians, in a particular field, this can be problematic if you are not aware of the various factions in different institutions. It pays to look through the general literature regularly and identify interest groups in particular fields.

There is an increasing move towards pre-publication of papers before they go for formal peer review - the idea being that anyone in the field can look, comment and discuss, potentially adding value to a discussion that would otherwise be limited to perhaps two or three formal reviewers. I have been generally in favour of this practice, having seen many instances where good research has come up against bias that has prevented publication. However, there are pitfalls here as well and the current COVID-19 pandemic has shown these very effectively. In April 2020 a group of Dutch engineers self-published a study that they said showed that the 1.5-m distance rule for people exercising outdoors was flawed.² Their premise was that an infected person who exhaled, coughed or sneezed while walking, running or cycling would leave behind them a slipstream of microdroplets. Someone following after such a person would then move through this cloud and potentially inhale the virus. These conclusions were reached by simulating the release of saliva particles from people in motion. They displayed their results as a series of animations and figures in which the cloud of droplets behind a moving person is clearly visible. The simulations led the group to advise that walkers, runners and cyclists should keep a distance of at least 4-5 m behind the leading person while walking in the slipstream, 10 m while running or cycling slowly and at least 20 m when cycling fast. Their argument against waiting for peer review was the 'urgency of the situation and the world-wide crisis'. This led them, in their words, to turn things 'upside down. First research results and communication to the public', then the proposal for funding and only then submission of an article for peer review.

The study went viral. The lead author gave an interview to a Belgian paper and also tweeted about their results. The media picked it up, leading to horrific headlines in, for example, the Daily Mail, 'Horrifying simulation reveals the dangers of jogging during the coronavirus pandemic'.³ Locally, the various forms of 'publication' were spread around Facebook and WhatsApp running and cycling groups (personal observation). A fierce debate started, and other non-academic media picked this up, with 'peer review', much of it from virologists, with some excellent discussions of whether or not the engineers' results could be translated into the real world. One of the best was in wired.com⁴, in which Eric Niiler carefully picked the study apart, pointing out that at the time of publication, although we knew that the virus could spread from person to person from coughing or sneezing, we did not know the amount of time that the virus could survive outside the body, or the dose of the virus required to result in infection. At that time there were no published studies of the spread of SARS-CoV-2 from person to person outdoors. A study at the time of 318 outbreaks of three or more COVID-19 patients found that all but one transmission occurred indoors, but that study was also pre-published on medRxIV, a preprint server for health sciences.⁵

More recently, and of far more significance, is the announcement by Russia that they have developed a vaccine against COVID-19. So far no-one has seen a publication, let alone a peer-reviewed publication, on the safety and efficacy of this vaccine. The World Health Organization is currently calling for a 'rigorous safety data review', as the vaccine is currently listed as being in Phase 1 trials, meaning that by most normal criteria, it is nowhere near ready for approval, manufacture and roll out.⁶ A search of Pubmed retrieved zero results, showing that not only have studies on the vaccine not been peer reviewed, but they have not even been published. Apparently 20 countries have already pre-ordered more than a billion doses of this vaccine, which by any reasonable criteria is completely unproven.

Here we have, in a nutshell, an example of the importance of peer review in trust. For all the flaws inherent in academic publishing, this is the gold standard that is required before research results can be put into practice.

References

1. Wikipedia. Peer review [webpage on the Internet]. No date [cited 2020 Aug 08]. Available from: https://en.wikipedia.org/wiki/Peer_review        [ Links ]

2. Bocken B, Malizia F, Van Druen T, Marchal T. Social distancing v2.0: During walking, running and cycling [document on the Internet]. c2020 [cited 2020 Aug 15). Available from: https://nysoralms.com/wp-content/uploads/2020/04/Social-Distancing-v20_White_Paper.pdf        [ Links ]

3. Kekatos M. Horrifying simulation reveals the dangers of jogging during the coronavirus pandemic. Daily Mail. 2020 April 08 [cited 2020 Aug 15]. Available from: https://www.dailymail.co.uk/health/article-8201877/Horrifying-simulation-shows-six-feet-not-jogging-someone.html        [ Links ]

4. Niiler E. Are running or cycling actually risks for spreading COVID-19? Wired. 2020 April 14 [cited 2020 Aug 15]. Available from: https://www.wired.com/story/are-running-or-cycling-actually-risks-for-spreading-covid-19/        [ Links ]

5. Qian H, Miao T, Liu L, Zheng X, Luo D, Li Y. Indoor transmission of SARS-CoV-2 [preprint]. medRxiv. https://doi.org/10.1101/2020.04.04.20053058        [ Links ]

6. WHO wants a safety data review on Russian COVID-19 vaccine. Medical Brief. 2020 August 12 [cited 2020 Aug 15]. Available from: https://www.medicalbrief.co.za/archives/who-wants-a-safety-data-review-on-russian-covid-19-vaccine/        [ Links ]

Correspondence:
Bridget Farham
Email: ugqirha@iafrica.com

Published: 21 September 2020