EDITORIAL

 

Reporting of diagnostic accuracy studies in radiology

 

 

Eleanor A. Ochodo

Centre for Evidence-based Health Care, Faculty of Medicine and Health Sciences, Stellenbosch University, South Africa

Correspondence

 

 

Introduction

The use of technologically based imaging tests to diagnose diseases and lesions is a fundamental function in radiology. Such tests must be evaluated before their introduction, and during their routine use, in clinical practice to ensure that their performance remains constant. One way of measuring the performance of a medical test is by evaluating its accuracy. Accuracy is defined as the ability of a test to correctly distinguish those subjects who have a target condition from those who do not.^{1, 2, 3} A target condition can be the presence or stage of a disease or infection. Accuracy is commonly measured by parameters such as sensitivity (the probability of a positive test result amongst those with disease) and specificity (the probability of a negative test result amongst those without disease). Other parameters include predictive values, likelihood ratios and area under the receiver operating characteristic (ROC) curve. In research, the accuracy of a test is estimated by comparing the results of the test of interest (index test) with the results of the best available test (reference test).^{1, 2, 4} For instance, one may wish to estimate the accuracy of chest radiographs in detecting a chest lesion. This may be done by subjecting a sample of patients to chest radiography and to computed tomography (CT) scanning, which is the best available test. Chest radiographs can then be compared with CT scans.

Once a study has been carried out, it is important to report the results adequately. Comprehensive and transparent reporting of the conduct and results of a study enable an objective appraisal by readers of the study.⁵ A study may report impressive accuracy measures but fail to report flaws in the study design or study conduct that might have compromised the accuracy measures. The accuracy of diagnostic tests is also known to vary according to the spectrum of patients tested, the severity or stage of disease, differences in test types or technology, and how a test is conducted.⁶ A test may perform well in one setting but dismally in another setting. Poor or inadequate reporting may consequently lead to the uptake or use of suboptimal tests in practice, unnecessary testing, high health care costs and compromising the wellbeing of patients. More guidance and information on the importance of adequate reporting can be obtained via a network that is devoted to enhancing the quality and transparency of health research (the EQUATOR network; http://www.equator-network.org).

To guide authors of diagnostic accuracy studies on how to report their work for publication, a guideline for Standards for the Reporting of Diagnostic Accuracy (STARD) was introduced in 2003. The STARD guideline comprises a checklist of 25 items that are recommended for reporting.⁵ More information on the STARD initiative can be found at http://www.stard-statement.org.

In view of the existence of STARD, has reporting improved? In general, evidence suggests that reporting is slowly improving.^{7, 8, 9} A recently published systematic review by Korevaar and colleagues that examined 16 studies which assessed adherence to STARD showed that there was a small improvement in the reporting quality of diagnostic accuracy studies.⁸ In the area of diagnostic radiology, one study showed that the reporting quality of many published diagnostic accuracy studies remained low.¹⁰ The former study was published 5 years after the introduction of STARD. A slow uptake has also been observed in reporting guidelines for other types of studies, including the Consolidated Standards for Reporting of Trials (CONSORT).¹¹ This slow uptake may be attributed to lack of knowledge of the existence of STARD, or vagueness in journal instructions to authors on the use of STARD. In addition, some of the reporting items may not be applicable to some accuracy studies, leading to authors abstaining from using the guideline.¹²

Against this background, how can adherence to STARD guideline be improved? Increased awareness of the existence and importance of STARD by authors, peer reviewers and journals would be a good start; this could be done by alerting journals to the existence of STARD by publishing more articles and letters on the importance of the proper reporting of diagnostic accuracy studies; more and clearer author instructions in journals on adherence to the STARD guideline; and via other forums such as medical conferences and interactive medical meetings. Evidence is that these interventions may not be effective in changing behaviour when used in isolation, but are more effective when in combination.^{13, 14, 15, 16} In addition, the CONSORT statement has been revised twice (in 2001 and 2010) since its initial introduction in 1996.^{17, 18} Likewise, plans are underway to revise the STARD guideline to make it simpler and accommodate other aspects of diagnostic accuracy.¹² It is hoped that this revised version will become more known to authors and further improve the reporting of accuracy studies.

Lastly: change in practice takes time. Given sufficient time, growing awareness of STARD should lead to more improvements in reporting diagnostic accuracy studies. In South Africa, increased awareness may begin within radiology, a field that is characterised by the widespread use of imaging tests for diagnostic purposes.

 

Acknowledgements

The author acknowledges Professor Charles Wiysonge for critical comments on an earlier version of this submission.

 

References

1. Macaskill P, Gatsonis C, Deeks JJ, Harbord RM, Takwoingi Y. Chapter10: Analysing and presenting results. In: Deeks JJ, Bossuyt PM, Gatsonis C, editors. Cochrane handbook for systematic reviews of diagnostic test accuracy, Version 1.0. London: The Cochrane Collaboration, 2010. Available from http://srdta.cochrane.org/        [ Links ]

2. Van Stralen KJ, Stel VS, Reitsma JB, Dekker FW, Zoccali C, Jager KJ. Diagnostic methods I: sensitivity, specificity, and other measures of accuracy. Kidney Int. 2009;75(12):1257-1263. http://dx.doi.org/10.1038/ki.2009.92        [ Links ]

3. Reitsma JB, Glas AS, Rutjes AWS, Scholten RJPM, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005;58(10):982-990. http://dx.doi.org/10.1016/j.jclinepi.2005.02.022        [ Links ]

4. Ochodo EA, Leeflang MMG. Systematic reviews of diagnostic test accuracy for evidence-based diagnostic practice in Africa. Afr J Lab Med. 2011;1(1):1-3. http://dx.doi.org/10.4102/ajlm.v1i1.7        [ Links ]

5. Bossuyt PM, Reitsma JB, Bruns DE, et al. Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD Initiative. Ann Intern Med. 2003;138(1):40-44. http://dx.doi.org/10.7326/0003-4819-138-1-200301070-00010        [ Links ]

6. Whiting P, Rutjes AWS, Reitsma JB, Glas AS, Bossuyt PMM, Kleijnen J. Sources of variation and bias in studies of diagnostic accuracy: A systematic review. Ann Intern Med. 2004;140(3):189-202. http://dx.doi.org/10.7326/0003-4819-140-3-200402030-00010        [ Links ]

7. Bossuyt PMM. STARD statement: Still room for improvement in the reporting of diagnostic accuracy studies. Radiol. 2008;248(3):713-714. http://dx.doi.org/10.1148/radiol.2483080868        [ Links ]

8. Korevaar DA, Van Enst WA, Spijker R, Bossuyt PMM, Hooft L. Reporting quality of diagnostic accuracy studies: A systematic review and meta-analysis of investigations on adherence to STARD. Evid Based Med. 2014;19(2):47-54. http://dx.doi.org/10.1136/eb-2013-101637        [ Links ]

9. Korevaar D, Wang J, Van Enst W, et al. Reporting diagnostic accuracy studies: Still improving after nine years of STARD?        [ Links ]

10. Wilczynski NL. Quality of reporting of diagnostic accuracy studies: No change since STARD statement publication - before-and-after study. Radiol. 2008;248(3):817-823. http://dx.doi.org/10.1148/radiol.2483072067        [ Links ]

11. Hopewell S, Altman DG, Moher D, Schulz KF. Endorsement of the CONSORT statement by high impact factor medical journals: A survey of journal editors and journal 'Instructions to Authors'. Trials. 2008;9:20. http://dx.doi.org/10.1186/1745-6215-9-20        [ Links ]

12. Ochodo EA, Bossuyt PM. Reporting the accuracy of diagnostic tests: The STARD initiative 10 years on. Clin Chem. 2013;59(6):917-919.         [ Links ]

13. Forsetlund L, Bjørndal A, Rashidian A, et al. Continuing education meetings and workshops: Effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009;15(2):CD003030.         [ Links ]

14. Giguère A, Légaré F, Grimshaw J, et al. Printed educational materials: Effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;17(10):CD004398.         [ Links ]

15. Flodgren G, Parmelli E, Doumit G, et al. Local opinion leaders: Effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2011;10(8):CD000125.         [ Links ]

16. Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: Effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;13(6):CD000259.         [ Links ]

17. Moher D, Schulz KF, Altman DG. The CONSORT statement: Revised recommendations for improving the quality of reports of parallel-group randomized trials. Ann Intern Med. 2001;134(8):657-662. http://dx.doi.org/10.7326/0003-4819-134-8-200104170-00011        [ Links ]

18. Moher D, Hopewell S, Schulz KF, et al. CONSORT 2010 explanation and elaboration: Updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol. 2010;63(8):e1-e37. http://dx.doi.org/10.1016/j.jclinepi.2010.03.004        [ Links ]

 

 

Correspondence:
Eleanor Ochodo
Centre for Evidence-based Health Care
Faculty of Medicine and Health Sciences Stellenbosch University
PO Box 19063 Tygerberg 7505, South Africa
Email: eochodo@sun.ac.za

 

 

 

Note: Dr Eleanor Ochodo is a specialist in diagnostic test evaluation, systematic reviews and evidence-based medicine. She has an MB ChB degree from the University of Nairobi (Kenya), a master's degree in International Health from the Free University of Amsterdam (the Netherlands), and a doctorate in clinical epidemiology from the University of Amsterdam. She is currently a senior scientist at the Centre for Evidencebased Health Care at Stellenbosch University. Dr Ochodo serves as an extended member of the STARD initiative, in a group that is currently revising the original STARD guidelines.