SciELO - Scientific Electronic Library Online

 
vol.16 issue1Orthopaedic training in South AfricaRheumatoid arthritis author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Article

Indicators

Related links

  • On index processCited by Google
  • On index processSimilars in Google

Share


SA Orthopaedic Journal

On-line version ISSN 2309-8309
Print version ISSN 1681-150X

SA orthop. j. vol.16 n.1 Centurion Feb./Mar. 2017

 

EDITORIAL

 

The changing landscape of evidence-based orthopaedics

 

 

Leonard C Marais

MBChB, FC Orth SA, MMed Ortho, PhD Head Clinical Unit, Department of Orthopaedics, School of Clinical Medicine, University of KwaZulu-Natal

 

 


ABSTRACT

It is difficult to find fault with the reasoning behind the move towards an evidence-based approach in the teaching and practice of orthopaedics. With numerous options available, treatment strategy selection has to be based on more than just intuition and prior experience. Furthermore, there are several strong arguments for the need to practice Evidence-Based Orthopaedics (EBO). We have seen novel implant technologies enter the market, only to exit relatively shortly afterwards. Recall metal-on-metal articulations being hailed as the solution to all our problems? Less than five years later we saw reports of 49% failure rates at six-year follow-up.1 In addition, research continues to disprove longstanding orthopaedic axioms. We can now say, with relative confidence, that debriding an open fracture within six hours is not as important as previously believed.2 This principle is also illustrated by a recent randomised study that found no advantage in the damage control concept in the treatment of femur shaft fractures in polytrauma patients.3 Interestingly, patients treated with external fixation in this series had an increased time in ICU on ventilation compared to patients treated by reamed nailing of the femur. And thus, the evidence-based tenet remains largely intact.


 

 

 

It is, however, becoming increasing difficult to practise and teach evidence-based orthopaedics. The first obstacle we face is the sheer quantity of data. For example, a cursory Google Scholar search for articles related to femur neck or hip fractures revealed 6 910 items. Even scanning through the titles would take an inordinate amount of time. Staying abreast of the available evidence is made even more difficult by the number of journals that we have to follow. It is no longer sufficient to follow the two or three 'major' orthopaedic journals. Articles of significance are now found in a wide range of publications, necessitating an alternative approach. The second, and perhaps the more pertinent, challenge we face relates to the quality of data and the interpretation thereof. In his landmark article, Ioannidis (by means of a rather complex argument) proposes that most published research findings may be false.4 The author argues that research findings from underpowered, early-phase clinical trials would be true about one in four times, or even less frequently if bias is present. Ioannidis goes one step further stating that claimed research findings may often be simply accurate measures of the prevailing bias. Thus, most medical research operates in areas with very low probability for true findings and large or highly significant effects may actually more likely be signs of large bias. The problem is that as clinicians we are not necessarily adequately equipped with either the skill or the time to detect all the subtle errors, biases or statistical manipulations present in the evidence base. A recent personal experience, while reviewing a 'big data' paper for an international publication, attests to this. On the face of it all appeared well with the manuscript. Something peculiar, however, prompted me to discuss the findings with an experienced researcher and statistician. After careful scrutiny, several serious scientific flaws (albeit accidental) came to light, which essentially invalidated the findings. None-the-less, the article was subsequently published in a different international medical journal. Another problem with the quality of 'the evidence' has to do with the interpretation of the data analysis. Abdullah and co-workers found that 28% of orthopaedic randomised controlled trials with negative findings were underpowered.5 This means that a large proportion of studies reporting 'no significant difference' is in fact not adequately powered to detect a clinically meaningful difference between groups, which then leads to inappropriately failing to reject the null hypothesis. These factors have led to many questioning the value of evidence-based medicine (EBM) in its current guise.

In 2014 Gary Klein argued for the retirement of the idea of EBM based on the fact that the science behind science is neither infallible nor comprehensive.6 The author points out that too many medical studies cannot be replicated and that many studies with negative findings never get published. This so-called publication bias is illustrated by the fact that only 17% of surgical papers published between 2000 and 2006 reported negative findings.7

Furthermore, Stahel and Mauffrey argue that EBO may not only be stifling innovation in orthopaedic surgery, but may also compromise patient safety.8 Notably, there is a lack of evidence supporting the EDM approach. Perhaps Every-Palmer and Howick stated it best: 'Given that EBM firmly favours an empirical approach over expert opinion and mechanistic rationale, it is ironic that its widespread acceptance has been based on expert opinion and mechanistic reasoning, rather than EBM 'evidence' that it actually works'.9

But, if EBM is somewhat flawed - where to from here? The problem does not necessarily lie with the EBM premise but 'the evidence' or data, and the access to it. In fact, thoughtful meta-analysis is an extremely useful tool to address many of the preceding problems. We simply cannot return to the days of unverified anecdotes. In terms of the sheer quantity of data, several platforms are emerging to assist the clinician to assimilate the widely dispersed data out there. Bone & Joint360 is an example of a publication that utilises experts in the field to present and interpret new research findings from across the globe on a regular basis. In terms of the quality of data, novel methodologies have emerged that also rekindles the value offered by experienced and knowledgeable experts. Expert consensus-based medicine (ECBM) is a new concept that appears to be sensible alternative to guide our practice in areas where evidence is limited.8 A recent example of this strategy is the 'International Consensus Meeting on Surgical Site and Periprosthetic Joint Infection'.10 This guideline was developed using the Delphi method.11 The popularity of this approach is growing and there are currently some Delphi projects being undertaken in South Africa.

Instead of retiring EBM in its entirety we could focus our attention on improving the quality of our evidence. Traditionally, the ideal trial is described as having high internal validity while maintaining high external validity.12 This balance is unfortunately difficult to achieve and high internal validity often comes at the expense of external validity.13 Internal validity reflects the elimination of bias from a study, ensuring that the findings are representative of the true association between exposure and outcome. Internal validity can be increased, for example by performing larger, registered (and therefore scrutinised) studies with standardised outcome measures and extremely low risk of bias. External validity refers to the degree to which research findings can be applied to other groups or populations.

Clinical scenarios are complex, involving a wide range of variables, while scientific research tends to be simplified. Take the following hypothesis: Does total hip arthroplasty result in a decreased rate of unplanned re-operation when compared to hemiarthroplasty in patients over the age of 50 years? While the research question needs to be pragmatic in order to allow the necessary scientific rigour, and thus maximise internal validity, it can rarely mimic every clinical scenario. Therefore, external validity may be increased, for example, by minimising inclusion criteria (a strategy popular in some of the current large research projects).

Increasing the power of a study can also enhance internal validity. While increasing the sample size remains a valuable strategy, large studies have numerous problems. Caution should be applied when interpreting large studies, as they are more likely to find a formally statistical significant difference for a trivial effect that is not really meaningfully different from the null.4 Ioannidis, therefore, suggests that large studies should ideally target major concepts (rather than specific questions) with a considerably high pre-test probability of being true so that a significant research finding will lead to a post-test probability that would be considered quite definitive. Another strategy to improve external validity would be to keep the source populations closely related to the target population. In other words, the population studies should accurately reflect the population in whom the study findings are to be implemented. This may imply smaller, more focused studies.

Logistical and financial challenges are significant obstacles to large multicentre randomised controlled trials in South Africa. It is probably best to collaborate with international partners, in this regard. However, there is a growing awareness that size does not always matter and a randomised controlled trial may introduce its own biases.14 Where does that leave the South African orthopaedic surgeon interested in research? In my opinion there is still significant value to be had from smaller, well-designed, focused analytical research projects, especially in relation to innovative research fields where the principles and premises are not well established. Performing high-quality research that is based on excellent protocols and uses standardised international outcome measures will allow subsequent incorporation into meta-analysis, which in turn strengthens our evidence base. Ultimately, it is not necessarily the EBO principle that appears to be the major concern but the reliability of the data.

 

References

1. Langton DJ, Jameson SS, Joyce TJ, et al. Accelerating failure rate of the ASR totsl hip replacement, J Bone Joint Surg (Br) 2011;93-B:1011-16.         [ Links ]

2. Prodromidis AD, Charalambous CP. The 6-hour rule for surgical debridement of open tibial fractures: a systematic review and meta-analysis of infection and nonunion rates. J Ortho Trauma 2016;30:397-402.         [ Links ]

3. Rixen D, Steinhausen Ε, Sauerland S, et al. Randomized, controlled, two-arm, interventional, multicenter study on risk-adapted damage control orthopedic surgery of femur shaft fractures in multipletrauma patients. Trials 2016;17:47.         [ Links ]

4. Ioannidis JPA. Why most published research findings are false. PLoS Med 2005;2(8):e124.         [ Links ]

5. Abdullah L, Davis DE, Fabricant PD, et al. Is there truly 'no significant difference'? Underpowered randomized controlled trials in the orthopaedic literature, J Bone Joint Surg Am 2015;97:2068-73.         [ Links ]

6. Klein G. 2014: What scientific idea is ready for retirement? Evidence-Based Medicine. Available from: https://www.edge.org/responses/what-scientific-idea-is-ready-for-retirement. Date last accessed: 17 January 2017.         [ Links ]

7. Hasenboeler EA, Choudry IK, Newman JT, et al. Bias towards publishing positive results in orthopedic and general surgery: a patient safety issue? Patient Safety in Surgery 2007;1:4.         [ Links ]

8. Stahel PF, Mauffrey C. Evidence-based medicine. A 'hidden threat' for patient safety and surgical innovation? Bone Joint J 2014;96-B:997-9.         [ Links ]

9. Every-Palmer S, Howiek J. How evidence-based medicine is failing due to biased trials and selective publication, ƒ Eval Clin Prac 2014;20:908-914.         [ Links ]

10. Gehrke T, Parvizi J. Proceedings of the International Consensus on Periprosthetic Joint Infection, J Arthsoplasty 2014;29(2 Suppl):4.         [ Links ]

11. Hsu C-C, Sandford BA. The Delphi technique: making sense of consensus. Prac Assess Res Eval 2007;12(10):1-8.         [ Links ]

12. Pannucci CJ, Wilkins EG. Identifying and avoiding bias in research. Plat Reconstr Surg 2010;126(2):619-25.         [ Links ]

13. Godwin M, Ruhland L, Casson I, et al. Pragmatic controlled clinical trials in primary care: The struggle between external and internal validity. BMC Med Res Methodol 2003;3:28.         [ Links ]

14. Ornish D. 2014: What scientific idea is ready for retirement? Large randomized controlled trials. Available from: https://www.edge.org/responses/what-scientific-idea-is-ready-for-retirement. Date last accessed: 17 January 2017         [ Links ]

Creative Commons License All the contents of this journal, except where otherwise noted, is licensed under a Creative Commons Attribution License