On-line version ISSN 2309-8309
SA orthop. j. vol.9 n.4 Pretoria Jan. 2010
Dr Rob DachsI; Dr Steve RocheII; Dr Basil VrettosII; Dr Kevin MacIntyreIII; Professor Johan WaltersIV
IMBChB, Registrar. Department of Orthopaedic Surgery, University of Cape Town
IIMBChB(UCT), FCS(Orth), LMCC(Canada), Consultant. Department of Orthopaedic Surgery, University of Cape Town
IIIRegistrar. Department of Orthopaedic Surgery, Stellenbosch University
IVMBChB(UCT), FCS(Orth), Head of Department. Department of Orthopaedic Surgery, University of Cape Town
BACKGROUND: A familiarity with basic musculoskeletal disorders is of vital importance for medical school graduates. The purpose of this study was to assess a group of newly qualified South African medical school graduates commencing internship at Groote Schuur and Tygerberg Hospitals for competency in musculoskeletal injury and disease.
METHODS: An internationally validated competency examination in musculoskeletal medicine was used as the assessment tool. The examination consisted of 25 short-answer questions and was marked using a validated answer key and scoring system. Topics included fractures and dislocations, back pain, arthritis, basic anatomical knowledge and emergencies that require urgent referral to an orthopaedic surgeon.
The study group comprised 79 interns who were in their first postgraduate year at Groote Schuur or Tygerberg Hospitals. The examination was administered during the orientation programme on their first day at work. The examination was also administered to all registrars in orthopaedic surgery at the University of Cape Town. Data was analysed using Stata 11 to estimate percentages and their binomial exact 95% confidence intervals.
RESULTS: The recommended mean passing score for the examination was 73.1 ± 6.8 per cent. The mean score for the 17 orthopaedic registrars was 96.0 per cent, and that for the 79 interns in their first postgraduate year was 45.3 per cent (95% CI 42.3-48.4). Seventy-two (91 per cent) of the 79 interns failed to demonstrate basic competency in the examination.
CONCLUSION: In summary, 91 per cent of medical school graduates in our study failed a valid musculoskeletal competency examination. We therefore believe that medical school preparation in musculoskeletal injury and disease in South Africa is inadequate and that undergraduate training programmes should be reassessed throughout the country.
Musculoskeletal symptoms account for 23% to 27.8% of primary care, and approximately 20% of accident and emergency attendances internationally.1,2,3
A solid knowledge base in orthopaedic medicine should be acquired in medical school and refined during post graduate training. However, only a small percentage of the undergraduate curriculum at South African medical schools is allocated to teaching orthopaedic medicine. The national average is six weeks of time allocated to training, and in some institutions this time is combined with training in other specialties, thereby effectively reducing the exposure to orthopaedic medicine even further. One-third of medical schools do not have on-call duties for their medical students.
The provision of musculoskeletal care comes from a broad spectrum of practitioners, including family practitioners, emergency physicians, rheumatologists, paediatricians and internists. Orthopaedic surgeons currently provide only 6% of musculoskeletal care in many developed countries.4 Good knowledge of the basics in musculoskeletal disorders is therefore essential for all medical school graduates.
In addition community service officers and medical officers in South Africa are exposed to a considerable load of musculoskeletal illness and injury, often without readily accessible specialist support.
Freedman and Bernstein designed a basic competency examination in musculoskeletal medicine.5 The examination was validated by 124 chairs of Orthopaedic residency programmes in the United States. The chairs weighted the questions according to importance and recommended a pass mark of 73.1%.
In their original study from Philadelphia in 1997, Freedman and Bernstein used their examination to assess 85 residents at the start of their first post-graduate year. Seventy (82 per cent) of the residents failed the basiccompetency examination in musculoskeletal medicine.
Al-Nammari, James and Ramachandran from Barts in London assessed 112 interns at the end of their two-year foundation programme. One-hundred-and-two of 112 (91.1%) failed the same musculoskeletal assessment.6
We wanted to investigate the magnitude of this problem in a South Africa context. We applied Freedman and Bernstein's assessment tool to a group of recently graduated doctors during their intern orientation programme.
Materials and methods
We enrolled interns who were in their first post-graduate year in 2010 at Groote Schuur and Tygerberg Hospitals and asked them to complete the Freedman and Bernstein musculoskeletal examination (Table I).5 The test was administered on the first day of the intern orientation programme at the two respective hospitals. Testing was performed with the co-operation of the intern curators. Verbal informed consent was obtained and the examination was anonymous. The participants were asked to record only the medical school from which they obtained their medical degree. No time limit was applied.
All interns who were approached agreed to participate. In total 79 interns completed the examination, 53 from Groote Schuur Hospital and 26 from Tygerberg Hospital.
The Freedman and Bernstein musculoskeletal examination was developed to test how well medical school graduates understood basic musculoskeletal problems. It was produced and later validated by those chairing residency programmes in both orthopaedic and internal medicine.5,7 The pass mark was set at 73.1% by the orthopaedic surgeons and 70% by the physicians. The examination consists of 25 short-answer questions with an open response format. The questions were also weighted according to importance from 0 to 10 by the orthopaedic chairs.
Topics include fractures and dislocations, back pain, arthritis, basic anatomical knowledge and emergencies that require immediate referral to an orthopaedic surgeon. Details on treatment and outcome were omitted from the examination.
The examination was scored anonymously according to the validated scoring system and answer key. In our study, the overall unweighted score was calculated as described in the original paper and the recommended pass mark set at 73.1% as recommended by the 124 chairpersons of orthopaedic residency programmes in the United States. Each question was worth a maximum of 1 point. In order to obtain a score from 0 to 100, raw scores were multiplied by 4.
Weighted scores were also calculated in order to examine the hypothesis that the overall score may inadequately reflect the participants' level of competence in orthopaedic medicine because interns may perform better on the most important questions and worse on the least important ones. For example, question 2: 'What is a compartment syndrome?' was weighted twice as important as question 25: 'What muscle(s) control(s) external rotation of the humerus with the arm at the side?'
As an additional test of validity, the examination was administered to all registrars in orthopaedic surgery at the University of Cape Town. This step was performed to ascertain whether a high score would be attained given an appropriate knowledge of orthopaedics.
Data were analysed using Stata 11 to estimate percentages and their binomial exact 95% confidence intervals.
Overall unweighted score
The mean score for the 17 orthopaedic registrars was 96.0 per cent, and that for the 79 interns in their first postgraduate year was 45.3 per cent (95% CI 42.3-48.4), with a range of 8.0 to 77.0 per cent. Seventy-two interns (91 per cent) had a score of less than 73.1 ± 6.8 per cent and thus failed to demonstrate basic competency on the examination. The scores for the individual questions ranged from as high as 92.3 per cent to as low as 9.0 per cent (Table I).
In order to examine the hypothesis that the interns may have scored well on the most important questions and poorly on the least important questions, a weighted score was calculated. The overall weighted score for all interns was 47.0 per cent (95% CI 43.9-50.1). Seventy-one (90 per cent) of the 79 interns failed the examination when the questions were weighted according to their attributed importance.
Individual component scores
Anatomy-based questions (questions 8, 10, 11, 12, 15, 20, 22, 23, 24, 25) were answered poorly, with an average score of 37.9 per cent (95% CI 33.6-42.3) compared to the overall average score of 45.3 per cent. 'Red flag' questions (questions 2, 4, 5, 6, 7) were better answered, with an average score of 55.3 per cent (95% CI 50.7-59.8), compared to the overall average score of 45.3 per cent.
Our study suggests that the majority of newly qualified South African doctors do not have a basic level of competence in orthopaedic medicine. At the start of their internship, with their final undergraduate medical exams just recently completed, only 9 per cent (7 out of 79) passed the basic Freedman and Bernstein musculoskeletal assessment. This is extremely alarming given the high prevalence of orthopaedic conditions in trauma and general medicine in an environment where a large number of interns, community service officers, medical officers and general practitioners work without specialist support.
However, our findings were comparable with similar studies in other centres. In their original study from Philadelphia in 1997, Freedman and Bernstein assessed 85 residents at the start of their first postgraduate year.5 Seventy (82 per cent) of the residents failed the same musculoskeletal assessment. Al-Nammari, James and Ramachandran (London, 2008) assessed 112 interns at the end of their two-year foundation programme.6 One hundred and two of 112 (91.1%) failed the Freedman and Bernstein musculoskeletal assessment. These studies signal similar concern, internationally, about the adequacy of training among medical practitioners.
The interns performed particularly poorly in the anatomy-based questions, with an average score of 37.9 per cent. While the 'red flag' group of questions was better answered, this is misleading as only half the candidates knew to look for a vascular injury in a patient with a dislocated knee, and only 13 per cent knew the basic management of an open fracture. In addition, only 6 per cent of the study group considered both tumour and infection in someone with lower back pain which woke them from sleep.
We corresponded with our fellow medical schools and collected data relating to the nature of their undergraduate orthopaedic programmes. While our study population was too small to allow any statistically significant sub-analysis, we were able to demonstrate a trend, with the interns who scored higher in the assessment coming from programmes with more time allocated to orthopaedic training and from programmes which included on-call duties for the students. These findings are supported by Freedman and Bernstein's original study, which found that residents who had taken an elective course in orthopaedic surgery in medical school had a significantly higher mean score in the assessment.5
These findings suggest that the undergraduate medical curriculum in South Africa should allow for more exposure to training in orthopaedic medicine. Currently the average length of time set aside for undergraduate orthopaedic teaching in South African medical schools is six weeks, a disproportionate amount of time given the orthopaedic load experienced in our hospitals.
It is also worth considering that the content of the current curriculum may need to be reassessed. In a later study, Freedman and Bernstein asked 240 programme directors of residency programmes in internal medicine to rate the importance of various topics in orthopaedic medicine.7 They suggested the current curriculum in the United States probably over-emphasised surgical practice, and that the 'ideal course in musculoskeletal medicine should concentrate on common outpatient orthopaedic problems, orthopaedic emergencies, and the musculoskeletal physical examination'.
The particularly poor knowledge of anatomy demonstrated by the interns in the assessment suggests that this component should receive increased emphasis in the undergraduate curriculum. A good knowledge of anatomy is essential, not only to understanding any injury or disease process but also to safe practice in any clinical medical field. Our study group scored lowest in the anatomybased questions, with an average score of 37.9 per cent, compared to the overall average score of 45.3 per cent.
Further study to define why our medical students perform poorly is required. A review of the curriculum including aspects like time allocation and content, as well as teaching methods and teacher skills is essential if we are to ensure that medical students, during the course of their undergraduate training, obtain the knowledge and skills which will enable them to go out into the community and practise good medicine.
There are several limitations to this study. Freedman and Bernstein's musculoskeletal examination is the only validated assessment tool currently available. The authors of the questionnaire accept its weaknesses and acknowledge that its validity may be limited by 'the distribution of the topics, the open response format, the wording of the questions, and the accepted answers'.
We used the pass mark of >73.1% established by 124 orthopaedic programme directors instead of that of >70% established by 240 internal medicine programme directors. Selecting the lower pass mark may have changed some of our findings. Our study may have been limited by sample bias, with participants coming exclusively from two large tertiary hospitals in Cape Town. Our intern group did however include graduates from all of the South African medical schools.
Conflict of interest statement
The content of this article is the sole work of the authors. We, the authors, confirm there is no financial or personal relationship with other people or organisations that may have inappropriately influenced this work. All authors were fully involved in this study and the preparation of the manuscript. We confirm that the material has not been submitted for publication elsewhere. Approval by Ethical Committee: Yes Informed consent was obtained: Yes. No benefits of any form have been received from a commercial party related directly or indirectly to the subject of this article.
Statistical analysis by Dr H Carrara, Analytical Epidemiologist/Biostatiostician, School of Public Health and Family Medicine, University of Cape Town.
1. Kahl LE. Musculoskeletal problems in the family practice setting: guidelines for curriculum design. J Rheumatol 1987;14:811-14. [ Links ]
2. Spitzer WO, Harth M, Goldsmith CH, et al. The arthritis complaint in primary care: prevalence, related disability and costs. J Rheumatol 1976;3:88-99. [ Links ]
3. De Lorenzo RA, Mayer D, Geehr EC. Analyzing clinical case distributions to improve an emergency medicine clerkship. Ann Emerg Med 1990;19:746-51. [ Links ]
4. Karpman RR. Musculoskeletal disease in the United States: who provides the care? Clin Orthop 2001;385:52-56. [ Links ]
5. Freedman KB, Bernstein J. The adequacy of medical school education in musculoskeletal medicine. J Bone Joint Surg [Am] 1998;80-A:1421-27. [ Links ]
6. S. S. Al-Nammari, B. K. James, and M. Ramachandran. The inadequacy of musculoskeletal knowledge after foundation training in the United Kingdom. J Bone Joint Surg Br, Nov 2009; 91-B: 1413-18. [ Links ]
7. Freedman KB, Bernstein J. Educational deficiencies in musculoskeletal medicine. J Bone Joint Surg [Am] 2002;84-A:604-8. [ Links ]
Dr Steve Roche
Department of Orthopaedic Surgery University of Cape Town
H49 Old Main building Groote Schuur Hospital Observatory 7925
Tel: (021) 404 5108