RESEARCH

 

Perceptions of changes made to a clinical skills curriculum in a medical programme in South Africa: A mixed methods study

 

 

S R Pattinson^I; P McInerney^II

^IMB ChB, PGDip HSE, MHSE; Unit for Undergraduate Medical Education, School of Clinical Medicine, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa
^IIPhD; Centre for Health Science Education, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa

Correspondence

 

 

---

ABSTRACT

BACKGROUND: In 2015, a medical curriculum review at the University of the Witwatersrand, Johannesburg, South Africa, identified too large a gap between the medical school-based teaching in the fourth year of the course and the hospital-based teaching in the fifth year, when students begin their clinical clerkships. A number of changes were made to the curriculum to improve the preparation of students for the expectations of the clinical setting
OBJECTIVES: To determine students' perceptions of how well their clinical skills curriculum in the fourth year prepared them for their clinical clerkships in the fifth year
METHODS: An exploratory mixed methods approach was used. Phase I was a narrative qualitative study. The data underwent qualitative analysis and the categories that emerged informed the development of a questionnaire for phase II. This was a cross-sectional, comparative, quantitative study comparing students taught in the new curriculum (2018; fifth-year students (n=299)) with those taught in the old curriculum (2018; sixth-year students (n=291
RESULTS: The fifth- and sixth-year students had response rates of 50% and 34%, respectively. The results showed a perception of improved preparation for clinical clerkship through the changed clinical skills training. The p-value for 14 of the 16 questions was <0.05, with a 95% confidence interval, indicating that the difference between the two cohorts was statistically significant
CONCLUSION: The new curriculum has resulted in a significant improvement in students' perceived preparation for their clinical clerkships

---

 

 

In 2015, a review of the University of the Witwatersrand, Johannesburg, South Africa (SA) MB BCh curriculum identified that there was too large a gap between the medical school-based teaching in the fourth year of study and the hospital-based teaching in the fifth year, when students begin their clinical clerkships (K Mfenyana et al., Medical and Dental Professions Board of the Health Professions Council of South Africa (HPCSA) accreditation of undergraduate education and training in medicine in the MB BCh programme, University of the Witwatersrand, 2015 - unpublished; R Rispel et al., Quinquennial review, University of the Witwatersrand, 2015 - unpublished). The reviewers found that there was a theory-practice gap; the students were underprepared for the application of clinical skills in a real clinical setting with patients. This difficult transition from the medical school-based curriculum to the hospital-based curriculum has been identified as an important problem in need of attention in medical education,^[1,2] as it results in student emotional distress, stress, burnout^[3,4] and unpreparedness that puts patient safety as risk.^[5]

In an attempt to bridge this gap and remedy this concern, a number of changes were made to the fourth-year clinical skills curriculum for 2017. These changes were based on the latest medical education literature and consultations with staff and students. Adjustments to the curriculum included centralising the teaching of clinical skills in a clinical skills unit and standardising the content and objectives of the teaching, which were aligned with the theoretical curriculum. More teachers were employed to improve the tutor-student ratio and allow for small-group learning and increased monitoring and support. The clinical skills sessions were designed to allow for supervised deliberate practice^[6] with simulated patients and models in small groups. The sessions included feedback on practice, followed by interactive debriefing. Formative assessment was introduced in the form of regular tutor-student assessments and formative objective structured clinical examinations (OSCEs) with feedback. The weighting of the final summative OSCE was increased to communicate its importance to the students. A clinical methods programme was also introduced to allow for early application of clinical skills in a real-life clinical setting. This was in the form of structured bedside-teaching tutorials, based on clearly defined outcomes, and was facilitated by the clinical departments. The structured tutorials replaced less structured clinical encounters, where learning tended to be opportunistic.

The literature on teaching clinical skills recommends the use of simulation to bridge the theory-practice gap, decrease the cognitive load and support this challenging transition.^[5,7,8] The key aspects of successful clinical skills training identified in the literature include clear outcomes,^[5] standardised and structured sessions,^[9,10] deliberate practice,^[6] a safe mistake-forgiving environment^[1] and regular feedback and debriefing.^[1] Small-group teaching allows for teamwork and collaboration among students and improved monitoring and supervision from tutors, who can act as role models and mentors and encourage open discourse.^[11-13] Other evidence in the literature includes systematically assessing clinical skills to drive learning, informative feedback on assessment and elevating the importance of clinical skills in the curriculum.^[14,15] Ramani^[14] recognises the importance of having both simulated clinical skills teaching and hospital-based bedside clinical skills teaching in an integrated curriculum for the successful development of clinical skills. Peters and ten Cate^[16] identify the bedside as a valuable opportunity to integrate and improve skills.

While the evidence suggests that changes made to the new curriculum should help with this difficult transition, none of the research identified was conducted in the SA medical education context. The SA context provides unique challenges, including financial constraints, limited resources, a need for generalist, primary care over specialist training and a demand for doctors who can provide care in rural areas and meet the burden of the HIV epidemic.^[17] Limited human resources, particularly of academic staff who have to meet a high clinical burden and find time for teaching, is a major constraint.^[17] Our student group is diverse and comes from a wide range of educational backgrounds, including many underprivileged and previously disadvantaged students.^[17] It is important to determine what influence these curriculum changes have had on students' preparation for their clinical years in the current context. One of the ways to evaluate the curriculum change is to explore the students' perceptions of how well prepared they felt for the clinical setting, based on their experiences in their clinical clerkships. The perceptions of students who were taught under the new and old curricula were compared with regard to the question: Have these changes had the desired outcome in the current context and to what extent have the problems identified been resolved?

 

Methods

An exploratory, sequential mixed methods study design was adopted, with a dominantly quantitative paradigm.^[18] This is represented based on Creswell's notation in mixed methods research, as follows: Qual -> QUANT -> Interpretation^[18]

As a suitable, validated quantitative tool could not be found, the researchers developed such a tool for the purpose of gathering data regarding the students' perceptions of their preparation for clinical clerkships by means of the clinical skills curriculum.

Phase I was a narrative qualitative study. An open-ended question was used to identify the themes, ideas, concerns and perceptions of the students. These were used to inform the development of a tool for the collection of the quantitative data in the second phase of the study, which accurately measured these constructs while exploring the research question. The following question was posed in phase I: What is your perception of the fourth-year clinical skills curriculum's preparation of you for your clinical rotations in fifth year? Systematic sampling was used; every tenth fifth-year and sixth-year student on an alphabetised class list was emailed a link to an online system (Research Electronic Data Capture (REDCap)). A total of 62 students were sampled (31 from the fifth year and 31 from the sixth year).

Phase II was a cross-sectional, comparative, quantitative study. A preliminary questionnaire was developed based on the objectives, a literature review and the researchers' experiences of teaching in the programme. The qualitative analysis of the narrative data from the open-ended question in phase I informed the modification of the phase II questionnaire to best reflect the students' ideas, concerns and expectations. This process included assessing the relevance and value of the questions already included in the questionnaire (13 questions) and 3 additional questions that measured the constructs identified as dominant categories, as described under Results, phase I, below.

The questionnaire was piloted on the target population of the study (20 students from the fifth year and 20 from the sixth year, using systematic sampling). There were no problems or errors detected with the administration of the questionnaire, understanding of the questions or the collection of data. No changes were made to the data collection tool.

In phase II, the refined questionnaire, consisting of 5-point Likert scale-type questions (n=16), was distributed to the entire target populations of the fifth-year (n=299) and sixth-year (n=291) students. The Likert scale questions had 5 options (strongly disagree, disagree, neutral/undecided, agree, strongly agree), which were assigned values of 1 - 5 to allow for numerical data for quantitative analysis. A research database was created using the REDCap (version 8.4.5) tool, and the link to the questionnaire was made available online. The link was then emailed to all students in the target populations, except those who had been invited to participate in the pilot study (fifth-year sample population (n=299); sixth-year sample population (n=291)). Data were collected for 80 days after distribution of the questionnaire to allow for maximum opportunity for responses within the time constraints of the study. Fifth-year students had completed at least one, and at most four, of their clinical rotations at the time of data collection. Data were captured anonymously using the REDCap tool, and were exported for analysis.

Ethical approval

This research was approved unconditionally by the Human Research Ethics Council (HREC), University of Witwatersrand (ref. no. M171052).

 

Results

Phase I

Of the 62 students included in phase I, 28 responded - a response rate of 45%. An inductive and iterative approach was used to identify patterns, similarities and differences in the narrative texts. Data were coded into meaningful segments and then into frequent, dominant and significant categories for interpretation into findings and conclusions. Three categories emerged from the data. The first was that the students felt underprepared to detect actual pathology in the clinical setting. They indicated that, having learnt skills on healthy simulated patients and models, they struggled to confidently detect and interpret clinical signs and abnormalities, making transfer to the clinical setting difficult. The second category, closely related to the first, was that they felt that they had had insufficient exposure to real patients in their preparation for their clinical clerkships. The third category related to the high student-tutor ratio, which prevented students from getting hands-on experience with a skill and being able to perform the skill with guidance and feedback. All 13 questions of the preliminary, self-developed questionnaire were retained and 3 additional questions that measured the abovementioned constructs were added.

Phase II

Data from the Likert scale-type questions were analysed as ordinal, as although the categories have rank order, the distances between them cannot be presumed to be equal.^[19] A total of 247 students (n=590) responded to the questionnaire, giving a response rate of 42%, which met the requirements for a 95% confidence level and a confidence interval (CI) of 5 for the total sample. Of the respondents, 149 were in their fifth year, giving a response rate of 50% for this class of 299 students. Of the sixth-year class (n=291), 98 students responded - a response rate of 34%. While neither of these subsamples met the requirements for a confidence level of 95% and a CI of 5, they were still above the average online response rate of 33%^[20] and did meet the liberal conditions set for an adequate response rate, a 10% sampling error and an 80% confidence level.^[20] The response rate with regard to the questionnaire may have been limited by the reliance on responses to emails, the questionnaire being online rather than a face-to-face engagement, survey fatigue and limited time available for data collection.^[20]

A Shapiro-Wilk test was performed on each of the closed-ended questions. The test was performed on the data for the two groups combined and for each group independently for each question. The results indicated that the data were not normally distributed, and therefore non-parametric tests would need to be employed.

The means, medians and standard deviations (SDs) were calculated for every closed-ended question in the questionnaire for each cohort. The means for the fifth-year sample were higher than those for the sixth-year sample for every question. The means for 13 of the 16 questions in the fifth-year sample were >3, indicating a generally more positive view. Three questions had means <3. The first was their perception of the difficulty of the transition from the fourth year to the fifth year of study. The other 2 questions related to their perception of how the clinical skills curriculum and the hospital-based curriculum, respectively, prepared them for performing procedures. In the sixth-year cohort, 13 of the 16 responses had means <3, indicating a more negative view of the curriculum.

Because the ordinal data were not normally distributed, the non-parametric Mann-Whitney U-test was used to compare the responses of the two cohorts for each question to test for significance (Table 1). The difference between the two groups was statistically significant for 14 of the 16 questions (p<0.05; 95% CI). The only 2 questions that did not have a statistically significant difference were the 'difficulty in the transition to fifth year from fourth year' and the 'ability to transfer skills learnt on manikins to real-life patients'. The mean for both groups for these 2 questions was <3, showing that both groups had a more negative perception for these questions.

 

Discussion

The mean values for each question were consistently higher in the fifth-year sample. This suggests better preparation among fifth-year students for their clinical clerkships.

The first statement in the questionnaire was: 'The clinical skills teaching curriculum in fourth year adequately prepared me for application of clinical skills in the clinical setting in fifth year.' There was a statistically significant difference between the two groups, showing that the students who were taught under the new curriculum had a perception of markedly improved preparation for the clinical setting. This significant finding speaks directly to the research question. The second statement required the students to respond to: 'The transition from the fourth year to the fifth year was very difficult.' The Mann-Whitney 17-test for this question showed no statistically significant difference between the two groups; both perceived the transition as difficult. While the previous statement showed that fifth-year students felt better prepared than sixth-year students, the former still found the transition challenging. The transition from the medical school setting to the clinical setting will always be challenging; the academic, skill and emotional demands of performing on real patients are high. Okuda et al.,^[1] as well as Eyal and Cohen,^[2] recognise that students often feel inadequately trained and ill-equipped for the clinical setting. Therefore, finding a way to continue to improve the curriculum to reduce this difficulty is critically important.

The data showed a significantly better perception among students taught under the new curriculum of how well equipped they were with regard to the skills needed to meet the expectations of the fifth year and how confident they were in performing the clinical skills they learnt in the fourth year. The data showed an improved ability to transfer skills learnt and practised with simulated patients to real patients, but indicated that both cohorts found it difficult to transfer skills learnt on manikins to real patients. It may be that the learning done on manikins too distantly resembled the real-life experience of performing a procedure on a patient, and thus the students struggled to transfer skills.^[21] This is an area that needs revision and improvement in the new curriculum.

Even though there was a notable improvement from the old curriculum in the students' perceived ability to identify clinical findings in patients with pathology, students taught under the new curriculum were neutral with regard to their ability to detect pathology. Remmen et al.^[7]recommend that students need a longitudinal, integrated curriculum with increased clinical exposure. De Boulay^[22] suggests that the best way to learn pathology is in an integrated, interactive curriculum, where students are asked to solve real clinical problems. As the perceived lack of preparedness to detect pathology is a concern for teachers, improvement in the integration of the curriculum and increased clinical experience earlier in the curriculum need to be considered. Fifth-year students had a perception of being significantly better prepared to take histories, physically examine patients and perform clinical procedures than those in the sixth year. The mean values for both groups were low regarding the perception of their preparation for the performance of clinical procedures on real patients. This is possibly related to the earlier finding that it is difficult for students to transfer skills from practise on manikins. While examination and history taking are taught using simulated patients in the theoretical setting and on real patients at the bedside, clinical procedures are taught on manikins. Moss et al.^[23] suggest that doctors underestimate the anxiety of students when having to perform simple practical procedures, indicating that they need better preparation to increase confidence, as well as more support and guidance when they first perform the procedures on real patients.

Fifth-year students had a significantly more positive view regarding the student-tutor ratio and the resources available for learning clinical skills. This justifies the employment of new tutors to allow for effective small-group teaching. Likewise, they had a significantly better perception than those in the sixth year of how well the clinical methods curriculum prepared them for their clinical clerkships. This may be attributed to the value of well-structured bedside teaching methods with clear objectives that are aligned with the curriculum.^[24]

 

Conclusion

The new curriculum has resulted in a significant improvement of the students' perceived preparation for their clinical clerkships compared with those taught under the old curriculum. The new clinical skills curriculum has had a positive effect on, and has begun to address, the concerns raised by the curriculum review committee.

Improvements need to be made to further reduce the difficulty of the transition to the clinical setting, advance the detection of pathology and make more effective use of manikins for preparing students for the performance of clinical procedures.

Several recommendations for practice can be identified from the results of the research. The changes led to a perceived improvement in preparation among the students being taught under the new curriculum. These changes need to be reinforced and strengthened, particularly through faculty development in learning about and applying these teaching and learning strategies. There needs to be an effort to find more ways to make the transition from the theoretical setting to the clinical setting less difficult.

The amount of time spent in the real-life clinical setting and on the practical, clinical aspects of the curriculum needs to be increased and needs to start earlier in the curriculum. The students' learning opportunities should more closely resemble the real-life clinical setting to better prepare them for the expectations and demands. This applies to the use of manikins, which, in addition, needs more time for deliberate practice and feedback.

The changes made to the curriculum have resulted in students' perception of improved preparation, but further research is needed to determine if these changes have advanced the achievement of curricular outcomes in the clinical setting and improved patient outcomes, as well as having a positive effect on reducing students' emotional distress, stress and burnout.

Study limitations

As they were in the second year of their clinical clerkships at the time of data collection, the perceptions of sixth-year students of how well prepared they were when they started their clerkships may have been affected by how much time had passed and the experiences they had had in the clinical setting since then. This may have limited the comparison between the 2 years.

Because of the timing of the research, the fifth-year students had only completed 1, 2, 3 or 4 of their clinical rotations and had to evaluate their perceptions of their preparedness for their clinical clerkship based only on those rotations.

Declaration. This research was done in partial fulfilment of SRP's postgraduate degree (Master in Health Sciences Education).

Acknowledgements. Colleagues from the Unit for Undergraduate Medical Education at the University of the Witwatersrand Medical School for their advice, support and assistance during the conduct of the study.

Author contributions. SRP: conception of research question, collection and analysis of data for a higher degree; PM: supervised and contributed towards development of the study and analysis of the findings.

Funding. None.

Conflicts of interest. The researcher (SRP) is a clinical lecturer at the University of the Witwatersrand Medical School, teaching clinical skills to the third- and fourth-year students in the revised clinical skills curriculum. The data were gathered in the fifth and sixth years of study, during which time he had no part in the facilitation, teaching or assessment of the students, nor will he have for the remainder of their studies.

 

References

1. Okuda Y, Bryson EO, DeMaria S Jr, et al. The utility of simulation in medical education: What is the evidence: Mt Sinai J Med 2009;76(4):330-343. https://doi.org/10.1002/msj.20127        [ Links ]

2. Eyal L, Cohen R. Preparation for clinical practice: A survey of medical students' and graduates' perceptions of the effectiveness of their medical school curriculum. Med Teach 2006;28(6):162-170. https://doi.org/10.1080/01421590600776578        [ Links ]

3. Schiller JH, Stansfield RB, Belmonte DC, et al. Medical students' use of different coping strategies and relationship with academic performance in preclinical and clinical years. Teach Learn Med 2018;30(1):15-21. https://doi.org/10.1080/10401334.2017.1347046        [ Links ]

4. Dendle C, Baulch J, Pellicano R, et al. Medical student psychological distress and academic performance. Med Teach 2018;40(12):1257-1263. https://doi.org/10.1080/0142159X.2018.1427222        [ Links ]

5. Cleland J, Patey R, Thomas I, Walker K, O'Connor P, Russ S. Supporting transitions in medical career pathways: The role of simulation based education. Adv Simul 2016;1(14):1-9. https://doi.org/10.1186/s41077-016-0015-0        [ Links ]

6. Duvivier RJ, van Dalen J, Muijtjens AM, Moulaert VRMP, van der Vleuten CPM, Scherpbier AJJA. The role of deliberate practice in the acquisition of clinical skills. BMC Med Educ 2011;11(101):1-7. https://doi.org/10.1186/1472-6920-11-101        [ Links ]

7. Remmen R, Scherpbier A, van der Vleuten C, et al. Effectiveness of basic clinical skills training programmes: A cross-sectional comparison of four medical schools. Med Educ 2001;35(2):121-128. https://doi.org/10.1111/j.1365-2923.2001.00835.x        [ Links ]

8. Teteris E, Fraser K, Wright B, McLaughlin K. Does training learners on simulators benefit real patients: Adv Health Sci Educ 2012;17(1):137-144. https://doi.org/10.1007/s10459-011-9304-5        [ Links ]

9. Remmen R, Derese A, Scherpbier AJJA, et al Can medical schools rely on clerkships to train students in basic clinical skills? Med Educ 1999;33(8):600-605. https://doi.org/10.1046/j.1365-2923.1999.00467.x        [ Links ]

10. Remmen R, Scherpbier AJJA, Derese A, et al. Unsatisfactory basic skills performance by students in traditional medical curricula. Med Teach 1998;20(6):579-582. https://doi.org/10.1080/01421599880328        [ Links ]

11. Zeng R, Xiang L, Zeng J, Zou C. Applying team-based learning of diagnostics for undergraduate students: Assessing teaching effectiveness by a randomized controlled trial study. Adv Med Educ 2017;8(1):211-218. https://doi.org/10.2147/AMEP.S127626        [ Links ]

12. Jacques D. ABC of learning and teaching in medicine - teaching small groups. BMJ 2003;326(7387):492-494. https://doi.org/10.1136/bmj.326.7387.492        [ Links ]

13. Middendorf J, Pace D. A model for helping students learn disciplinary ways of thinking. New Direct Teach Learn 2004;2004(98):1-12. https://doi.org/10.1002/tl.142        [ Links ]

14. Ramani S. Twelve tips for excellent physical examination teaching. Med Teach 2008;30(9-10):851-856. https://doi.org/10.1080/01421590802206747        [ Links ]

15. Leppink J, Duvuvier R. Twelve tips for medical curriculum design from a cognitive load theory perspective. Med Teach 2016;38(7):669-674. https://doi.org/10.3109/0142159X.2015.1132829        [ Links ]

16. Peters M, ten Cate O. Bedside teaching in medical education: A literature review. Med Educ 2014;3(2):76-88. https://doi.org/10.1007/s40037-013-0083-y        [ Links ]

17. Kent A, de Villiers MR. Medical education in South Africa - exciting times. Med Teach 2007;29(9-10):906-909. https://doi.org/10.1080/01421590701832122        [ Links ]

18. Creswell JW, Creswell JD. Research Design. 5th ed. Los Angeles: SAGE, 2018.         [ Links ]

19. Norman G. Likert scales, levels of measurement and the 'laws' of statistics. Adv Health Sci Educ 2010;15(5):625-632. https://doi.org/10.1007/s10459-010-9222-y        [ Links ]

20. Nulty DD. The adequacy of response rates to online and paper surveys: What can be done? Assess Eval High Educ 2008;33(3):301-314. https://doi.org/10.1080/02602930701293231        [ Links ]

21. Herrmann-Werner A, Nikendei C, Keifenheim K, Bosse HM, Lund F, Wagner R. 'Best practice' skills lab training vs. a 'see one, do one' approach in undergraduate medical education: An RCT on students' long-term ability to perform procedural clinical skills. PloS ONE 2013;8(9):e76354. https://doi.org/10.1371/journal.pone.0076354        [ Links ]

22. Du Boulay C. Learning pathology: Why? how? when? J Clin Pathol 1997;50(8):623-624. https://doi.org/10.1136/jcp.50.8.623        [ Links ]

23. Moss HA, Derman PB, Clement RC. Medical student perspective: Working toward specific and actionable clinical clerkship feedback. Med Teach 2012;34(8):665-667. https://doi.org/10.3109/0142159X.2012.687849        [ Links ]

24. Ramani S. Twelve tips to improve bedside teaching. Med Teach 2003;25(2):112-115. https://doi.org/10.1080/0142159031000092463        [ Links ]

 

 

Correspondence:
S R Pattinson
stuart.pattinson@wits.ac.za

Accepted 11 October 2019