RESEARCH

 

A mixed-methods study of students' research assignments of a Master's in the clinical epidemiology programme

 

 

II Kallon^I; TM Esterhuizen^II; T Young^III; M McCaul^IV

^IPhD, MTech (Envr Health); Centre for Evidence-based Health Care, Division of Epidemiology and Biostatistics, Department of Global Health, Faculty of Medicine and Health Sciences, Stellenbosch University, Cape Town, South Africa
^IIMSc ; Division of Epidemiology and Biostatistics, Department of Global Health, Faculty of Medicine and Health Sciences, Stellenbosch University, Cape Town, South Africa
^IIIPhD, MMed (Public Health), FCPHM (SA), MB ChB ; Centre for Evidence-based Health Care, Division of Epidemiology and Biostatistics, Department of Global Health, Faculty of Medicine and Health Sciences, Stellenbosch University, Cape Town, South Africa
^IVPhD, MSc (Clin Epi); Division of Epidemiology and Biostatistics, Department of Global Health, Faculty of Medicine and Health Sciences, Stellenbosch University, Cape Town, South Africa

Correspondence

 

 

---

ABSTRACT

BACKGROUND: Evaluation of postgraduate programmes is limited in Africa, especially in the design of academic programmes and the breadth of a student's research and throughput. We reviewed students' research assignments and explored examiners' feedback on these assignments of a Master of Science in Clinical Epidemiology (MClinEpi) at Stellenbosch University.
OBJECTIVES: To classify and describe the research of the MClinEpi students that they submitted for examination from 2008 to 2020, and to describe the contents of internal and external examiner feedback.
METHODS: We conducted a retrospective, mixed-methods descriptive study of students' research submitted to examiners between 2008 and 2020 in two phases. The first phase included a quantitative description of the research topics, study designs and publications. The second phase was a quantitative and qualitative description of examiners' feedback.
RESULTS: During the study period, 161 of a total of 193 (83%) students graduated. Over a third of the students (n=58, 36%) conducted research in infectious diseases, with about half (n=83, 53%) using cross-sectional study designs. Nearly half of the students (n=65, 41%) passed in the range of 60 - 69% and 84 (53%) published their research, mostly in international journals (n=60, 71.4%). From the review of the examiners' reports, students demonstrated adequate understanding of concepts and the literature, different levels of grasping the methodological approach and originality in selection of topics relevant to the local settings, but struggled to adequately discuss and summarise the findings of their study.
CONCLUSION: The programme showed a consistently high pass rate and relatively successful publication rate. Further strengthening of the teaching and learning of research is imperative to drive successful output and competence among students. Dedicated formative assessments, driving research teaching and learning, can be implemented together with regular curriculum evaluations.

Keywords:  Clinical epidemiology programme, research assignments, mixed-methods.

---

 

 

Clinical epidemiology, a merger of clinical medicine and epidemiology,^[1] is the science of applying the best available research evidence to patient care. Its application is especially important today, given the overwhelming quantity of both accurate and inaccurate evidence available, artificial intelligence and the extent of research waste globally.^[2] Over the past two decades, important strides have been made to advance evidence-based healthcare, as well as clinical epidemiology, regionally, nationally and at local levels in African countries.^'31 This is notably so in developing biostatistics and evidence synthesis capacity, including conducting and disseminating various types of evidence synthesis, and facilitating that reliable research continues to inform decision-making on the continent.^[3]

The Faculty of Medicine and Health Sciences at Stellenbosch University convenes a Master of Science in Clinical Epidemiology (MClinEpi) programme. It offers rigorous methodological training for those with a background in a health-related discipline who wish to pursue a career in clinical research or evidence-based practice. To date, evaluation of clinical epidemiology programmes has been few and far between,^'4-61 with limited research in sub-Saharan Africa.^[7] Previous research has argued that despite the challenges encountered, MClinEpi programmes in Africa are still able to advance the capacity and roles of health professionals in advancing evidence-informed practices in the region.^[7] Although the impact of COVID-19 exposed the inequalities that exist in accessing optimal education, including increased knowledge of digital learning,^[8] it is assuring to know that research from other countries indicates that training in clinical epidemiology builds future leaders, including building capacity in others^[5]

The MClinEpi programme at Stellenbosch University is a structured (coursework) Master's degree. The programme offers a blend of face-to-face contact sessions, online synchronous learning and self-study. Students are expected to complete 10 modules covering various topics, totalling 120 credits, and to conduct research (60 credits) under supervision, which is then submitted for examination as a publishable article to an internal and external examiner. In 2017, the 100th student to receive an MClinEpi from Stellenbosch University was announced in less than a decade of programme implementation. Currently, the university is close to graduating the 200th student. The programme has steadily produced graduates from around Africa and beyond (Fig. 1). This article reflects on the scope and outcomes of the research of graduates of this programme, and conducts an in-depth analysis of examiner feedback to inform strategies to strengthen research output, quality and throughput. It provides valuable insights into future proofing of the MClinEpi programme for the next decade, strengthening the teaching of research in the faculty and beyond.

 

Methods

Objectives and research design

The first objective of this study was to classify the research assignments of the MClinEpi students submitted from 2008 to 2020. The second objective was to describe the contents of internal and external examiners' feedback of these assignments within the same study period. The study was conducted using a retrospective, mixed-methods descriptive design implemented in two phases. The first phase was a quantitative review of the research topics, study designs, pass rates and publications of all students' research submitted for examination in the study period. The second phase was an exploration of examiners' feedback. This design allows for a convergent mixed-methods approach,^[9,10] where examiners provided a quantitative assessment followed by a qualitative narrative of students' research.

Subjects

In both scenarios, the unit of analysis was the students' research.

Data collection and management

In phase one, the authors developed a pre-piloted data extraction form to collect data, which were captured into an Excel data set and imported into SPSS software package version 28 (IBM, USA) for analysis. This extraction form was developed by the team, and included general information, such as the students' assignment identification number (ID), topic discipline and year of submission and completion. It also captured the number and designation of supervisors, research types and paradigm. Students' hard copy research and examiner reports were accessed from the administration office or e-thesis library, which routinely keeps all examination reports and students' research (from 2008 to date). The data extraction form was tested on a sub-sample of two research submissions and internally revised by the study team. Revision was incorporated as required. All data were extracted by the first author and checked by the co-authors.

In phase two, all examiner reports (n=202) were included in the analysis to maximise depth of data and to inform internal recommendation for the Master's programme. Examiner reports were typically presented as structured examination sheets (quantitative data) and narratives (qualitative data). Examiner reports were individually linked to student research assignments, with a unique student ID. The qualitative data were imported into Atlas.ti Web version 22^[11] for analysis.

Data analysis

The phase one data were analysed using standard descriptive statistics. Categorical data were tabulated as frequencies with percentages. Continuous data were presented as means, depending on the data distribution. Data were reported graphically where appropriate, over time (years) and by study design. In phase two, the examiners' narrative reports were analysed using thematic content analysis to develop themes from the data. We used a hybrid coding format and a deducted and inductive approach. An initial sample of five randomly selected research projects was taken to develop a codebook to guide thematic analysis and reporting of results. Examiners' reports were purposely selected from each year to have a wider range of feedback and reports of varied length. Feedback from 65 examiners reached saturation, as similar narratives were emerging from the data. Of the 65 examiners, 27 were internal and 38 were external. The codebook was discussed and refined by the study team in an iterative process. Using the convergent mixed-methods approach, we drew conclusions from the analysis of both datasets, informed by a checklist for reporting a mixed-methods study.^[12] For example, the examiners' narratives clarified some of the findings from the quantitative data. This is seen where students' grades in specific areas of their research were aligned with examiners' feedback. This convergent analysis provided a better understanding of the strengths and weaknesses of the students' research outputs.

Reflexivity

Reflexivity was observed in phases of the research process.^[13,14] Firstly, all authors considered their own academic positions and bias when interpreting and drawing conclusions, especially in phase two of the study, where we could see the names of examiners and their feedback. We ensured that any personal practices and/or differences regarding examining students' marks did not influence our selection of the sample of examiners' reports, as the data extraction was done by the first author, who did not know any of the examiners. The verification process was done by the other authors in the research team. Furthermore, even though the first author could see the names of the examiners, neither their names nor style of examining students' research was considered when sampling the scripts. As with the data extraction, the initial analysis was done by the first author, who did not teach on the MClinEpi programme and was not familiar with the students or examiners of the programme. We ensured trustworthiness of the findings through iterative discussions among the research team to verify the analysis of both datasets.

 

Results

Description of research and marks

A total of 227 students enrolled for the MClinEpi programme between 2008 and 2020. Thirty-four were still registered by 2020, 32 did not graduate, and 161 (83%) graduated from the programme by December 2020. Only examined research and graduated students were included in the analysis, as this cohort is the article's primary focus.

More than a third of the students (n=58, 36%) conducted research on infectious diseases, followed by non-communicable diseases (n=41, 26%). Designs were mostly cross-sectional (n=83, 53%) and research syntheses (n=42, 27%) (Fig. 2). Slightly over half of the students (n=84, 53%) published their research results, mostly in international peer-reviewed journals (n=60, 71%) (Fig. 3). Most publications were seen among students who conducted research synthesis (n=29, 67%).

Students spent a median of 3 years to complete the programme. Just under half of the students (n=65, 41%) passed in the range of 60 - 69%, with 21 (13%) distinctions (>75%) (Fig. 3). There were 45 cum laude passes between 2010 and 2020. Based on examiners' (internal and external) feedback on an average of 196 students' research reports, most students (n=118, 60%) scored in the excellent or good category (58 - 63%) in the introduction, methods and other features of their research, with n=92 (47%) in the results, discussion and summary sections (44 - 52%) (Table 1).

Review of examiner reports

Feedback from 65 examiners was reviewed when the qualitative analysis reached theoretical saturation. Twenty-seven were internal examiners and 38 were external examiners. All the reports were in a narrative format. Four key themes emerged: Theme 1: Adequate understanding of the concepts and relevant literature; Theme 2: Different levels of grasping the extent of the various research methods demonstrated when interpreting their research results; Theme 3: Struggle to discuss the findings and provide a clear interpretation of the findings of their research; and Theme 4: Examiners were impressed with the originality of the research and the relevance to the local settings.

Theme 1: Adequate understanding of the concepts and relevant literature

Students demonstrated adequate understanding of the concepts and relevant literature linked to their research topics. Some examiners mentioned that students were able to succinctly define the field of study and provided good motivation for the study that they undertook:

'The field is succinctly defined in the background and the need for the review well motivated.' (Examiner 2, 2010)

'The student has the ability to engage with relevant literature and build a case for her research, although more relevant literature pertaining to similar environments to that of the research would have been welcomed.' (Examiner 182, 2018)

Even though some examiners explained what could have been done better by the students, they reported that generally the students demonstrated an understanding of the topic researched and concepts used.

Theme 2: Different levels of grasping the methodology and presentation of results

Examiners' feedback showed that students demonstrated different levels of grasping of what was required in the methodology section of their assignments and the way they needed to present the results. Some students had a good understanding or adequate command of methods. They also presented their findings well, but other students needed to clarify certain areas. Therefore, some of them did not thoroughly demonstrate the insights of all the required concepts to score higher marks from their assignments, in spite of having a well-designed methodology and presentation of results:

'Command of the appropriate research methodology is clearly demonstrated in the manuscript.' (Examiner 38, 2013)

'... methods, study setting, it would make sense to provide the population substructure since this is where the study was focused.' (Examiner 53, 2014)

'Candidate shows a clear understanding of the Cochrane methodology for systematic reviews.' (Examiner 29, 2013)

Some of the students could not clearly explain the differences within the target population they studied. The differences in presenting the results ranged from stylistic format and errors to omitting relevant parts of the methods. However, generally these were also adequate.

Theme 3: Inadequate discussion and interpretation of the findings

Even though students demonstrated an adequate understanding of the concepts and relevant literature, many examiners reported that students found it hard to adequately discuss and summarise the findings of their study:

'The findings are reasonably well described but the findings are not adequately discussed. How do these findings compare to other reviews that have been conducted?' (Examiner 70, 2015)

'The author should consider that the conclusions should relate to the study population (Examiner 155, 2017)

Some of the problems mentioned by examiners were students not being able to critically interpret the findings in light of strengths and limitations of other studies.

Theme 4: Originality and topics relevant to the local setting

Various examiners were satisfied with the selection of topics and reported that the findings of the studies were very relevant to the countries and health systems in which the studies were conducted. Furthermore, some of the studies showed originality:

'The candidate has selected an important area of research. The research is more relevant to South Africa.' (Examiner 180, 2018)

'To my knowledge this is the first systematic review of this topic.' (Examiner 70, 2015)

 

Discussion

With the growth in the scope and number of Master's programmes offered by various academic institutions in sub-Saharan Africa, it is key to embed evaluation of these programmes to inform curriculum renewal. This study used a descriptive mixed-methods approach to take stock of student research submitted for examination, explore examiners' feedback and describe publication outputs. As with other regional programmes,^[15,16] students chose research topics that harmonise with key sub-Saharan priority burden of diseases, including tuberculosis, HIV and non-communicable diseases. Evidence synthesis, which provides students with a relatively holistic skill set, including clear question formulation, searching for evidence, critical appraisal, qualitative or quantitative data analysis and interpretation of results, was the popular choice of research method. Various students published impactful Cochrane^[15,16] and non-Cochrane^[17,18] systematic reviews, including qualitative evidence synthesis.^'191 Furthermore, a key finding was the high research publication rate that was enabled by the requirement to submit the research for examination as a publication-ready manuscript - promoting publication follow-through. In comparison, publication rates were lower in countries such as France (17%),'²⁰¹ Cameroon (among postgraduate students' research topics involving HIV or AIDS) (14%)^[21] and Finland (23.8%).^[22]

Our study, however, being descriptive in nature and thus prone to various biases, has only scratched the surface. It used a singular descriptive lens of the examiners and student research. We need to consider how our curriculum enables knowledge (the know), skills (the do) and attitudes (the be) of being a clinical epidemiologist or similar orator for research evidence to inform healthcare decision-making,^'231 likely through more in-depth robust curriculum evaluation models. Towards strengthening the teaching of research and student research throughput in Master's and similar programmes, reflecting on our results, we suggest that co-ordinators invest in a dedicated curriculum driven through formative assessment to drive teaching (and importantly learning by students) of research literacy. This could be in the form of a formative task throughout the programme, addressing: (i) scientific writing (such as writing a clear argument, summary of findings and presentation of research results); (ii) research question conceptualisation; and (iii) project planning, dependent on the programme and student needs.

Study limitations

Our work has several limitations, as we only described research conducted by students who graduated from the Master's programme and reviewed examiners' feedback. We did not assess the research of those who did not graduate, and did not seek input from the students, supervisors and programme faculty. Furthermore, we present a specific snapshot of the Master's programme, which may not be generalisable to other programmes with different student characteristics and programmatic offering.

Despite these limitations, considering the paucity of literature describing students' research outputs and academic programmes in the Global South, this study provides a platform to build on for assessing and enhancing capacity to initiate, conduct and report on regionally relevant research.^'24,251

 

Conclusion

Our article reflects on the scope, outcomes and student publication throughput of the research component in the MClinEpi programme at Stellenbosch University over more than a decade. Overall, the programme demonstrated a consistently high pass rate and relatively successful publication rate compared with other international programmes. Our results highlight successes to build on and some challenges and spaces for improvement in student research. Dedicated investment into formative assessments driving research competency can further strengthen research output and student throughput.

Data availability. The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.

Declaration. This retrospective descriptive study posed minimal risk, as we reviewed open-access (phase 1, student research assignments) and internal Stellenbosch University-owned data (phase 2, examiner reports). No data was collected on humans, only published/open-access or the internal Stellenbosch University secondary data. Examiner or student personal details or confidential information was not recorded or reported and thus no informed consent was required. All data were stored and cleaned using standard methods (i.e. secure, password-protected database in Excel (Microsoft, USA)). Each student assignment was given a unique ID. All personal information was removed during data extraction. Despite the research posing minimal risk open access, and internal university data, ethical clearance was sought from the Research Ethics Committee: Social, Behavioural and Education Research (ref. no. REC-2021-21829). Additionally, as we used the internal university data (i.e. examiner reports), we requested permission via the Division of Information Governance of Stellenbosch University.

Acknowledgements. We thank the administrative staff of the Division of Epidemiology and Biostatistics, Department of Global Health, Faculty of Medicine and Health Sciences, Stellenbosch University. We also thank the staff at the Centre for Evidence-based Health Care for their help in accessing students' records that could not be found on the electronic database for Master's theses on the Faculty's website.

Author contributions. MM, TY and TE designed and drafted the proposal of the study. All authors developed the methods of the study. IIK collected and analysed the data, verified by all co-authors. IIK wrote the drafts and final manuscript. All authors revised the drafts and approved the final manuscript.

Funding. None.

Conflicts of interest. MM, TY and TE are all involved in the co-ordination and teaching of the MClinEpi programme at Stellenbosch University. IIK is not part of the teaching staff on the MClinEpi programme. The views reported in this article are based on the research findings and do not represent views of the Division of Epidemiology and Biostatistics, Department of Global Health, Stellenbosch University.

 

References

1. Fletcher RH, Fletcher SW. Clinical Epidemiology: The Essentials. 4th ed. Baltimore: Lippincott Williams & Wilkinson, 2005:1-252.         [ Links ]

2. McCaul M, Tovey D, Young T, et al. Resources supporting trustworthy, rapid and equitable evidence synthesis and guideline development: Results from the COVID-19evidence network to support decision-making (COVID-END). J Clin Epidemiol 2022;151:88-95. https://doi.org/10.1016/j.jclinepi.2022.07.008        [ Links ]

3. Young T, Garner P, Clarke M, Volmink J. Clinical epidemiology in South Africa. Evidence-based health care and policy in Africa: Past, present, and future. J Clin Epidemiol 2017;83:24-30. https://doi.org/10.1016/j.jclinepi.2016.06.006        [ Links ]

4. Sana EA, Atienza MA, Mojica JAP, Abarquez LF, Fajutagana NS. Evaluation of the Master of Science in Epidemiology (Clinical Epidemiology) curriculum. Acta Medica Philippina 2009;43(4):35. https://doi.org/10.47895/amp.v43i4.7390        [ Links ]

5. Hart J, Hakim J, Kaur R, et al. Research supervisors' views of barriers and enablers for research projects undertaken by medical students; a mixed methods evaluation of a post-graduate medical degree research project program. BMC Med Educ 2022;22(1). https://doi.org/10.1186/s12909-022-03429-0        [ Links ]

6. Pillai RK. Clinical epidemiology in developing countries: Current situation and suggestions for the Indian context. Clin Epidemiol Global Health 2016;4:S1-S5. https://doi.org/10.1016/j.cegh.2016.08.007        [ Links ]

7. Young T, Naude C, Brodovcky T, Esterhuizen T. Building capacity in clinical epidemiology in Africa: Experiences from Masters programmes. BMC Med Educ 2017;17(1). https://doi.org/10.1186/s12909-017-0885-4        [ Links ]

8. Robinson L, Schulz J, Khilnani A, et al. Digital inequalities in time of pandemic: COVID-19exposure risk profiles and new forms of vulnerability. First Monday 2020;25(10). https://doi.org/10.5210/fm.v25i7.10845        [ Links ]

9. O'Cathain A, Murphy E, Nicholl J. Three techniques for integrating data in mixed methods studies. BMJ 2010;341(7783):1147-1150. https://doi.org/10.1136/bmj.c4587        [ Links ]

10. McKim CA. The value of mixed methods research: A mixed methods study. J Mix Methods Res 2017;11(2):202-222. https://doi.org/10.1177/1558689815607096        [ Links ]

11. ATLAS.ti Scientific Software Development GmbH. ATLAS.ti Web (version 22). 2021. https://atlasti.com (accessed 11 March 2025).         [ Links ]

12. Cameron R, Dwyer T, Richardson S, Ahmed E, Sukumaran A. Lessons from the field: Applying the Good Reporting of A Mixed Methods Study (GRAMMS) framework. Electr J Business Res Methods 2008;11(2):53-66.         [ Links ]

13. Onwuegbuzie AJ, Leech NL. Sampling designs in qualitative research: Making the sampling process more public. Qualitative Report 2007;12(2):238-254. https://doi.org/10.46743/2160-3715/2007.1636        [ Links ]

14. Punch KF. Introduction to Social Research: Qualitative and Qualitative Approaches. 2nd ed. Perth, Australia: The University of Western Australia, 2005.         [ Links ]

15. Rohwer C, Rohwer AC, McCaul M, Hofmeyr GJ. Hormonal contraception for women at risk of HIV infection. Cochrane Database Syst Rev 2023;2023(9). https://doi.org/10.1002/14651858.CD015701        [ Links ]

16. Mccaul M, Lourens A, Kredo T. Pre-hospital versus in-hospital thrombolysis for ST-elevation myocardial infarction. Cochrane Database Syst Rev 2014;(9):1-40. https://doi.org/10.1002/14651858.CD010191.pub2        [ Links ]

17. Lukusa LA, Ndze VN, Mbeye NM, Wiysonge CS. A systematic review and meta-analysis of the effects of educating parents on the benefits and schedules of childhood vaccinations in low and middle-income countries. Human Vaccines Immunotherapeutics 2018;14(8):1-2068. https://doi.org/10.1080/21645515.2018.1457931        [ Links ]

18. Ngah VD, Mazingisa AV, Zunza M, Wiysonge CS. A review of adherence and predictors of adherence to the CONSORT statement in the reporting of tuberculosis vaccine trials. Vaccines (Basel) 2020;8(4):1-13. https://doi.org/10.3390/vaccines8040770        [ Links ]

19. Hendricks L, Eshun-Wilson I, Rohwer A. A mega-aggregation framework synthesis of the barriers and facilitators to linkage, adherence to ART and retention in care among people living with HIV. Syst Rev 2021;10(1):1-28.https://doi.org/10.1186/s13643-021-01582-z        [ Links ]

20. Salmi LR, Gana S, Mouillet E. Publication pattern of medical theses, France, 1993 - 98. Med Educ 2008;35(1):18-21. https://doi.org/10.1111/j.1365-2923.2001.00768.x        [ Links ]

21. Munung NS, Vidal L, Ouwe-Missi-Oukem-Boyer O. Do students eventually get to publish their research findings? The case of human immunodeficiency virus/acquired immunodeficiency syndrome research in Cameroon. Ann Med Health Sci Res 2014;4(3):436. https://doi.org/10.4103/2141-9248.133474        [ Links ]

22. Nieminen P, Sipilä K, Takkinen HM, Renko M, Risteli L. Medical theses as part of the scientific training in basic medical and dental education: Experiences from Finland. BMC Med Educ 2007;7(51):1-7. https://doi.org/10.1186/1472-6929-7-51        [ Links ]

23. Barnett R. Knowing and becoming in the higher education curriculum. Stud Higher Educ 2009;34(4):429-440. https://doi.org/10.1080/03075070902771978        [ Links ]

24. 2Forland F, Rohwer AC, Klatser P, Boer K, Mayanja-Kizza H. Strengthening evidence-based healthcare in Africa. Evid Based Med 2013;18(6):204-206. https://doi.org/10.1136/be-2012-101143        [ Links ]

25. Young T, Garner P, Kredo T, Mbaugbaw L, Tharyan P, Volmink J. Cochrane and capacity building in low- and-middle-income countries: Where are we at? Cochrane Database Syst Rev 2013;(11):1-3. https://doi.org/10.1002/14651858.ED000072        [ Links ]

 

 

Correspondence:
II Kallon
kalloni@sun.ac.za

Received 13 July 2024
Accepted 9 February 2025