RESEARCH

 

Nominal group technique review of the emergency care content of the clinical skills module in the undergraduate medical programme at the University of the Free State

 

 

D T Hagemeister

BA, MPH, EMMB, FAfAllgMed, DA, Dip HIV Man (SA), DIRE; Department of Family Medicine, Faculty of Health Sciences, University of the Free State, Bloemfontein, South Africa

Correspondence

 

 

---

ABSTRACT

BACKGROUND: Handling medical emergencies is essential for medical practitioners. Medical students at the University of the Free State have an emergency care block in their third year. Nominal group technique (NGT) has been introduced for programme development, and has been used for the assessment of educational programmes
OBJECTIVES: To identify the strengths and weaknesses of the current teaching programme, as experienced by the students, and to obtain advice from lecturers on available resources and additional requirements
METHODS: A two-stage NGT was used to identify strengths and weaknesses of the programme from the 'clients' (students), and for the 'experts' (clinicians and educators) to suggest possible improvements. Two NGT sessions were conducted with students that had either recently (third-year students) or 2 years ago (fifth-year students) been exposed to the module. Students were asked to identify positive and negative aspects. Based on these sessions, two further NGTs were conducted with groups of 'experts' from the School of Medicine, asking for suggestions for improvement in the current resources, and for additional resources necessary
RESULTS: Students valued the practical skills obtained and some of the format of the teaching, but requested an increase of practical content, as well as additional tools and modes of teaching. Lecturers suggested co-ordinating outcomes to clarify basic concepts and to use additional media, but emphasised the need for human resources, teaching tools and functional clinical equipment
CONCLUSION: NGT provides a valuable tool to obtain critical suggestions from students and lecturers for improvement of the clinical teaching of emergency care

---

 

 

Medical graduates are rightfully expected to be competent in managing clinical emergencies. South African (SA) regulations that govern undergraduate medical curricula, however, are fairly quiet about the matter, and in the absence of a national 'standard' curriculum, approaches to how to convey these capabilities might vary widely between the eight SA medical schools. Internationally, bodies representing emergency medicine and emergency physicians, such as the International Federation for Emergency Medicine and the American College of Emergency Medicine, have published recommendations on the emergency content of undergraduate medical training.^[1,2]

At the University of the Free State (UFS), the MB ChB undergraduate medical programme runs over 5 years and is divided into the pre-clinical and clinical phase. At the beginning of their third year, the medical students receive seven 3-hour lecture blocks and two practical sessions (totalling 4 hours per student) during the 'clinical skills' module. These sessions form part of the students' introduction into clinical practice in phase 2, which spans semesters 4 and 5. In phase 3 (semesters 6 - 10), the students are exposed to clinical emergency conditions and their management during the rotations in specific medical disciplines. Without a department of emergency medicine at the UFS, there is currently no integrated emergency care curriculum, with only limited co-ordination between the different stakeholders.

Nominal group techniques (NGTs) were initially introduced in the 1960s as a project management tool, allowing a structured approach to the identification of 'client' needs and the development of projects in response to such demands.^[3] The NGT is regarded as a 'consensus' tool, producing prioritised outcomes to guide a project implementation,^[4,5] and has been gainfully applied in the assessment of a new undergraduate nursing sciences^[6] and a redesigned undergraduate medical programme,^[7] and in the evaluation of courses within undergraduate medical programmes.^[8,9] The NGT has been credited for being creative by containing dominant group members and allowing quieter members to express their ideas.^[3,10]

This study will hopefully contribute to the continuous critical review of current teaching practices to improve the quality of the MB ChB programme and optimise the competencies of graduates.

 

Objective

The aim of this study was to explore undergraduate medical students' perceptions of the strengths and weaknesses of the current emergency care clinical skills module, and to identify, with the help of technical experts, available or desirable solutions to address some of the challenges.

The specific objectives were:

• to identify the strengths and weaknesses of the current approach as experienced and expressed by the students who were exposed to the module

• to identify existing resources and potential new approaches based on the insights of local technical experts (academic clinicians/educationalists)

• to include the refined findings of this study in future teaching practice

• to establish a baseline for future empirical research on health professions education in emergency care in the MB ChB programme at UFS.

 

Methods

In this study, a prospective, cross-sectional, qualitative approach was applied. As part of ongoing efforts to improve undergraduate medical education at UFS, it was decided to use the NGT to assess the experiences of undergraduate students who had participated in the clinical skills module either recently (current third-year students) or 2 years ago (current fifth-year students). This assessment was complemented by a second round of two NGTs with academic clinicians/educationalists involved in the teaching, who were asked to 'respond to' or 'work with' the findings of the first round.

Approval was obtained from the Health Sciences Research Ethics Committee, UFS (ref. no. HSREC 111/2017), and permission was granted from the authorities at the university. Written informed consent was obtained from all participants. Participation in the research was voluntary, and due to the nature of the method, results or statements are not attributable to an individual, and confidentiality of the participants was maintained throughout the process.

Data collection and processing

In keeping with the processes described in the literature, four research assistants who had received prior instructions facilitated the NGT sessions. After a brief introduction and explanation of the process, a silent round of idea generation took place. The participants were subsequently asked to present one of their ideas each, which was captured on a flip chart. This was repeated until no additional ideas were presented. The ideas as written down were then discussed for clarification, and where the group felt that two ideas overlapped and should be combined, this was done. This revised list was put up for 'voting' when each participant was asked to choose the most important statement (5 points), second-most important (4 points) down to the fifth-most important statement (1 point). Ideas were ranked according to the number of votes each idea received. The NGT sessions followed the structure suggested by Gallagher et al.^[10] An overview of the process and stages is given in Table 1. In the case of a large number of participants, the group was split into two desks, where the process, as described above, took place with a second round of ranking and voting to combine the results from the two desks added.

 

 

Study population

First-round NGT - undergraduate medical students

Medical students from two different year groups at UFS who had previously done the emergency care clinical skills module constituted the study population for the first round of the NGT. This included the third-year class, who had done the module earlier in the year, and the fifth-year class, who had completed the module 2 years previously and had been exposed to most of the clinical rotations, thus having a better overview over the full clinical content of the MB ChB programme.

With the assistance of class representatives, purposive sampling within the current third- and fifth-year classes was done, with the aim of achieving good gender and ethnic representation in samples from each of the two academic years, ideally totalling 10 - 15 participants per session.

Second-round NGT - technical experts

Academic clinical staff on the joint staff establishment for the School of Medicine, together with affiliated lecturers in the clinical disciplines and university staff from the School of Medicine (e.g. programme director, departmental teaching and learning co-ordinators and academic staff of the simulation and skills unit) constituted the total study population for the technical expert group. Based on the concept of purposive sampling,^[11] a good representation of clinical disciplines and professional functions was aimed for when recruiting the total of 10 - 15 participants for the second round of the NGT process.

 

Results

Four NGT sessions were conducted in September 2017. After conducting the NGT process (idea generation, clarification, vote to rank at the individual desk), the ranked statements, as listed in Tables 2, 3, 4 and 5, were obtained, with the number of votes for each statement. The results are presented separately for the two rounds (undergraduate students and technical experts).

For referencing in the discussion section, responses are coded in the tables. The code for the students consists of the question (student question 1/2 = SQ1/SQ2), year group (third-years = 3; fifth-years = 5) and the final ranking. For example, 'SQ1.3.4' refers to student question 1, third-year group, fourth-ranked response. The code for the technical experts consists of the question (expert question 1/2 = EQ1/EQ2), NGT session (panel 1 = 1; panel 2 = 2) and the ranking. For example, 'EQ2.1.5' refers to expert question 2, first NGT session (panel 1) and fifth-ranked responses.

First-round NGTs (undergraduate students)

The two sessions with undergraduate students included 10 third-year medical students (4 men and 6 women, 2 of whom only joined for the second question) and 18 fifth-year medical students (4 men and 14 women).

Because of the number of fifth-year students (18), the NGT for this group was run at two desks concurrently, with an additional round of clarification and voting added to consolidate the results from the two desks into one.

Second-round NGTs (technical experts)

The second round of NGT sessions was held with academic clinicians and health educationalists in two separate events with 8 (panel 1) and 7 participants (panel 2), respectively.

Panel 1 comprised 1 female and 7 male participants. These included consultants and medical officers from the departments of trauma, family medicine and surgery as well as a paramedic and an information technology (IT)/technical support staff member.

Panel 2 comprised 2 female and 5 male participants. This panel included consultants and medical officers from the departments of family medicine, anaesthesiology and internal medicine as well as from the Clinical Simulation and Skills Unit.

 

Discussion

Students' NGT

The students' responses did not contain any critical comments about individual lecturers, and only one lecturer was mentioned by name, to the effect that this lecturer should present a specific topic (SQ2.3.7). Students in both years appreciated the course's practical value in enabling them to address life-threatening situations (SQ1.3.1, SQ1.3.3, SQ1.3.4, SQ1.3.6, SQ1.5.1, SQ1.5.4, SQ1.5.6 and SQ1.5.7), which is in keeping with international literature.^[12,13] As the module is taught at the beginning of, and as an introduction to, the clinical years in the MB ChB programme, this is to be expected, but also shows that the module met the students' expectations in this regard. The way the content was delivered also enjoyed positive feedback in some aspects (SQ1.3.2, SQ1.3.5, SQ1.5.2, SQ1.5.4 and SQ1.5.5).

When it came to students' suggestions for improvements (question 2), the 'practical theme' again dominated, with clear requests to increase the practical parts of the learning experience (SQ2.3.1, SQ2.3.2, SQ2.3.4, SQ2.5.1, SQ2.5.4 and SQ2.5.7). In addition to this, the students requested changes to the resources ('module guide'/'case studies' - SQ2.3.3; and SQ2.5.3, 'videos'/'real world' - SQ2.3.5 and SQ2.5.5), hinting at the use of blended learning/flipped classroom techniques.^[14,15] Based on their greater clinical exposure, the fifth-year students also requested additions to the content (specific clinical emergencies - SQ2.5.2; specific scenarios - SQ2.5.6; and dealing with family members and staff - SQ2.5.9), while the third-year students were more concerned with administrative issues ('mark' SQ2.3.6) and the role of a specific esteemed lecturer ('Prof. X' - SQ2.3.7).

Further comparing the third- and fifth-year students, the latter displayed a more differentiated appreciation of technical aspects ('manikins' - SQ1.5.1; 'scenarios' - SQ1.5.2; 'fixing mistakes'/'immediate remediation' - SQ1.5.4; 'real life stories' - SQ1.5.6), probably owing to their greater exposure to clinical teaching by the time of the NGT session. The fifth-year students also addressed the different professions/ranks in the health system in a more differentiated way ('EMS personnel' - SQ1.5.1; 'consultants' - SQ1.5.4), again probably owing to their greater exposure to the system. Comments on the timing of the module about the other clinical teaching are only possible in retrospect (SQ1.5.5).

Technical experts' NGTs

Clinicians and educationalists put a strong emphasis on the clarification of outcomes and concepts in the module (EQ1.1.1, EQ1.2.1, EQ1.2.3), suggesting a proper 'scaffolding' of the content in the Vygotskian sense, rather than a mere accumulation of clinical conditions.^[1,16]

With regard to the mode of delivery of the content, emphasis was again placed on the importance of practical aspects (EQ1.1.2) and the use of 'stimulating' methods, including simulation exercises (EQ2.1.2, EQ1.1.4) as well as online resources and videos (EQ1.1.3, EQ1.2.8). Continuous evaluation of the students and the inclusion of mock examinations would also aid the learning process (EQ1.2.4), mirroring the emphasis on 'assessment drives learning' in the literature.^[17,18]

Standardisation of the teaching materials and the management approaches was suggested to improve the consistency of the teaching (EQ1.1.5, EQ1.2.5). The content should be integrated with and linked to modules in earlier and later parts of the programme (EQ1.1.5, EQ1.2.6) and this was mentioned under both questions 1 and 2 (also EQ2.2.5).

When considering additional resources, the top priority was the availability of well-trained facilitators, or in other words, human resource management and development issues (EQ2.1.1, EQ2.2.1, EQ2.2.6), including IT training for the 'older folks' (EQ2.1.6). The availability of equipment for teaching, simulators and videos was the second major theme (EQ2.1.2, EQ2.2.4), including the availability of appropriate clinical equipment at the actual clinical service delivery sites (EQ2.2.8).

Additional ideas that came up were the opportunities offered by the changed language policy of the university, which potentially frees up resources (EQ2.1.5), the utilisation of other emergency care personnel such as the emergency medical services (EMS) and their training facilities (EQ2.1.3), and the creation of a question bank of multiple-choice questions for formative and summative assessment (EQ2.2.7). Lastly, the general need for additional financial resources was expressed (EQ2.2.3).

Implementation of ideas/suggestions

In the next run of the module after the NGTs, additional practical CPR training sessions and mock examinations have been included as suggested during the research, while much more work remains to be done regarding a general overhaul of the content and structure of the module, possibly in connection with a major review of the undergraduate curriculum.

Weaknesses and possible confounders

Firstly, it needs to be acknowledged that the relatively small sample from both student groups might not be regarded as fully representative for the respective years. An additional quantitative, questionnaire-based vote on the finding by a larger sample could have added reliability through 'triangulation'.^[7] Given the 2 years between the exposure to the module of the two year groups, staff attrition and reworking of course content might have introduced differences between the two 'runs' of the same module.

On the side of the 'technical experts', the composition of the group and the absence of some disciplines (due to lack of response from certain departments) might have biased the outcomes.

The results of the two sessions in the second round of NGTs, with the technical experts, have not been consolidated into a single 'rank order' as suggested by some authors.^[19,20]

 

Conclusion

Through the use of a two-staged NGT, it was possible to gain valuable feedback from undergraduate students who had experienced the teaching in the module that was under review. Both the positive comments and suggestions for improvement as expressed during this process led to fruitful discussions with the team of academic clinicians and educators who were currently involved in the programme.

As also reported in other studies, the use of the NGT provides concise and 'easy-to-apply' results on the needs of the 'users' (students) and the possible approaches from the 'experts'. By involving both 'sides' in this two-staged NGT, the experts respond directly to the identified challenges. Further evaluation will be necessary to measure the actual impact of this research, including the implementation of the findings.

Several challenges identified by the students, and some of the suggested solutions, are beyond the scope of the current research, which was focused narrowly on the emergency care teaching occurring in the third year. A more comprehensive approach to review the overall framework of emergency care teaching in the undergraduate programme is currently projected to cover these aspects.

Declaration. None.

Acknowledgements. The author gratefully acknowledges Prof. Yvonne Botma, Sr Rynsie Hattingh, Mrs Nokuthula Tlalajoe and Dr Joleen Cairncross for their help in conducting the nominal group technique sessions. Ms Theanette Mulder is much appreciated for her support in editing the manuscript.

Author contributions. Sole author.

Funding. None.

Conflicts of interest. None.

 

References

1. Hobgood C, Anantharaman V, Bandiera G, et al. International Federation for Emergency Medicine model curriculum for medical student education in emergency medicine. Emerg Med Australas 2009;21(5):367-372. https://doi.org/10.1111/j.1742-6723.2009.01213.x        [ Links ]

2. American College of Emergency Physicians. Guidelines for undergraduate education in emergency medicine. Irving: ACEP, 2015. https://www.acep.org/patient-care/policy-statements/guidelines-for-undergraduate-education-in-emergency-medicine/#sm.00001ogy4kwqeodpat4wmro3ufk77 (accessed 18 September 2018).         [ Links ]

3. Delbecq AL, Van de Ven AH. A group process model for problem identification and program planning. J Appl Behav Sci 1971;7(4):466-492. https://doi.org/10.1177/002188637100700404        [ Links ]

4. Allen J, Dyas J, Jones M. Building consensus in health care: A guide to using the nominal group technique. Br J Community Nurs 2004;9(3):110-114. https://doi.org/10.12968/bjcn.2004.9.3.12432        [ Links ]

5. Jones J, Hunter D. Consensus methods for medical and health services research. BMJ 1995;311(7001):376-380. https://doi.org/10.1136/bmj.311.7001.376        [ Links ]

6. Chapple M, Murphy R. The nominal group technique: Extending the evaluation of students' teaching and learning experiences. Assess Eval High Educ 1996;21(2):147-160. https://doi.org/10.1080/0260293960210204        [ Links ]

7. Lloyd-Jones G, Fowell S, Bligh JG. The use of the nominal group technique as an evaluative tool in medical undergraduate education. Med Educ 1999;33(1):8-13. https://doi.org/10.1046/j.1365-2923.1999.00288.x        [ Links ]

8. Dobbie A, Rhodes M, Tysinger JW, Freeman J. Using a modified nominal group technique as a curriculum evaluation tool. Fam Med 2004;36(6):402-406.         [ Links ]

9. Lancaster T, Hart R, Gardner S. Literature and medicine: Evaluating a special study module using the nominal group technique. Med Educ 2002;36(11):1071-1076. https://doi.org/10.1046/j.1365-2923.2002.01325.x        [ Links ]

10. Gallagher M, Hares T, Spencer J, Bradshaw C, Webb I. The nominal group technique: A research tool for general practice? Fam Pract 1993;10(1):76-81. https://doi.org/10.1093/fampra/10.L76        [ Links ]

11. Henning E, Van Rensburg W, Smit B. Finding your Way in Qualitative Research. Pretoria: Van Schaik, 2004.         [ Links ]

12. Ten Eyck RP, Tews M, Ballester JM. Improved medical student satisfaction and test performance with a simulation-based emergency medicine curriculum: A randomised controlled trial. Ann Emerg Med 2009;54(5):684-691. https://doi.org/10.1016/j.annemergmed.2009.03.025        [ Links ]

13. Morris MC, Conroy P. Development of a simulation-based sub-module in undergraduate medical education. Ir J Med Sci 2019;(epub ahead of print). https://doi.org/10.1007/s11845-019-02050-3        [ Links ]

14. Parandavar N, Rezaee R, Mosallanejad L, Mosallanejad Z. Designing a blended training program and its effects on clinical practice and clinical reasoning in midwifery students. J Educ Health Promot 2019;8:131-131. https://doi.org/10.4103/jehp.j ehp_22_18        [ Links ]

15. Nakanishi T, Goto T, Kobuchi T, Kimura T, Hayashi H, Tokuda Y. The effects of flipped learning for bystander cardiopulmonary resuscitation on undergraduate medical students. Int J Med Educ 2017;8:430-436. https://doi.org/10.5116/ijme.5a2b.ae56        [ Links ]

16. Tews MC, Hamilton GC. Integrating emergency medicine principles and experience throughout the medical school curriculum: Why and how. Acad Emerg Med 2011;18(10):1072-1080. https://doi.org/10.1111/j.1553-2712.2011.01168.x        [ Links ]

17. Phillips AW, Smith SG, Straus CM. Driving deeper learning by assessment: An adaptation of the Revised Bloom's Taxonomy for medical imaging in gross anatomy. Acad Radiol 2013;20(6):784-789. https://doi.org/10.1016/j.acra.2013.02.001        [ Links ]

18. Keeley MG, Gusic ME, Morgan HK, Aagaard EM, Santen SA. Moving toward summative competency assessment to individualise the postclerkship phase. Acad Med 2019;94(12):1858-1864. https://doi.org/10.1097/ACM.0000000000002830        [ Links ]

19. McMillan SS, Kelly F, Sav A, et al. Using the nominal group technique: How to analyse across multiple groups. Health Serv Outcomes Res Methodol 2014;14(3):92-108. https://doi.org/10.1007/s10742-014-0121-1        [ Links ]

20. Van Breda AD. Steps to analysing multiple-group NGT data. Soc Work Pract Res 2005;17(1):1-14.         [ Links ]

 

 

Correspondence:
D Hagemeister
hagemeisterdt@ufs.ac.za

Accepted 17 August 2020