EDITORIAL

Work-based assessment: A critical element of specialist medical training

In the recent Transactions, the Colleges of Medicine of South Africa (CMSA) states that work-based assessment (WBA) should be an essential part of an assessment system, alongside the national professional examination.^[1] One of the main reasons for highlighting this topic is the fact that assessment enables our profession to demonstrate our accountability, and our commitment to reflecting professional practice standards, sustaining competence, improving performance and engaging in continuous quality improvement.^[1,2]

To this effect, most examiners would like WBA to be mandatory, because it would be a means of informing the supervisor's report for MMed and Fellowship degrees, as well as an opportunity to oblige the registrars to receive feedback, and to reflect and develop. A registrar would have to demonstrate important skills such as professionalism and decision-making under compulsory supervision in real life. In some of our training platforms this valuable tool to promote active learning is already in place, and yet in some departments or institutions it is absent. Perhaps this inhomogeneous practice could explain the finger-pointing game about the examination standards and failure rate. Compulsory WBA could be used to identify, wherever possible, areas of practice where further learning should be focused. WBA should be able to support and contribute to summative assessment strategies that define successful achievement of curricular objectives, competence to entry into practice and continued competence in practice.^[2]

However, as stated in the CMSA message, colleagues have raised serious concerns about the utility of WBA for high-stakes summative assessment purposes, indicating that assessment tasks in the real world are unpredictable and inherently unstandardised, and therefore will not be equivalent across different hospitals and universities. This could pose a serious threat to the reliability and validity of assessment. There are also serious concerns about the subjectivity of assessments.^[1] To address these challenges, the modernised approach to WBA makes a distinction between formative assessment (i.e. assessment for learning) and summative assessment (i.e. assessment of learning), and includes both measures. Furthermore, recent work has shown that the combination of several WBA tools, such as the mini-clinical-evaluation exercise, case-based discussion, objective structured assessments of technical skills and multisource feedback, in a portfolio, is a feasible and reliable method for high-stakes exams.^[3,4] Of note is the fact that the new categorisation of WBA tools used by the Royal College of Obstetricians and Gynaecologists divides the WBA into formative encounters, referred to as 'supervised learning events, and summative encounters, referred to as 'assessments of performance' (Fig. 1).^[1,3] With this approach, patients, registrars and examiners are all beneficiaries, as the formative WBA focuses on the provision of structured, mandatory feedback, while encouraging reflective practice by the registrar, and the summative WBA constitutes assessments of observed performance, which allow registrars to demonstrate competence. The summative encounters of the WBA, referred to as 'assessments of performance', will consider a model that consists of both local and external examiners, which will improve on validity, and therefore allow inclusion in the high-stakes exam. Therefore, the model could help to remove potential bias, as the assessments will be conducted in consultation with the CMSA and SA Committee of Medical Deans (SACOMD). In part, this will also address the fact that universities have to fulfil the mandate to conduct the assessment for the MMed, in addition to the compulsory high-stakes national professional examination (singleexit examination) that is run by the CMSA. Furthermore, since the formative encounters of WBA aim to identify the registrar's areas of strength and development, and the summative encounters of WBA aim to determine fitness to progress, this will help to address the issues of standards and failure rates. Of importance is that WBA will be able to assess the 'does' level in Millar's prism of clinical competence, and therefore demonstrate mastery.^[5,6] Fortunately, the modified approach still emboldening the philosophy which underpins WBA is the assessment of several domains by multiple assessors over a period of time, with feedback built into each encounter.^[7] Excitingly, WBA is an opportunity to develop skills of self-awareness for registrars, as well as a tool to identify and monitor struggling registrars.

Clearly the development of such an assessment system must consider and overcome a number of implementation barriers, including assessment fatigue.^[8] Therefore, in order for WBA to be successful, the CMSA should ensure that WBA is relevant to professional practice, and provides feedback to promote personal or collective reflection on competence or performance. Additionally, the proposed meaningful partnership between the CMSA and SACOMD should make participation in WBA a mandatory programme requirement that is linked to progression through training, as well as an accreditation standard for training platforms.^[2]

WBA tools, when combined in a registrar's e-portfolio to create a central and integrating concept whose purpose is both formative (educational) and summative (high stakes), are a means of planning and implementing lifelong learning to enhance performance, improve quality of care and enhance the effectiveness of our health systems.^[1-3]

M M Sathekge

The Colleges of Medicine of South Africa, Cape Town, South Africa; Department of Nuclear Medicine, Faculty of Health Sciences, University of Pretoria, South Africa; and Steve Biko Academic Hospital, Pretoria, South Africa mike.sathekge@up.ac.za

References

1. Sathekge M. Time to review the contribution of work-based assessment in our high-stakes exams. Transactions 2017;61(1):4-5. https://www.cmsa.co.za/view_document_list.aspx?Keyword=Transactions (accessed 5 June 2017).         [ Links ]

2. Watters DA, Campbell G, Crebbin W, Hickey K, Stokes ML. Work-based assessment: A practical guide. Melbourne: Tripartite Alliance (RACP, RACS, RACPSC), 2014. https://www.surgeons.org/media/20786937/bkt_tripartite_wba_march_7_2_.pdf (accessed 4 May 2017).         [ Links ]

3. Parry-Smith W, Mahmud A, Landau A, Hayes K. Workplace-based assessment: A new approach to existing tools. Obstetrician Gynaecol 2014;16:281-285. https://doi.org/10.1111/tog.12133        [ Links ]

4. Govaerts M, van der Vleuten C. Validity in the work-based assessment: Expanding our horizons. Med Educ 2013;47(12):1164-1174. https://doi.org/10.1111/medu.12289        [ Links ]

5. Miller GE. The assessment of clinical skills/competence/performance. Acad Med 1990;65(Suppl 9):S63-S67.         [ Links ]

6. Norcini J, Burch V Workplace-based assessment as an educational tool: AMEE Guide No. 31. Med Teach 200739(9):855-871. https://doi.org/10.1080/01421590701775453        [ Links ]

7. Nair BR, Hensley MJ, Parvathy MS, et al. A systematic approach to workplace-based assessment for international medical graduates. Med J Aust 2012;196(6):399-402. https://doi.org/10.5694/mja11.10709        [ Links ]

8. Van der Vleuten CP, Schuwirth LW, Scheele F, et al. The assessment of professional competence: building blocks for theory development. Best Pract Res Clin Obstet Gynaecol 2010;24(6):703-719. https://doi.org/10.1016/j.bpobgyn.2010.04.001        [ Links ]