SciELO - Scientific Electronic Library Online

 
vol.57 número4Incidence and histological features of colorectal cancer in the Northern Cape province, South Africa índice de autoresíndice de assuntospesquisa de artigos
Home Pagelista alfabética de periódicos  

Serviços Personalizados

Artigo

Indicadores

Links relacionados

  • Em processo de indexaçãoCitado por Google
  • Em processo de indexaçãoSimilares em Google

Compartilhar


South African Journal of Surgery

versão On-line ISSN 2078-5151
versão impressa ISSN 0038-2361

S. Afr. j. surg. vol.57 no.4 Cape Town Dez. 2019

 

EDITORIAL

 

College of Surgeons general surgery final examination

 

 

Z Koto

President of the College of Surgeons

Correspondence

 

 

The College of Surgeons of South Africa has a proud history and reputation of graduating surgeons of the highest quality. The aim of the exit examination is to assess and ensure that the graduand has achieved and fulfils the required standards and necessary competencies, the science and theory of general surgery, communication and technical skills, surgical decision-making, and the ability to critically appraise new knowledge so they can apply it to their practice.

The examination process during the FCS surgical training programme comprises three summative examinations, the primary examination which tests the basics of surgical pathology, anatomy and physiology, the intermediate examination which tests the principles of surgery and the final exit examination which tests the surgical competencies alluded to earlier. The article in this issue by Khan et al.1 evaluates the outcomes of the FCS final examination over a ten-year period and makes pertinent observations about the candidates' performance. Their interpretation of this analysis reinforce my views that essay or long-menu type questions are subjective and in the FCS final examination are often open-ended and challenging for the candidates to write sufficient information to achieve a pass mark in the allotted time. It must be said that significant effort has been made to improve these questions by making them into a multi-part format. This was an attempt to improve fairness, remove perceptions of a "trap door" and cover as wide a spectrum of general surgery as possible. Kahn and her co-authors found a median mark of 55% in Paper I and of 50% in Paper II. This is a relatively low mark which emphasises the challenges the candidates face in the written papers. The authors also found that more than 30% of the candidates who did not pass the written component of the examination, but were invited to the clinical component of the examination, eventually passed. This again raises questions about reliability of the long menu or essay questions for which it is difficult to set specific standards. Khan et al. conclude that the success in the written paper is a relatively good predictor of the overall success of the FCS final candidates. However as stated in the article, those who did poorly in the written examination and still passed overall, call into question the validity of the examination. Some would argue that, in the light of this, that it is not unreasonable to invite all candidates to the clinical component of the written examination. There may be some merit to this argument, however the critical issue is to improve our assessment processes to a format that is more valid and reliable.

Validity is the ability of the examination to test what it intends to test while reliability refers to whether the examination reliably tests what the student knows. The valid examination must match the course objectives and instructional goals. Important topics must be weighed more heavily than others and therefore the allotted time must reflect this. Many educationalists contend that the multiple choice question (MCQ) single best answer (SBA) format performs best in terms of validity and reliability.2 There should be blue-printing which ensures that all aspects of the syllabus are covered. In addition there should be standard setting to determine the pass mark beforehand and all questions should be standardised. Since 2015, the end of the study period in the article, a number of changes have been implemented by the Council of the College of Surgeons of South Africa. These include the introduction of the MCQ SBA questions in the next semester final exit examination. It should be remembered that this is only one component of the FCS final examination, an exit examination that necessarily holds a candidate to a standard for independent practice as a surgeon. The process of examination is continually evolving as the Council of College of Surgeons of South Africa is constantly striving to make assessment methods more valid and reliable. To that end, the MCQ SBA assessment method will be introduced in the first semester of 2020. Essay type questions, though less reliable, test the narrative skills of the candidate and short focused questions requiring written responses will be retained for the next examination.3 The future intention is to have the written examination only in the MCQ SBA format as a suitable bank of questions is developed. It is important that there is an ongoing evaluation of the examination results not only of the new MCQ SBA format but of all components of the exit examination to make the process more objective, reliable and valid.

An aspect that has not been assessed adequately are the technical aspects of surgery which are critical to the practice of the discipline. The concept of work based assessment (WBA) is slowly gaining traction. A reliable contemporaneously kept electronic consultant validated logbook is an important element of the WBA assessment of the technical skills of the candidate, i.e. can this surgeon operate safely and competently with acceptable outcomes. In short the way to go is the MCQ SBA format to assess knowledge, its synthesis and application and WBA to assess surgical skills and decision making.

That said, the low pass rate in the recent FCS final exit examination has raised a number of pertinent issues.These issues can be broadly categorised as candidate, training platform and examination related. It is clear that the reasons for the poor examination results are multifactorial and warrant an investigation.

Disclosures

The author has no conflict of interest or any financial disclosures to make.

 

REFERENCES

1. Khan M, Khan D, Klopper J. An audit of the outcomes of the Fellow of College Surgeons final Examinations; SAJS 2019;57(2).         [ Links ]

2. Ceruti B, Stolla F, Escher M, et al. Comparison of long-menu and single-best answer multiple questions in computer based summative assessment: A randomized control trial. BMC Med Education 19, (2019), 219.         [ Links ]

3. Botting N. Narrative as a tool for the assessment of linguistic and pragmatic impairment. SAGE publication. 2002.         [ Links ]

 

 

Correspondence:
Zach Koto
zachkoto@gmail.com

Creative Commons License Todo o conteúdo deste periódico, exceto onde está identificado, está licenciado sob uma Licença Creative Commons