On-line version ISSN 2078-5151
S. Afr. j. surg. vol.52 n.4 Cape Town Nov. 2014
D KrugerI; M G VellerII
IBSc, PhD; Department of Surgery, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa
IIMB BCh, MMed (Surg), FCS (SA); Department of Surgery, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa
BACKGROUND: The logged experience of specialist general surgical trainees has made it possible to analyse their surgical procedural exposure
OBJECTIVE: To evaluate the exposure to key surgical procedures of South African (SA) trainees in general surgery from logbooks submitted to the Colleges of Medicine of South Africa (CMSA
METHODS: Logbooks submitted and meeting the minimum requirements for the six final examinations for the fellowship of the College of Surgeons of the CMSA between August 2010 and March 2013 were selected. Consolidated surgical procedural experience was analysed according to procedural category, extent of supervision, procedure complexity and university at which the trainee performed the procedures
RESULTS: The 95 logbooks entered into the study recorded 144 499 procedures, 60.6% of which were unsupervised, 18.5% supervised and 20.9% assisting another surgeon. Major and minor procedures made up 40.4% and 54.6%, respectively, with the remaining 5% categorised as 'other'. A breakdown of procedural exposure per category, including the main contributing or key procedure for each category, is presented
CONCLUSION: Large numbers of procedures are logged by trainees during their surgical training. Inter-university and trainee key procedural exposure in SA differ to a small degree but are striking in some categories. Exposure to key procedures is insufficient in some categories. We are currently unable to assess the quality of training and quality of surgical skills from such logbooks. A standardised electronic logbook will facilitate future analyses of trainee procedural exposure, but other tools will be required to assess the quality of surgical skills training
Logbooks that document surgical procedures to which a specialist general surgical trainee has been exposed have been a prerequisite to enter the final examination of the Fellowship of the College of Surgeons of the Colleges of Medicine of South Africa (CMSA) since 2002. The requirement that this exposure be documented in a consolidated and standardised format, introduced in 2008, has made it possible to analyse the exposure of specialist general surgical trainees in South Africa (SA) to surgical procedures since then. The experience is logged using a standardised nomenclature of279 procedures divided into 24 anatomical and procedural categories. In addition, the extent of supervision in the performance of the logged experience is specified as unsupervised (US), supervised (S), or assisting (A) another surgeon.
The aim of this study was to evaluate the exposure to key surgical procedures of SA specialist general surgical trainees retrospectively from logbooks submitted to the CMSA during the period August 2010 - March 2013.
Logbooks submitted for the six final examinations for the Fellowship of the College of Surgeons of the CMSA between August 2010 and March 2013 and that met the minimum requirements of the College of Surgeons were initially selected for this study. Logbooks submitted by trainees from universities with fewer than five candidates during the study period were then excluded.
Of the 143 logbooks submitted to the College of Surgeons during the study period, 95 met the requirements for entry into the study.
This included logbooks from the following five faculties of health sciences (of eight in SA): University of KwaZulu-Natal (n=28), University of Cape Town (n=26), University of the Witwatersrand (n=24), Stellenbosch University (n=10) and University of Pretoria (n=7). These institutions were randomly named U1 to U5.
The consolidated surgical procedural experience of the remaining logbooks was entered into an Excel template, and pivot tables were used for analyses. For inter-university comparisons, non-parametric Kruskal-Wallis tests were done using STATISTICA 12, and a p-value of <0.05 was considered significant. In order to ensure that every available procedure was used in the evaluation, procedures not categorised into the 279 named procedures included in the standardised nomenclature were entered as 'other' under the relevant category. Analysis was then performed according to procedural category, extent of supervision (US, S or A), complexity of the procedure (according to the BUPA classification into major and minor), and the university at which the trainee performed the procedures.
The 95 logbooks from the five faculties recorded 144 499 procedures, of which 60.6% were US (n = 87 560), 18.5% S (n=26 776) and 20.9% A (n=30 163). The extent of supervision was similar across all five universities.
The experience included 279 named procedures. Major and minor procedures made up 40.4% and 54.6%, respectively (the remaining 5% fell into the category 'other' and could therefore not be classified). Here too the experience was similar at all the universities. The number of US major procedures performed (49% of the major procedures) compared with the number of US minor procedures performed (70% of the minor procedures) was statistically significant (p=0.003).
The trainees from U5 logged 66% of all trauma resuscitations (in the category 'intensive care') and 51% of the endoscopy procedures (in the category by that name). Analysis of inter-university experience is therefore performed using both the total number of procedures logged and the number of procedures logged after exclusion of the above two procedures (Fig. 1). Table 1 shows a breakdown of procedural exposure per category for all trainees. In addition, the data for the main contributing or key procedure for each category are presented. Table 2 shows an inter-university comparison of procedural exposure per category. The overall surgical procedural exposure at U5 was significantly higher than at the other universities, with the exception of U3 (Fig. 1 and Table 2); however, none of the other universities showed significant overall inter-university differences. Although mean surgical procedural exposure differs significantly between universities for most of the category entries shown (see p-values in Table 2), multiple comparative analyses show that in only a few cases is this difference accounted for by more than one university mean.
Surgical trainee logbooks submitted to the recent final fellowship examination of the College of Surgeons (FCS) final examinations now make it possible to analyse surgical procedural experience in SA. This study demonstrates that a large number of procedures are logged by trainees during their surgical training (1 200 per trainee, excluding trauma resuscitations and endoscopic procedures). In this study, this experience was gained on average over 4.3 years of training (range 2.8 - 7.4), with no significant difference in training periods between universities. Approximately 40% of the experience is gained on major procedures. The majority of all procedures performed by trainees are unsupervised (59.5%). Of the remaining 40.5%, 17% are performed by the trainee under supervision and a further 23.5% as an assistant. The current format of the logbooks does not allow us to provide any meaningful explanation on the level of supervision. The relatively high proportion of emergency procedures and the complexity of major elective surgery may be factors explaining the level of supervision.
Overall exposure to surgical procedures differs to a small degree between the universities. In most procedural categories, experience between universities is also consistent. Larger procedural exposure differences between the universities are striking with regard to exposure to endoscopy, trauma resuscitations, laparotomy, circumcisions and varicose vein procedures. While the number of surgical procedures appears to be adequate in some of the procedural categories, it is not in others. In particular, numbers seem insufficient for the following procedure categories: vascular (especially venous), major rectum and anus, burns, oesophagus and liver.
Using the current format of the submitted logbooks, it is not possible to assess the surgical skill of these trainees. This has been well documented by others.[2-5]
Limitations of this study include that the accuracy of the results from the overall consolidation sheets are only as accurate as the logging done by the trainees themselves. Also, no distinction can be made between emergency and elective procedures, or between procedural exposure during the junior v. the senior rotation of the training period. The data have also not been normalised for the training time of each trainee.
The study shows that changes need to be made to the current system of evaluating general surgical trainee skills in SA. Firstly, the method of logging data relies on the trainee supplying the data in a non-uniform manner, and the evaluation of data is manual so analysis is tedious.
The current system also does not verify that formative assessment has occurred, and verification of the data outside of such a process is not possible. What should be considered is specific procedure-based in-course assessments as are used by other colleges, such as the Royal Colleges of Surgeons in the UK, which allows them to adequately assess surgical skills of a trainee to perform key procedures. If the data currently logged in the College of Surgeons of the CMSA logbook will still be used, the recommendations made by members of the Royal Australasian College of Surgeons for surgical trainee logbooks should also be considered. They emphasise the importance of reporting procedural outcome, not only to aid self-learning from individual cases, but also to teach trainees the lifetime practice of effective surgical audit.
In conclusion, inter-university and trainee key procedural exposure in SA, even when the numbers seem adequate, vary in many categories. Confidence intervals for the numbers of procedures could guide in establishing minimum criteria for key procedures during surgical training. Limitations of the surgical trainee logbooks in assessing the quality of training and the quality of surgical skills are also evident.
Future analyses of procedural exposure of trainees in SA will be facilitated with the use of a standardised electronic logbook. Such a logbook should also allow quality of surgical skills training to be assessed, possibly by documenting regular formative procedure-based assessments that would be a requirement for progression through the various stages of general surgical training.
1. Colleges of Medicine of South Africa. Fellowship of the College of Surgeons of South Africa: FCS (SA), 2013. http://www.collegemedsa.ac.za/view_exam.aspx?examid=46 (accessed 3 December 2013). [ Links ]
2. Lonergan PE, Mulsow J, Tanner WA, Traynor O, Tierney S. Analysing the operative experience of basic surgical trainees in Ireland using a web-based logbook. BMC Med Educ 2011;11:70. [http://dx.doi.org/10.1186/1472-6920-11-70] [ Links ]
3. Merry C, Goodall-Wilson D, Guest G, Papas C, Selvidge J, Watters DA. The surgical trainee log we need: Minimum ofwork, maximum of output. Aust N Z J Surg 2006;76(3):185-189. [ Links ]
4. Watters DA, Green AJ, van Rij A. Requirements for trainee logbooks. Aust N Z J Surg 2006;76(3):181-184. [ Links ]
5. Brouwer R, Kiroff G. Computer-based logbook for surgical registrars. Aust N Z J Surg 2002;72(1):57-61. [ Links ]
6. Beard J, Rowley D, Bussey M, Pitts D. Workplace-based assessment: Assessing technical skill throughout the continuum of surgical training. Aust N Z J Surg 2009;79(3):148-153. [http://dx.doi.org/10.1111/j.1445-2197.2008.04832.x] [ Links ]