SciELO - Scientific Electronic Library Online

 
vol.111 issue2The Competition Commission Health Market Inquiry Report: An overview and key imperativesParents refusing blood transfusions for their children solely on religious grounds: Who must apply for the court order? author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Article

Indicators

Related links

  • On index processCited by Google
  • On index processSimilars in Google

Share


SAMJ: South African Medical Journal

On-line version ISSN 2078-5135
Print version ISSN 0256-9574

SAMJ, S. Afr. med. j. vol.111 n.2 Pretoria Feb. 2021

http://dx.doi.org/10.7196/samj.2021.v111i2.15310 

IN PRACTICE
HEALTHCARE DELIVERY

 

A quality improvement programme in radiotherapy using workflow audits

 

 

R P AbrattI; M RautenbachII; Y GovenderIII; I OelofseIV

IFC Rad One (SA), MMed (Rad T); Head of Clinical Governance, Icon Oncology, South Africa; and Emeritus Professor of Radiation Oncology, Faculty of Health Sciences, University of Cape Town, South Africa
IIBRad (Diagn), NHD Radiotherapy; Radiotherapy Area Manager, Icon Oncology, South Africa
IIIBTech (Radiography), MTech (Radiotherapy); National Radiotherapy Manager, Icon Oncology, South Africa
IVBSc Hons; National Medical Physics Manager, Icon Oncology, South Africa

Correspondence

 

 


ABSTRACT

BACKGROUND: Evaluation of the quality of healthcare depends on measures of structures, processes and outcomes. Progress in recording data allows for better measures of processes, such as the completeness of clinical data, the performance of professional tasks and the use of checklists.
OBJECTIVES: To report the results of a radiotherapy (RT) workflow audit and a subsequent online survey of user experience.
METHODS: The RT workflow audit was developed in 2016 and has been undertaken twice a year at 28 facilities or units, with a total of 32 linear accelerators. Electronic patient folders were reviewed to assess the documentation of 90 task items, of which 64 were scored. The auditor came from another facility. The online survey took place in July 2020. It contained questions on the audit's process, professional value and future use. Invitations were sent by email to the 151 radiotherapist staff at the 28 units where the audit had been implemented. Responses were anonymous.
RESULTS: For the RT workflow audit, scores improved from 60% in some units in 2016 to >90% in all units for at least 2 years since 2018. The number of responders to the online survey was 58, giving a responder rate of 38%. The margin of error of the results was 10%. The audit's task items were considered appropriate by 77% of responders, and feedback was reported by 78% of them. The audit was considered very or extremely valuable to their unit's service delivery by 58% of responders. Changes in the unit as a result of the audit were reported by 77% of responders. The audit was very useful or extremely useful to 75% of responders in maintaining personal professional standards. The proportion of responders who were very or extremely supportive of continuing with the audit was 77%. The comments in the online survey will be helpful for ongoing review of the RT workflow audit.
CONCLUSIONS: The RT workflow audit extends the scope of accreditation audits by including measures of processes. Users of the audit evaluate its processes favourably and report that it has value both in their unit's clinical service and for their personal professional standards. The audit is effective in developing quality improvement programmes.


 

 

Evaluation of the quality of healthcare depends on information from three categories, namely facility structures, operation of processes, and outcomes of the health services.[1,2] Quality assurance programmes and audits for the accreditation of facilities in medicine and oncology largely evaluate the first category, i.e. facility structures and the availability of policies.[3-8]

There has been progress in recording clinical data through commercial electronic information systems, e.g. the Mosaiq and Aria systems in oncology. These allow for the improved measurement of processes, such as the completeness of clinical data, the performance of professional tasks and the use of checklists as part of routine care. The inclusion of these in audits will help the development of quality improvement programmes by upholding high standards and reducing technical and clinical errors.

Radiotherapists undertake the administration of radiotherapy (RT) and maintain electronic records of treatments. They form an interdependent multidisciplinary team together with medical physicists and radiation oncologists for the safe and effective administration of RT to patients.

In 2016, an RT workflow audit was developed within an accountable care organisation, Icon Oncology, and implemented in their 28 units, with a total of 32 linear accelerators. In July 2020, an online survey of the user experience of the audit was held.

The RT workflow audit and the subsequent online survey findings are described and evaluated in this report, and their relevance to quality improvement programmes in multidisciplinary teams is discussed.

 

Methods

The RT workflow audit

The RT workflow audit consisted of a review of patient records that were stored in a commercial information system (Mosaiq; Elekta, Sweden). The RT audit measured documented completion of a list of task items that were developed by a radiotherapist task team in 2016. They included clinical data, the completion of professional tasks, and the use of checklists to be used in routine practice. The total number of task items on the audit document was 90, and the number that were scored was 64. The task items not scored were aspirational and not part of routine care.

The audit was conducted across users of 32 linear accelerators in 28 departments. Four patient records or folders were chosen randomly from each department and reviewed. The review of each folder took ~1 hour. The auditor was a radiotherapist from another department. All the individual audits were then submitted to the task team chairperson, who assessed them to make sure that the audit was done correctly and fairly. The results of the audit were scored by dividing the proportion of list of task items that were documented as completed in the folders. Feedback was subsequently given to the staff in the audited unit. The audits were completed twice a year.

The online survey

An online survey of the radiotherapist user experience was undertaken in July 2020 using a commercial service (SurveyMonkey, USA). The survey contained questions on the RT workflow audit's processes (Table 1), its professional value (Table 2) and future RT workflow audits (Table 3). In addition, there was a question inviting free-text comment.

 

 

An information sheet and invitations to participate were sent by email to all the radiotherapist staff in the 28 departments where the RT workflow audit had been implemented. Responders were informed that the findings might be published and that their responses would be anonymous.

 

Results

The RT workflow audit

Scores improved from 60% in some units in 2016 to >90% in all units for at least 2 years since 2018. Sample task items that showed progressive improvement were: (i) the details on the request forms for RT, e.g. patient anatomical region, which should have a planning computed tomography scan to determine the tumour and normal tissue volumes; (ii) a treatment planning checklist, including the tumour and normal tissue volumes to be planned and the radiation dose prescribed; (iii) weekly review of the patient's set-up for treatment, including the distance of the patient's surface to the radiation source; and (iv) weekly patient assessment, e.g. side-effects, patient weight and check radiographs of the linear accelerator set-up or portal image.

The online survey

A total number of 151 radiotherapists were invited to participate in the online survey. The number of responders was 58, giving a responder rate of 38%. The margin of error of the results was 10% using the SurveyMonkey calculator (https://www.surveymonkey.com/mp/margin-of-error-calculator/).

The results of the online survey are set out in Tables 1, 2 and 3. Running totals are given for the sum of the highest two answers for responses with a 5-point rating scale.

The responders reported favourable experience with the RT workflow audit. The audit's task items were considered appropriate by 77% of responders, and feedback was reported by 78% of them (Table 1). The audit was considered very or extremely valuable to their unit's service delivery by 58% of responders. Changes in the unit as a result of the audit were reported by 77% of responders. The audit was very useful or extremely useful in maintaining personal professional standards to 75% of respondents (Table 2).

The proportion of responders who were very and extremely supportive of continuing with the audit was 77% (Table 3).

The following are samples of free-text comments by responders: 'I liked that the individuals conducting the audits were very friendly yet professional, and did not make it feel as if they were faultfinding you, or the department, but made it feel as if they were here to help you achieve excellence in the units and as individuals.' 'In the beginning and when the information systems were new to us all, the audits were very helpful. It identified areas where we could improve workflow within our unit and standardise how staff perform a certain role. Since we are now fully using the quality check lists in our daily processes, I think it has become less helpful.' ' The audit process is essential to our business as radiotherapists, it provides a standard to which we all must subscribe and comply. It aids in the standardisation of our company's workflow as RTs.' 'It is time consuming but very necessary.'

 

Discussion

Assessment of the quality of healthcare includes evaluation of structures, processes and outcomes.[1,2]

Regular audits of key process measures provide important information for quality improvement programmes and complement more formal accreditation programmes. Audits used for accreditation of facilities, such as that performed by the South African Department of Health National Core Standards[3] and in oncology,[5-8] mainly assess structural measures, such as the details of the facilities and whether relevant polices are in place, rather than performance. Accreditation audits document the competence of units to function rather than the function itself. They may also be relatively onerous to undertake, so that they are repeated at relatively long intervals of 3 - 5 years.

The clinical use of checklists has been shown to reduce death and complication rates in surgery,[9] and is encouraged by the World Health Organization. However, implementation of their use has been shown to be problematic.[10] Measures to improve the use of checklists include providing education/training, feedback of local data, and fostering strong leaders. Inclusion of the records of checklist use, where possible, supports these approaches.

Radiotherapists form an interdependent multidisciplinary team with medical physicists and radiation oncologists for the safe and effective administration of RT to patients. It is common practice for radiation oncologists to seek to minimise errors through the review of treatment plans at 'chart rounds'. In a prospective study,[11] detectable errors were inserted into treatment plans. However, these errors were detected during the review in only 55% of cases. These authors urge that reviews should be standardised and systematic. The RT workflow audit in the present report improves documentation of processes and will be helpful in reducing errors and maintaining high standards.

The RT workflow audit scored the documentation of a list of task items that included clinical data, professional tasks, and checklists as measures of process. The score results of the audit improved progressively and have remained at >90% in all 28 units with 32 linear accelerators in recent audits over the past 2 years.

The online survey of users of the RT workflow audit was undertaken to assess their user experience and its professional value. The responder rate to the survey was 38%. This figure is in line with a benchmark responder rate for anonymous online surveys of 5 - 30%. It is also similar to the responder rate of 40% obtained in a 2017 survey of the user experiences of an audit for accreditation purposes.[8]

The RT workflow audit is sustainable, and its continuation is strongly supported by the online survey findings. The responders in the survey gave a high rating to the audit in terms of its processes, and its value in improving care and maintaining personal professional standards.

All the free-text comments in the survey are viewed as constructive. There is a concern that the audit takes time. The review of each folder takes 30 - 60 minutes, so the total RT workflow of a unit takes 2 -4 hours. This will be protected time as part of the working day. The RT workflow audit will also be conducted annually rather than every 2 years. The comment that the audit is now less helpful than when it was first introduced has been noted. This is acknowledged, although continuing the audit is considered necessary to maintain the scores achieved.

A multidisciplinary approach with inclusion of processes such as checklists into routine clinical documentation will assist in reducing the incidence of errors and in developing quality improvement programmes.

Declaration. None.

Acknowledgements. The authors are grateful to Drs Fergus Macbeth and Henriette Burger for helpful comments.

Author contributions. All authors contributed to the design of the study and reviewed the manuscript.

Funding. None.

Conflicts of interest. None.

 

References

1. McDonald KM, Sundaram V, Bravata DM, et al. 5: Conceptual frameworks and their application to evaluating care coordination interventions. In: Closing the quality gap: A critical analysis of quality improvement strategies (Vol. 7: Care Coordination). Rockville, Md.: Agency for Healthcare Research and Quality, June 2007. https://www.ncbi.nlm.nih.gov/books/NBK44008/ (accessed 4 January 2021).         [ Links ]

2. Donabedian A. The quality of care: How can it be assessed? JAMA 1998;260(12):1743-1748. https://doi.org/10.1001/jama.1988.03410120089033        [ Links ]

3. National Department of Health, South Africa. National Core Standards for Health Establishments in South Africa. 2011. https://static.pmg.org.za/docs/120215abridge_0.pdf (accessed 4 January 2021).         [ Links ]

4. Quality Assurance Team for Radiation Oncology (QUATRO) International Atomic Energy Agency. Comprehensive audits of radiotherapy practices: A tool for quality improvement. Vienna, 2007. http://wwwpub.iaea.org/MTCD/publications/PDF/Pub1297_web.pd (accessed September 2020).         [ Links ]

5. Thwaites DI, Scalliet P, Leer JW, Overgaard J. Quality assurance in radiotherapy. Radiother Oncol 1995;35(1):61-73. https://doi.org/10.1016/0167-8140(95)01549-V        [ Links ]

6. Royal Australian and New Zealand College of Radiologists (RANZCR), Faculty of Radiation Oncology (FRO), Australian Institute of Radiography (AIR), Australian College of Physical Scientists and Engineers in Medicine (ACPSEM). Tripartite Radiation Oncology Practice Standards. 2011. http://www.radiationoncology.com.au/supporting-docs/ranzcr-standards.pdf (accessed 4 January 2021).         [ Links ]

7. Cottera WG, Dobelbower RR jr. Radiation oncology practice accreditation: The American College of Radiation Oncology, Practice Accreditation Program, Guidelines and Standards. Crit Rev Oncol Hematol 2005;55(2):93-102. https://doi.org/10.1016/j.critrevonc.2005.03.002        [ Links ]

8. Abratt RP, Eedes D Bailey B, et al, Audits in oncology units - an effective and pragmatic approach. S Afr Med J 2017;107(6):493-496. https://doi.org/10.7196/SAMJ.2017.v107i6.12356        [ Links ]

9. Haynes AB, Weiser TG, Berry WR, et al. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med 2009;360:491-499. https://doi.org/10.1056/NEJMsa0810119        [ Links ]

10. Russ SJ, Sevdalis N Moorthy K, et al, A qualitative evaluation of the barriers and facilitators toward implementation of the WHO Surgical Safety Checklist across hospitals in England. Ann Surg 2015;261(1):81-91 https://doi.org/10.1097/SLA.0000000000000793        [ Links ]

11. Talcott WJ, Lincoln H, Kelly JR, et al. A blinded, prospective study of error detection during physician chart rounds in radiation oncology. Pract Radiat Oncol 2020;10(5):312-320. https://doi.org/10.1016/j.prro.2020.05.012        [ Links ]

 

 

Correspondence:
R P Abratt
raymond.abratt@iconsa.co.za

Accepted 6 October 2020

Creative Commons License All the contents of this journal, except where otherwise noted, is licensed under a Creative Commons Attribution License