SciELO - Scientific Electronic Library Online

 
vol.106 número10 índice de autoresíndice de materiabúsqueda de artículos
Home Pagelista alfabética de revistas  

Servicios Personalizados

Articulo

Indicadores

Links relacionados

  • En proceso de indezaciónCitado por Google
  • En proceso de indezaciónSimilares en Google

Compartir


SAMJ: South African Medical Journal

versión On-line ISSN 2078-5135
versión impresa ISSN 0256-9574

SAMJ, S. Afr. med. j. vol.106 no.10 Pretoria oct. 2016

http://dx.doi.org/10.7196/samj.2016.v106i10.10619 

RESEARCH

 

Comparison of the prevalence and characteristics of inpatient adverse events using medical records review and incident reporting

 

 

W M MacharíaI; C M MuteshiII; S Z WanyonyiII; A M MukaindoIII; A IsmailIV; H EkeaV; A AbdallahVI; J M ToleIV; A K NgugiVII

IMB ChB, MMed (Paeds), MSc; Department of Paediatrics, Faculty of Health Sciences, Aga Khan University, Nairobi, Kenya
IIMB ChB, MMed (O&G), Department of Obstetrics and Gynaecology, Faculty of Health Sciences, Aga Khan University, Nairobi, Kenya
IIIMMed (O&G), Department of Obstetrics and Gynaecology, Faculty of Health Sciences, Aga Khan University, Nairobi, Kenya
IVMB ChB, MMed (Paeds), Department of Paediatrics, Faculty of Health Sciences, Aga Khan University, Nairobi, Kenya
VMB ChB, Department of Medicine, Faculty of Health Sciences, Aga Khan University, Nairobi, Kenya
VIMB ChB, MMed (Surg), Department of Surgery, Faculty of Health Sciences, Aga Khan University, Nairobi, Kenya
VIIBSc, MSc, PhD, Population Health Sciences, Faculty of Health Sciences, Aga Khan University, Nairobi, Kenya

Correspondence

 

 


ABSTRACT

BACKGROUND. Information on adverse events (AEs) in hospitalised patients in developing countries is scanty.
OBJECTIVE. To compare the magnitude and characteristics of inpatient AEs in a tertiary, not-for-profit healthcare facility in Kenya, using medical records review and incident reporting.
METHODS. Estimation of prevalence was done using incidents reported in 2010 from a random sample of medical records for hospital admissions. Nurse reviewers used 18 screening criteria, followed by physician reviewers to confirm occurrence. An AE was defined as an unexpected clinical event (UE) associated with death, disability or prolonged hospitalisation not explained by the disease condition. The kappa statistic was used to estimate inter-rater agreement, and analysis was done using logistic regression.
RESULTS. The study identified 53 UEs from 2 000 randomly selected medical records and 33 reported UEs from 23 026 admissions in the index year. The prevalences of AEs from medical records review and incident reports were 1.4% (95% confidence interval (CI) 0.9 - 2.0) and 0.03% (95% CI 0.012 - 0.063), respectively. Compared with incident reporting, review of medical records identified more disability (13.2% v. 0%; p=0.03) and prolonged hospital stays (43.4% v. 18.2%; p=0.02).
CONCLUSIONS. Review of medical records is preferable to incident reporting in determining the prevalence of AEs in health facilities with limited inpatient quality improvement experience. Further research is needed to determine whether staff education and a positive culture change through promotion of non-punitive UE reporting or a combination of approaches would improve the comprehensiveness of AE reporting.


 

 

The incidence of medical adverse events (AEs) in hospitalised patients was estimated at 9.2% (interquartile range 4.6 - 12.4) by de Vries et al.[1]in 2008. An event was defined as an incident resulting in death, a prolonged hospital stay or disability but not attributable to the underlying medical condition. The Canadian AEs study[2] found a rate of 7.5% from review of 3 745 medical records in four hospitals. Public hospitals in a multicentre study conducted in Africa and the Middle East had a range of 2.5 - 18.4%.[3] Other methods used to assess AEs include incident reports, interviewing of healthcare providers, direct observation, external audits, confidential inquiries and complaints. Use of a hybrid electronic medical record system has demonstrated some benefit in combining prospective and retrospective approaches to uncover surgical AEs. Laing et al.[4] identified 71.4% errors prospectively and 28.6% retrospectively. They found that a tick-box system improved the quality of documentation, which could help solve the disadvantage inherent in retrospective reviews.[5] Prospective approaches involve voluntary written documentation or reporting of incidents, with concerns of underreporting when a culture of quality monitoring is not entrenched.[6]

 

Objective

The practice of AE reporting has not gained ground in many institutions in low-income countries. This may be due to inadequate awareness of its importance in informing service quality improvement to minimise harm to patients. This study was carried out with the objective of comparing the prevalence and patterns of AEs in hospitalised patients, using review of medical records and incident reporting.

 

Methods

The study was carried out in a 254-bed tertiary hospital in Nairobi, Kenya. The hospital has ~20 000 admissions per annum, comprising medical, surgical, paediatric and obstetric patients. A retrospective review of randomly selected medical records of patients admitted in 2010 was undertaken to identify the presence or absence of AEs documented in the index year. All inpatient medical records and reported clinical incidents for the index year were eligible for inclusion. When more than one incident occurred in a patient in the index year, all were included as separate occurrences provided they were not related.

Using an estimated AE prevalence of 14.5%,[3] power of 80% and 5% precision, a sample size estimate of about 200 medical records was determined adequate. We assumed that 20% of the records would be incomplete and therefore added an extra 40 medical charts, giving a total sample size of 245 records. However, in order to obtain a larger number to enable meaningful comparison, we opted to increase the number of randomly sampled files to 2 000. All incidents recorded in the year were reviewed for fulfillment of predefined criteria for unintended or adverse clinical events. All the 1 665 incidents reported in the index year were included in the review.

Unexpected clinical outcomes were defined as unexpected clinical events (UEs) to distinguish them from AEs. Events had to have: (i) occurred at any time before the index admission and been detected during, or been responsible for, the index admission; (ii) occurred and been detected during the index admission; or (iii) occurred during the index admission but been detected on a subsequent admission. An AE was defined as an incident that resulted in death, disability or a prolonged hospital stay and was not explainable by an underlying medical condition. Other occurrences were defined as UEs.

Two experienced nurses used RF1 forms (Appendix 1) to screen for unexpected events using 18 criteria comprising conditions or circumstances commonly associated with AEs.[2,3] Four physicians then reviewed selected files using RF2 forms (Appendix 2). A panel of content experts in the surgical and medical fields was identified to act as a resource for technical input, but not to determine the occurrence of incidents. All reviewers underwent 2-day training. Pilot testing was conducted using a convenience sample of 20 records to ensure comprehension of the study techniques.

The nurse reviewers ensured that the physicians were not assigned to review medical records of patients in whose care they could previously have participated.

The physician reviewers then scrutinised all the records that screened positive using RF2 forms for the presence of AEs. They then categorised incidents according to severity, location where the event occurred, attribution and preventability.

Incident reports were identified by accessing both computerised records and paper files to determine the number of events reported in the year, the nature of the incidents and the circumstances surrounding them. Initial screening was done to exclude non-clinical incidents. Information on remedial measures taken following root-cause analysis was abstracted and entered into data collection forms. A random sample of 10% of the medical records was subjected to a second review by a reviewer with longer experience in similar studies, and who was not involved in the earlier reviews, for the purpose of validation.

The primary outcome measure was the prevalence of AEs, while secondary outcomes included UEs, circumstances leading to the UEs, site of incident occurrence and preventability.

The prevalence of AEs from medical records review was computed as a percentage using the total number of AEs as numerator and the total number of sampled admissions as denominator. Similarly, for incident reporting the numerator was the number of AEs and the denominator the total number of hospital admissions in the index year. A p-value of <0.05 was considered significant, and 95% confidence intervals (CIs) were determined around primary outcome estimates. The χ2 test was used to compare independent categorical variables. Prevalence and bias-adjusted kappa statistics were used to estimate inter-rater agreement, with a value of <0.4 considered poor agreement, 0.4 - 0.6 moderate agreement and >0.6 good agreement.[12]

The study was approved by the World Health Organization (Ref: PS09004) and the Aga Khan University Hospital Ethics Review Committees (AKU/REC-06052011). Anonymous record keeping was used to delink case records from study data, and patient confidentiality was maintained.

 

Results

There were 23 026 hospital admissions in the year 2010, from which 2 000 records were randomly selected for review. Of these, 317 were screening criteria-positive, with 53 (2.7%; 95% CI 2.0 -3.5) having documented occurrence of a UE, but only 28/2 000 cases were associated with disabilities, a prolonged hospital stay or disabilities not attributable to the primary medical conditions, giving a prevalence of 1.4% (95% CI 0.9 - 2.0). During the same study period, 233/1 665 (14.0%) of the reported incidents were of a clinical nature. From the total of 23 026 admissions for 2010, 33 (0.14%, 95% CI 0.10 - 0.20) had UEs but only 7 fulfilled the criteria for an AE, giving a prevalence of 0.03% (95% CI 0.01 - 0.06).

Forty-three out of 428 (10.0%) of the combined medical review and incident reports were reviewed by the 'expert reviewer' for validation. The inter-reviewer agreement for the reviewers was moderate at κ=0.40 (95% CI 0.13 - 0.66). The figure improved marginally to κ=0.45 (95% CI 0.38 - 0.74) after adjusting for prevalence and bias.

Evidence of healthcare causation was identified in 31/53 cases (58.5%) from medical records and in 20/33 cases (60.6%) by incident reporting. Comparing review of medical records with incident reporting, disability (13.2% v. 0%) and prolonged hospital stay (43.4% v. 18.8%) were more likely to be detected from review of records (Table 1).

 

 

The majority of UEs took place in the study hospital, predominantly in the patient's ward or room (Table 2).

 

 

The most frequently observed consequence of UEs from both data sources was prolonged hospital stay. Of the UEs detected by medical review, 11/15 (73.3%) occurred while the patient was hospitalised and 4/15 (26.7%) outside the facility.

Thirty-one of 53 medical records (58.5%) attributed UE causation to healthcare management. Another 77.4% (95% CI 52.0 - 87.8) of the events were non-procedure-related. Drugs were associated with UEs in 86.9% and 80.0% according to medical records and incident reports, respectively. Other health products that contributed to UEs, but less commonly, were blood products, medical devices and medical equipment. Clinical circumstances associated with UEs as determined by medical records and incident reports are shown in Table 3.

 

 

Of procedure-related UEs, 87.1% (95% CI 70.2 - 96.4) occurred during therapeutic interventions. Medical records provided better prediction of UEs than incident reporting (62.3% v. 36.4%; p=0.02).

However, the two methods did not differ in ability to detect preventability of UEs (30.2% v. 42.4%; p=0.18).

According to medical records, the underlying health status of the patient alone could have contributed to the occurrence of UEs in 64.7% of patients (95% CI 46.5 - 80.3). Eight patients experienced UEs that were not associated with prolonged hospital stay, disability or death so were considered not to be serious, in keeping with the a priori definition. Incident reports identified 5 cases related to a therapeutic intervention, in 2 of which there was an error during administration, while the rest were considered unavoidable or due to undeterminable circumstances. Three cases were rather complex to manage, so it was not possible to attribute their cause to any specific care omission or commission.

Of 19 cases categorised by degree of urgency, 4 were of moderate urgency while 15 were of low or no urgency. Four of 6 events were related to inadequate communication or reporting by the healthcare team. Seventy-five percent of cases (9/12) could have benefited to a great extent or moderately if appropriate management had been given. The risk that a UE or AE would have resulted from management provided was moderate to high in 57.9% of the patients with UEs (95% CI 33.5 - 79.7%). Reviewers considered that 81% of competent healthcare professionals would have managed the patients with UEs in a similar manner.

 

Discussion

Our previous work in a multicentre study involving review of medical records in public hospitals in Africa and the Middle East found a prevalence of AEs of 2.5 - 18.4%.[3] The mean in the two hospitals in Kenya was 14.5%, which is much higher than the 1.4% in the medical records review in this study. A similar methodology was used for the medical records review, and some reviewers who had participated in the previous study were involved. This difference may be attributed to the higher standards of care in the current study site, which is a tertiary not-for-profit academic medical centre that at the time of the study was in the preparatory stages of Joint Commission International accreditation, which was eventually granted in 2013. Whereas AEs that are not associated with serious complications could fail to be documented, leading to underestimation, we only sought to document the severe ones that would readily be picked up from records, since death, disabilities and prolonged hospital stay would be evident.

Review of medical records in this study identified many more AEs than incident reporting (1.4% v. 0.03%), clearly indicating that many important events are never reported. This very low reporting may be a result of fear of being held responsible for omissions or commissions. It could also be due to inadequate staff education on the importance of reporting.

The majority of AEs in this study occurred in the rooms in which patients were receiving medical care. There can be major variations in the quality of care provided to hospitalised patients at different levels of care, especially for trauma patients.[7] Medical records review was better than incident reporting at identifying events that resulted in disability (13.2% v. 0%; p=0.03) and prolonged hospitalisation (43.4% v. 18.2%; p=0.02). This is not really surprising considering that these factors were triggers for AE scrutiny in medical records reviews, while incident reporting is expected to be spontaneous. An important drawback of the medical records approach is inability to fully reveal circumstances surrounding the event. Complementing review of records with structured morbidity and mortality meetings may be valuable. Clarke et al.[8] demonstrated the usefulness of such audit meetings in dissecting out human error contributions. Incident reporting requires a change in institutional culture so that it is not punitive to those who disclose UEs. Where such a culture is not entrenched, non-reporting would be even higher for patients on treatment for chronic and complex conditions, as true AEs in such patients may erroneously be assumed to be inevitable. There is also a tendency for health professionals to interpret AEs as expected complications of medical procedures even when they are not, and therefore failing to report them. This problem can, however, be overcome through comprehensive medical record review.[9]

Christians-Dingelhoff et aZ.[10] found that only 3.6% of AEs identified by record review were identified by other reporting methods. As in our study, others also found different methods to be complementary with little overlap in reporting of AEs.[11] This is especially important because some AEs not captured by incident reports would not have been detected using alternative methods, thus concealing opportunities for service improvement. Complementing incident reports with medical records review and other reporting modalities would ensure a more comprehensive assessment of AEs. Clarke et al.[12] demonstrated how modern error theory used in commercial aviation could be exploited in health to identify missed injuries in trauma. However, we did not find any literature on how this could be applied for missed AEs in facility incidence or prevalence estimation.

By its nature, incident reporting included a large number of non-clinical reports that tended to mask non-reporting of important clinical incidents. A good reporting system should be able to clearly separate clinical and non-clinical incidents, as users are different. Our review was made possible by robust medical record keeping and an electronic incident reporting system. Lack of appropriate systems for data retrieval and analysis would be a major constraint for resource-deprived facilities. Nurse and physician reviewers with ample experience and training, as we had in our study, help to minimise observer variability.

Management flaws were responsible for nearly 60% of the AEs, with management itself likely to have contributed to AEs in 58% of cases. Irrespective of the data collection method used, >80% of AEs related to medications. Our reviewers estimated that in 75% of AEs the patient could have benefited from more appropriate care than had been provided.

Level of agreement among physicians completing RF2 forms was estimated using the prevalence-adjusted kappa, as described by Bennet et al. in 1954 and elaborated by Nam,[13] to give a more reliable degree of agreement. This reflected an acceptable chance-corrected agreement of 0.45, representing moderately good agreement.

We observed that review of medical records also exposed more disabilities and prolonged hospital stay associated with the events. Laing et al.,[4]using a hybrid electronic method, found almost three times more events prospectively than by scrutiny of records. Although the settings may not be comparable, given that the studies were carried out in different continents serving different patient populations, the difference in findings suggests major under-reporting of incidents in our study. Institutional cultural change through staff education on the purpose of reporting and assurance that reporting will not result in punishment may be needed to alter perceptions and practice. There is little overlap in the events, emphasising the need for combining different approaches to be comprehensive. The search for innovative approaches to identify the many factors that impact on quality of care to hospitalised patients continues, even as known combined approaches demonstrate promise.

 

Conclusions

Review of medical records is preferable to incident reporting in health facilities with limited inpatient quality improvement experience. The approach identifies more AEs and exposes more factors associated with events. Further research is needed to determine whether staff education and positive culture change through promotion of non-punitive reporting, or a combination of approaches, would improve comprehensiveness of AEs reporting.

Authors' contributions. WMM was involved in conceptualisation, proposal writing, analysis and drafting of the final manuscript. CMM participated in study implementation, analysis and drafting of the final manuscript. SZW was involved in conceptualisation, proposal writing and drafting of the final manuscript. AMM, AI, HE, AA and JMT all participated in data collection and drafting of this manuscript. AKN participated in data analysis and drafting of the manuscript. All the authors reviewed the manuscript and approved submission.

Acknowledgements. This work was supported by the World Health Organization Small Grants Program (WHO Ref: 2011/167789-0). We wish to acknowledge the enormous amount of guidance we received from Nittita Prasopa-Plaizier (WHO, Geneva) during the proposal writing and manuscript preparation stages. Georghe Banica (WHO, Geneva) similarly provided us with requisite administrative support for completion of this work. Janet Musia made invaluable input in data and project management.

 

REFERENCES

1. De Vries EN, Ramrathan MA, Smorenburg SM, Gouma DJ, Boermeester MA. The incidence and nature of in-hospital adverse events: A systematic review. Qual Saf Health Care 2008;17(3):216-223. DOI: 10.1136/qshc.2007.023622        [ Links ]

2. Baker GR, Norton PG, Flintoft V, et al. The Canadian adverse events study: The incidence of adverse events among hospital patients in Canada. CMAJ 2004;170(11):1678-1686. DOI:10.1503/cmaj.1040498.         [ Links ]

3. Wilson RM, Michel P, Olsen S, et al. Patient safety in developing countries: Retrospective estimation of scale and nature of harm to patients in hospital. BMJ 2012;344:e832. DOI:10.1136/bmj.e832        [ Links ]

4. Laing G, Bruce J, Skinner D, et al. Using a hybrid electronic medical record system for the surveillance of adverse surgical events and human error in a developing world surgical service. World J Surg 2015;39(1):70-79. DOI:10.1007/s00268-014-2766-x        [ Links ]

5. Laing GL, Bruce JL, Clarke DL. Tick-box admission forms improve the quality of documentation of surgical emergencies but have limited impact on clinical behaviour. S Afr Med J 2014;104(6):435-438. DOI:10.7196/SAMJ.7693        [ Links ]

6. Vinen J. Incident monitoring in emergency departments: An Australian model. Acad Emerg Med 2000;7(11):1290-1297.         [ Links ]

7. Stewart WWD, Farina Z, Clarke DL, Thomson SR. Variations in levels of care within a hospital provided to acute trauma patients. S Afr J Surg 2011;49(4):194-198.         [ Links ]

8. Clarke DL, Furlong H, Laing GL, Aldous C, Thomson SR. Using a structured morbidity and mortality meeting to understand the contribution of human error to adverse surgical events in a South African regional hospital. S Afr J Surg 2013;51(4):122-126. DOI:10.7196/SAJS.1537        [ Links ]

9. Levtzion-Korach O, Frankel A, Alcalai H, et al. Integrating incident data from five reporting systems to assess patient safety: Making sense of the elephant. Jt Comm J Qual Patient Saf 2010;36(9):402-410.         [ Links ]

10. Christians-Dingelhoff I, Smits M, Zwaan L, Lubberding S, van der Wal G, Wagner C. To what extent are adverse events found in patient records reported by patients and healthcare professionals via complaints, claims and incident reports? BMC Health Serv Res 2011;11:49. DOI:10.1186/1472-6963-11-49        [ Links ]

11. De Feijter JM, de Grave WS, Muijtjens AM, Scherpbier AJ, Koopmans RP. A comprehensive overview of medical error in hospitals using incident-reporting systems, patient complaints and chart review of inpatients deaths. PLoS One 2012;7(2):e31125. DOI:10.1371/journal.pone-0031125        [ Links ]

12. Clarke DL, Gouveia J, Thomson SR, Muckart DJ. Applying modern error theory to the problem of missed injuries in trauma. World J Surg 2008;32(6):1176-1182. DOI:10.1007/s00268-008-9543-7        [ Links ]

13. Nam J. Comparison of validity of assessment methods using indices of adjusted agreement. Stat Med 2007;26(3):620-632. DOI:10.1002/sim.2562        [ Links ]

 

 

Correspondence:
WM Macharia
william.macharia@aku.edu

Accepted 7 February 2016.

 

 

Appendix 1. RF1 form: Adverse event detection questionnaire

 


Appendix 1- Click to enlarge

 

Appendix 2. RF2 form: Second questionnaire

 


Appendix 2- Click to enlarge

Creative Commons License Todo el contenido de esta revista, excepto dónde está identificado, está bajo una Licencia Creative Commons