CLINICAL TRIAL

 

Perioperative comparison of the agreement between a portable fingertip pulse oximeter v. a conventional bedside pulse oximeter in adult patients (COMFORT trial)

 

 

R Ν Smith^I; R Hofmeyr^II

^IMB ChB, DA (SA), FCA (SA), MMed (Anaes); Department of Anaesthesia and Perioperative Medicine, Groote Schuur Hospital and Faculty of Health Sciences, University of Cape Town South Africa
^IIMB ChB, Dip PEC (SA), DA (SA), FCA (SA), MMed (Anaes), FAWM; Department of Anaesthesia and Perioperative Medicine, Groote Schuur Hospital and Faculty of Health Sciences, University of Cape Town South Africa

Correspondence

 

 

---

ABSTRACT

BACKGROUND: Low-cost, portable fingertip pulse oximeters are widely available to health professionals and the public. They are often not tested to International Organization for Standardization standards, or only undergo accuracy studies in healthy volunteers under ideal laboratory conditions
OBJECTIVES: To pragmatically evaluate the agreement between one such device and a conventional bedside pulse oximeter in a clinical setting, in patients with varied comorbidities and skin pigmentations
METHODS: A single-centre equipment comparison study was conducted. Simultaneous measurements were obtained in 220 patients with both a Contec CMS50D Fingertip Pulse Oximeter and a Nihon Kohden Life Scope MU-631 RK conventional bedside monitor. Peripheral oxygen saturations (SpO₂) and pulse rates were documented, and patients' skin tone was recorded using the Fitzpatrick scale. Data were assessed using a Bland-Altman analysis with bias, precision and limits of agreement (LOA) calculated with 95% confidence intervals (CIs). A priori acceptability for LOA was determined to be 3%, in keeping with international standards
RESULTS: The mean difference (therefore bias) between the conventional and fingertip oximeters for all data was -0.55% (95% CI -0.73 --0.36). Upper and lower limits of agreement were 2.16% (95% CI 1.84 - 2.47) and -3.25% (95% CI -3.56 - -2.94). Regression analysis demonstrated worsening agreement with decreasing Sp0₂. When samples were separated into normal' (Sp0₂ >93%) and 'hypoxaemic' (Sp0₂ <93%) groups, the normal range displayed acceptable agreement between the two oximeters (bias -0.20% with LOA 2.20 - -2.27), while the hypoxaemic group fell outside the study's a priori limits. Heart rate measurements had a mean difference of -0.43 bpm (LOA -5.61 - 4.76). The study was not powered to detect differences among the skin tones, but demonstrated no trend for this parameter to alter the Sp0₂ measurements
CONCLUSIONS: During normoxia, portable fingertip pulse oximeters are reliable indicators of Sp0₂ and pulse rates in patients with various comorbidities in a pragmatic clinical context. However, they display worsening agreement with conventional pulse oximeters during hypoxaemia. Skin tones do not appear to affect measurements adversely

---

 

 

The ability to assess a patient's arterial oxygen saturation accurately and non-invasively has become an accepted standard of care in the perioperative period.^[1] Pulse oximetry is considered an essential item of equipment for providing safe anaesthesia by various regulatory bodies throughout the world, including the South African Society of Anaesthesiologists and the World Federation of Societies of Anaesthesiologists,^[2,3] which adopted the International Standards for a Safe Practice of Anaesthesia^[4] in 1992. Pulse oximeters are included in the World Health Organization (WHO)'s Surgical Safety Checklist (SSC),^[5] which is used in health facilities throughout the world before commencement of a surgical procedure. Studies have demonstrated that the WHO SSC has nearly doubled patients' likelihood of receiving proven standards of surgical care, and substantially reduced complications and deaths.^[6]

Pulse oximetry is non-invasive, safe and currently performed routinely on all surgical patients during admission, intraoperatively and postoperatively, without requiring specific consent. It is more cost-effective, less painful, easier to perform and more readily available than arterial blood gas analysis. It has largely replaced this method in many clinical situations, unless carbon dioxide or acid-base status is specifically required. Portable fingertip pulse oximeters have the additional benefits of being cost-effective, highly compact, portable, battery operated and easy to use. When pulse oximeters are manufactured, they are tested for accuracy against blood gas analysis in healthy volunteers breathing hypoxic gas mixtures under ideal laboratory conditions. There are published standards from the US Food and Drug Administration (FDA) on how this should be performed, including statistical methods appropriate for device comparison, such as Bland-Altman analysis.^[7] A review of the recent literature reveals a paucity of accuracy studies utilising fingertip pulse oximeters in patients in the clinical setting.

 

Objectives

To pragmatically investigate the performance of these devices in adult surgical patients in a hospital/theatre environment. Studying the performance of portable fingertip pulse oximeters may result in the availability of cheaper devices in medical facilities in low-to middle-income areas, give anaesthetists and other medical personnel the confidence to make clinical decisions based on these highly portable devices, add further data to existing studies, and help assess whether darker skin pigmentation affects the performance of these devices in the clinical setting, which is highly relevant in an African country such as South Africa (SA).

 

Methods

Patients

This prospective, quantitative equipment comparison study took place at Groote Schuur Hospital, a tertiary-level institution in Cape Town, SA, over a 4-week period. Institutional approval was granted by the University of Cape Towns Human Research Ethics Committee (ref. no. HREC 572/2017). Written informed consent was waived on the provision that no patient-identifiable data were collected and pulse oximetry was already being used for routine monitoring purposes. Furthermore, simple verbal consent for inclusion was obtained from conscious patients. All adult surgical patients aged >18 years who presented for elective or emergency surgery were considered eligible for the study and recruited by convenience sampling in perioperative areas such as pre-assessment clinics, recovery rooms, operating theatres and intensive care units (ICUs). Exclusion criteria (Table 1) were conditions known to cause inaccuracies in pulse oximetry, or infectious diseases with a high risk of transmission.^[8-14]

 

 

Measurements

Data were recorded by the principal author (RNS) on an Excel spreadsheet, version 16.19 (Microsoft, USA) and archived in password-protected Cloud storage. The CMS50D Fingertip Pulse Oximeter (Contec, USA) was selected as the test device for this study because of its relatively low cost (-ZAR500, compared with the control bedside pulse oximeter, which costs -ZAR200 000), its ease of availability in SA, and the fact that that it was one of two devices identified that met International Organization for Standardization (ISO) and FDA standards in healthy test subjects in a prior study^[15] The device was purchased privately by the authors. A Life Scope MU-631 RK bedside monitor (Nihon Kohden, Japan) was provided by the Department of Anaesthesia and Perioperative Medicine at Groote Schuur Hospital for the control measurements. It is a commonly used monitor in the hospital and was calibrated by the manufacturer.

Sample size was calculated using results obtained from an unpublished pilot study performed using the same methods. The Monte-Carlo simulation method, as outlined by Lu et al.,^[16]was used, utilising MedCalc Statistical Software version 18.6 (MedCalc, Belgium; http://www.medcalc.org). A maximum difference in peripheral oxygen saturation (Sp0₂) of 3%, type I error 0.05 and type II error 0.01 gave a sample size estimate of 220.

The fingertip and conventional bedside pulse oximeter probes were applied simultaneously to the same hand, contralateral to the blood pressure cuff. Once the waveform on both pulse oximeters was confirmed, readings were taken at 30 seconds post-application to allow for the time averaging of each pulse oximeter. Non-patient identifiable data were recorded, limited to Sp0₂, heart rate, skin colour (estimated by the Fitzpatrick scale) and qualitative waveform strength of signal (good or poor). Similar to other studies, the Fitzpatrick scale was selected (rather than methods such as the von Luschan scale or reflectance spectrophotometry) for reasons of simplicity, repeatability and cost containment.^[15,17,19] There was no interruption to existing patient monitors. The study devices were cleaned with a commercial antiseptic solution between patients.

Statistical analysis

Bland and Altman have established bias and precision estimates as the standard reported statistic when comparing agreement between a new or less-established measurement technique and an established one.^[20-23] When the a priori limits of agreement are small (95% limits of agreement defined as bias plus or minus 1.96 standard deviations) and not of clinical importance, the two measurement techniques can be considered interchangeable. For this study, we determined that an Sp0₂ difference of <3% is not of clinical importance, which is in keeping with the ISO and FDA testing protocol. Data analysis was performed by the authors with Medcalc (as above).

 

 

Results

We obtained 220 simultaneous pulse oximetry measurements. SpO₂, pulse rate, skin-tone classification and the clinical setting are summarised in Table 2. The mean difference (bias) between the conventional and fingertip oximeters for all data was -0.55% (95% confidence interval (CI) -0.73 - -0.36, Fig. 1). Upper and lower limits of agreement were 2.16% (95% CI 1.84 - 2.47) and -3.25% (-3.56 --2.94). Regression analysis demonstrated worsening agreement with decreasing SpO₂. When samples were separated into 'normal' (Sp0₂ >93%, Fig. 2) and 'hypoxaemic' (SpO₂ <93%, Fig. 3) groups, the normal range displayed acceptable agreement between the two oximeters (bias -0.20% with limits of agreement (LOA) 2.20 - -2.27), while the hypoxaemic group fell outside the study's a priori limits. Heart rate measurements had a mean difference of -0.43 bpm (LOA -5.61 - 4.76, Fig. 4). The study was not powered to detect difference among the skin tones, but demonstrated no trend for this parameter to alter the Sp0₂ measurements (Fig. 5).

 

 

 

 

 

 

 

 

 

 

Discussion

Pulse oximeters non-invasively measure and display heart rate and SpO, derived from photoplethysmographic measurements at two rapidly alternating wavelengths of light.^[11-12,24] Oxygenated blood (oxyhaemoglobin) and deoxygenated blood (deoxyhaemoglobin) differ in their absorption spectra. A sensor containing a light-emitting diode and a light-detecting photodiode are placed on a cutaneous vascular bed, such as a fingertip or earlobe, that can be transilluminated. Arterial pulsations are identified by plethysmography, allowing corrections for light absorption by non-pulsating venous blood and tissue. The ratio of absorption at the red and infrared wavelengths is analysed by a microprocessor, and SpO, is estimated from a stored calibration curve.

Oximeter testing is based on experimental measurements in healthy volunteers who are subjected to rebreathing controlled hypoxic gas mixtures under laboratory conditions, which includes controlling for subject movement and ambient light. Oxygen saturation of haemoglobin is then determined by both the pulse oximeter (SpO₂) and in vitro laboratory multiwavelength co-oximeter (SaO₂ considered the gold standard). Pulse oximeters are inaccurate at low blood oxygen saturations because researchers are limited in terms of the degree of hypoxaemia inducible in volunteers. The FDA requires pulse oximeters marketed for medical use to be tested in the SaO₂range 70 - 100%. The shape of the calibration curves is extrapolated at oxygen saturations below these levels. Each manufacturers calibration curve is proprietary.

Numerous studies have confirmed the accuracy of conventional pulse oximeters when compared with co-oximeters in the clinically relevant range of arterial oxygen saturation.^{[13, 25-27]} Accuracy progressively deteriorates below an SaO₂ of 90%. There is currently insufficient published evidence that portable fingertip pulse oximeters are as accurate as standard pulse oximeters in estimating arterial oxygen saturation in a clinical setting, as opposed to the controlled environment of a laboratory. Various unpublished accuracy studies have been performed on individual manufacturers' fingertip pulse oximeters.^[28] They are not peer reviewed and usually recruit healthy adult volunteers under laboratory-controlled hypoxic conditions.

Most fingertip pulse oximeters marketed to consumers for nonmedical use do not undergo stringent testing as laid out by the FDA and other regulatory bodies.^[15] Lipnick et al.^[15] found that only two of six such commonly used devices met ISO and FDA standards. Of these, one was the fingertip pulse oximeter included in this study. All oximeters demonstrated deteriorating accuracy at lower arterial oxygen saturations. Some of the important limitations of the study identified by the authors were the ideal conditions under which the pulse oximeters were tested (motionless hands of healthy volunteers with good perfusion in a laboratory utilising controlled hypoxic gas mixtures). They warned about extrapolating these data to the clinical setting, where multiple factors and patient comorbidities could potentially affect the pulse oximeters' accuracy. The majority of the volunteers had lighter skin tones, and were therefore not representative of a population like that of SA.

A study involving 55 dental patients in Brazil found no statistically significant difference between a portable fingertip oximeter and hospital pulse oximeters.^[29] The non-invasive SpO₂ of each device was measured by simultaneous application of the fingertip probe at six time intervals during the dental procedures. Although this study was conducted in a clinical setting, it included insufficient comparisons for a Bland-Altman analysis, and the Pearson correlation coefficient reported is not an appropriate method for equipment comparison.^[20] An additional 2015 clinical study examined the agreement between a portable fingertip pulse oximeter (Maxtec MD300 C2) and SaO₂ measured by a laboratory blood gas analyser.^[30] Patients in a pulmonary and renal intermediate care unit were recruited by convenience sampling when a 'therapeutically prescribed' arterial blood gas value was required. The SaO₂ was then compared with SpO₂ derived by the test fingertip pulse oximeter, which was applied within 3 minutes of the blood gas sample. The authors noted an unacceptable difference in bias and precision values, especially when the SaO₂ was <93%. There were some significant methodological flaws, however, including the study being underpowered (N=32), and exclusion criteria not consistent with previous studies that highlighted common causes of error in saturation measurement. The pulse oximeter was not always applied simultaneously with arterial sampling, and in 21 of the 32 samples the delay of up to 3 minutes in obtaining the SpO₂ may have resulted in measurement of a different value to that obtained by the arterial blood gas sample.

Our study adds value to the existing literature in that it evaluates the performance of fingertip pulse oximeters in the clinical setting, among patients with varying American Society of Anesthesiologists (ASA) status and more heterogeneous skin tones. Another merit of the study is that data measurements were obtained across a range of perioperative clinical settings (ICU, recovery room, clinic, theatre).

One of the goals of the COMFORT trial is to create a methodology that can be easily and cheaply reproduced in resource-constrained healthcare environments in other countries to test the clinical agreement of various inexpensive, portable fingertip pulse oximeters with the existing conventional pulse oximeters in these institutions. If there is a predominant/homogeneous skin tone in a particular setting, these data may be omitted.

Study limitations

An inherent limitation of this pragmatic study is the skewed data obtained when measuring SpO₂ in the clinical setting. Most samples (91.4%) were in the SpO₂ range >90%, and while the data do not address the questions regarding accuracy at Sp0₂ <80%, they may be a more accurate reflection of SpO₂ encountered in daily clinical practice. A further minor source of bias was that the fingertip pulse oximeter only displayed a two-digit reading for SpO₂ (omitting 100%). As there is no clinically meaningful difference between saturations of 99% and 100%, we elected to include these differences, which did not affect the primary outcome. However, the true value of an accurate pulse oximeter lies in its ability to correctly identify a hypoxaemic patient who would require prompt intervention. Clinicians could infer from this study that fingertip pulse oximeter readings >93% are reassuring while those <93% require further investigation and should prompt treatment if the patient has reversible causes of mild or profound hypoxaemia. Baseline saturations in the context of the patient's underlying respiratory condition should of course be considered when deciding whether treatment is required; the clinician is reminded to 'treat the patient, not the monitor'.

While the study does include a range of skin tones more representative of SA population demographics than previous studies, each subgroup is underpowered to detect a statistically significant difference. The authors noted no clinically significant difference in pulse rates or Sp0₂ values between lighter- and darker-pigmented patients, and both statistical and graphical analysis of the trends in measurement by skin tone grouping showed no difference between the devices. Lack of more patient-specific data (e.g. age, sex, ASA status) makes more detailed analysis of the data impossible. The difference in averaging time of the two devices may have adversely affected the data in situations where saturations were changing rapidly, as might be encountered in a setting such as the recovery room.

 

Conclusions

This pragmatic study demonstrated that a fingertip pulse oximeter was accurate (within 3% SpO₂) in perioperative patients with normal oxygenation (SpO₂ >93%) compared with a bedside pulse oximeter. As in previous studies, accuracy deteriorated with progressive hypoxaemia. A measurement of <93% on the portable device is cause for concern, and should prompt further investigation and management of hypoxia if necessary. Pulse rates measured by the portable devices were of clinically acceptable accuracy. We found that darker skin pigmentation showed no trend to an effect on the accuracy of either measurement. Future studies are required to investigate the agreement between these two devices at lower oxygen saturation levels, and studies should be specifically powered to detect differences in accuracy between varying skin tones.

Declaration. This study was performed in partial completion of the degree MMed (Anaesthesiology) at the University of Cape Town (RNS).

Acknowledgements. The authors wish to acknowledge the staff and patients of Groote Schuur Hospital, the experienced senior researchers in the Department of Anaesthesia and Perioperative Medicine, and the Western Cape Provincial Government Department of Health for their selfless assistance.

Author contributions. RH: study conception, initial design, pilot study statistical analysis, manuscript review, editing and supervision; RNS: study protocol, literature review, data collection, manuscript writing and review.

Funding. This study was supported by departmental funds. The CMS50D portable fingertip pulse oximeter used in the study was purchased by the authors.

Conflicts of interest. None.

 

REFERENCES

1. Merry AF, Eichhorn JH, Wilson IH. Extending the WHO 'Safe Surgery Saves Lives' project through global oximetry. Anaesthesia 2009,64(10).1045-1048. https://doi.org/10.1 lll/j.1365-2044.2009.06104.x        [ Links ]

2. Bettings P, Diedericks J, Fourie P, et al. South African Society of Anaesthesiologists Practice Guidelines. 2012 Revision. S Afr J Anaesth Analg 2013,19(l).Sl-42. https://doi.org/10.1080/22201173.2013.10872899        [ Links ]

3. Mitchell J. Recommendations for standards of monitoring during anaesthesia and recovery. Anaesthesia 2001,56(5).488-488. https://doi.org/10.1046/j.l365-2044.2001.02047-7.x        [ Links ]

4. Merry A, Cooper J, Soyannwo O, Wilson I, Eichhorn J. International Standards for a Safe Practice of Anesthesia 2010. Can J Anaesth 2010,57(11).1027-1034. https://doi.org/10.1007/sl2630-010-9381-6        [ Links ]

5. World Alliance for Patient Safety. WHO Surgical Safety Checklist and Implementation Manual. World Health Organization, 2008. http://www.who.int/patientsafety/safesurgery/ss_checklist/en/ (accessed 21 June 2017).         [ Links ]

6. Haynes AB, Weiser TG, Berry WR, et al. A surgical safety checklist to reduce morbidity and mortality in a global population. Ν Engl J Med 2009,360r491-499. https://doi.org/10.1056/NEJMsa0810119        [ Links ]

7. Food and Drug Administration. Pulse oximeters - premarket notification submissions [510(k)s]; Guidance for industry and Food and Drug Administration staff. 4 March 2013. https://www.fda.gov/RegulatoryInformation/Guidances/ucm341718 (accessed 12 February 2019)        [ Links ]

8. Moller JT, Pedersen T, Rasmussen LS, et al. Randomized evaluation of pulse oximetry in 20,802 patients. I. Design, demography, pulse oximetry failure rate, and overall complication rate. Anesthesiology 1993-,78(3):436-444. https://doi.org/10.1097%2F00000542-199303000-00006        [ Links ]

9. Moller JT, Johannessen NW, Espersen K, et al. Randomized evaluation of pulse oximetry in 20,802 patients. II. Perioperative events and postoperative complications. Anesthesiology 1993,78(3).445-453. https://doi.org/10.1097/00000542-199303000-00007        [ Links ]

10. Sinex J. Pulse oximetry. Principles and limitations. Am J Emerg Med 1999,17(1).59-66. https://doi.org/10.1016/S0735-6757(99)90019-0        [ Links ]

11. Jubran A. Pulse oximetry. Crit Care 2015:19:272. https://doi.org/10.1186/sl3054-015-0984-8        [ Links ]

12. Chitilian H, Kaczka D, Vidal Melo M. Respiratory monitoring. In. Miller R, ed. Miller s Anesthesia 8th ed. Philadelphia. Elsevier Saunders, 2015.1541-1579.         [ Links ]

13. Jensen LA, Onyskiw JE, Prasad NGN. Meta-analysis of arterial oxygen saturation monitoring by pulse oximetry in adults. Heart Lung 1998-,27(6):387-408. https://doi.org/10.1016/S0147-9563(98)90086-3        [ Links ]

14. Bidder P, Feiner J, Severinghaus J. Effects of skin pigmentation on pulse oximeter accuracy at low saturation. Anesthesiology 2005,102(4).715-719. https://doi.org/10.1097/00000542-200504000-00004        [ Links ]

15. Lipnick M, Feiner J, Au Ρ, Bernstein M, Bidder P. The accuracy of 6 inexpensive pulse oximeters not deared by the Food and Drug Administration. The possible global public health implications Anesth Analg 2016-,123(2).338-345. https://doi.org/10.1213/ANE.0000000000001300        [ Links ]

16. Lu MJ, Zhong WH, Liu YX, Miao HZ, Li YC, Ji MH. Sample size for assessing agreement between two methods of measurement by Bland-Altman method. Int J Biostat 2016,12(2). https://doi.org/10.1515/ijb-2015-0039        [ Links ]

17. Fitzpatrick T. Soleil et peau. J Med Esthet 1975,2.33-34.         [ Links ]

18. Fitzpatrick T. The validity and practicality of sun-reactive skin types I through VI. Arch Dermatol 1988-,124(6):869-871. https://doi.org/10.1001/archderm.l988.01670060015008        [ Links ]

19. Sanyal S, Nundy KK. Algorithms for monitoring heart rate and respiratory rate from the video of a user's face. IEEE J Transi Eng Health Med 2018,6(May). https://doi.org/10.1109/jtehm.2018.2818687        [ Links ]

20. Bland J, Altman D. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986,327(8476).307-310. https://doi.org/10.1016/S0140-6736(86)90837-8        [ Links ]

21. Bland JM, Altman DG. Comparing methods of measurement. Why plotting difference against standard method is misleading. Lancet 1995,346(8982).1085-1087. https://doi.org/10.5555/uri.pii.S0140673695917489        [ Links ]

22. Altman DG, Bland JM. Measurement in medicine. The analysis of method comparison studies. J R Stat Soc Ser D Statistician 1983,32(3).307-317. https://doi.org/10.2307/2987937        [ Links ]

23. Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res 1999-,8(2).135-160. https://doi.org/10.1177/096228029900800204        [ Links ]

24. Jubran A. Advances in respiratory monitoring during mechanical ventilation. Chest 1999,116(5).1416-1425. https://doi.org/10.1378/chest.H6.5.1416        [ Links ]

25. Webb R Ralston A RW. Potential errors in pulse oximetry. II. Effects of changes in saturation and signal quality. Anaesthesia 1991,46(3).207-212. https://doi.org/10.111 l/j.l365-2044.1991.tb09411.x        [ Links ]

26. Taylor M, Whitwam J. The accuracy of pulse oximeters. Anaesthesia 2007,43(3).229-232. https://doi.org/10.1111/j.1365-2044.1988.tb05549.x        [ Links ]

27. Nickerson B, Sarkisian C, Tremper Κ. Bias and precision of pulse oximeters and arterial oximeters Chest 1988:93(3).515-517. https://doi.org/10.1378/chest.93.3.515        [ Links ]

28. Colechin E, Bousfield D, Reay C, Sims A. Market review. Pulse oximeters in primary and prehospital care. 2010,(March). Originally available from Centre for Evidence-based Purchasing (CEP), web page decommissioned. Now available from http://openairway.org/colechin-et-al-2010-market-review-pulse-oximeters-in-primary-and-prehospital-care/ (accessed 12 February 2019).         [ Links ]

29. Takeshita WM, Iwaki LCV, Pupim D, Filho L. Evaluation of accuracy of portable fingertip pulse oximeter, as compared to that of a hospital oximeter with digital sensor. Indian J Dent Res 2013·,24(5).542-546. https://doi.org/10.4103/0970-9290.123362        [ Links ]

30. Jones M, Olorvida E, Monger K, et al. How well do inexpensive, portable pulse oximeter values agree with arterial oxygenation saturation in acutely ill patients? Medsurg Nurs 2015,24(6).391-396. https://www.thefreelibrary.com/How_well_do_inexpensive_portable_pulse_oximeter_values_agreewith...-a0439362279 (accessed 12 February 2019).         [ Links ]

 

 

Correspondence:
R Ν Smith
reubensmith@hotmail.co.za

Accepted 13 November 2018