RESEARCH

 

The accuracy of a point-of-care test among different operators using the QBC Autoread Plus Analyzer for the measurement of a basic full blood count

 

 

W Laney^I; R Mawelele^II; S Omar^III

^IMB BCh; Chris Hani Baragwanath Academic Hospital and Intensive Care Unit, Faculty of Health Sciences, University of the Witwatersrand., Johannesburg, South Africa
^IIBTech Clin Tech; Intensive Care Unit, Chris Hani Baragwanath Academic Hospital, Johannesburg, South Africa
^IIIMB ChB, FCPath (SA) Chem, DA (SA), Cert Critical Care (SA); Intensive Care Unit, Chris Hani Baragwanath Academic Hospital, and School of Clinical Medicine, Faculty of Health Sciences., University of the Witwatersrand, Johannesburg, South Africa

Correspondence

 

 

---

ABSTRACT

BACKGROUND. Major issues around the implementation of point-of-care testing (POCT) include: user type, regulatory control, ongoing quality monitoring and limited guideline adherence.
OBJECTIVES. To determine if there are significant differences in the results of a POC full blood count test between different levels of healthcare and non-medical 'lay users compared with laboratory users (technologists/pathologists).
METHODS. This article retrospectively reviews the technical evaluation database of the intensive care unit (ICU), Chris Hani Baragwanath Academic Hospital, Johannesburg, South Africa. We searched for samples analysed by doctors, clerks and laboratory personnel. A minimum number of 60 comparisons were required. Bland-Altman plots, Spearman's correlation and Passing-Bablok fit were used to analyse the dataset.
RESULTS. There were 72 comparisons for haematocrit (Hct), 98 for white cell count (WCC) and 137 for platelets (Pit) between the clerk and laboratory personnel. The correlations were 0.91, 0.96 and 0.92, respectively. All were statistically significant. Using the Bland-Altman method, there was good agreement between results of the clerk and those of the laboratory staff, with a mean bias of 0.5% (Hct), 0.1 χ 10⁹/L (WCC) and 10 χ 10⁹/L (Pit). An insufficient number of tests were performed by medical doctors for statistical comparison.
CONCLUSIONS. With appropriate training, non-medical, non-healthcare lay users are able to perform a moderately complex POCT with similar accuracy as trained laboratory professionals. The focus should shift to equipment and quality management processes rather than the medical/technical qualification of the user.

---

 

 

Point-of-care testing (POCT) is defined by the College of American Pathologists as tests designed to be used at or near the site where the patient is located. The tests do not require permanent dedicated space and are performed outside the facilities of clinical laboratories. When POCT is used as a clinical adjunct, results can be obtained rapidly, clinical decisions may be expedited and the clinical process can be enhanced. Certain clinical settings, e.g. the intensive care unit (ICU) and the emergency department (ED), depend on timely action for improved outcomes.^[1,2]

Although POCT equipment is readily available, its efficacy is contingent on it being an extension of the clinical process. It is therefore intuitive that the test should be performed by the clinical team. There is, however, debate and regulation around the complexity of specific tests, level of expertise and qualifications required, e.g. a laboratory-trained professional, pathologist, technologist or phlebotomy technician.^[3]

Test devices are regulated internationally and in South Africa (SA). The Clinical Laboratory Improvement Amendments (CLIA) of 1988 are the US federal regulatory standards that apply to all clinical laboratory testing in that country.^[3] According to CLIA, there are different personnel requirements for different complexities of testing, which may include laboratory directors, clinical consultants and technical consultants.

In SA, the SA Health Products Regulatory Authority (SAHPRA) has gazetted several acts, regulations and notices, including regulation of medical devices and in vitro diagnostics that cover POCT.^[4]

When managed by the SA National Accreditation System (SANAS), laboratories using POCT have quality assurance based on the International Organization for Standardization (ISO, nos 15189 and 22870).^[5] However, the increased uptake of POCT has resulted in new entrants in pathology diagnostics: doctors, hospitals and other non-laboratory users, who are unfamiliar with the concepts of quality management.

The two key components of POCT systems are interrelated and can be simplified into users and system managers. Since the process of the management of POCT, including maintenance and quality, is well described, we assessed different user levels. Using a moderately complex full blood count (FBC) POCT (QBC Autoread Plus Analyzer) (QBC POC), we compared the performance of different users after a standardised 4-hour training session using manufacturer's guidelines.

The aim was to determine if there are significant discrepancies in results of QBC POC between different levels of healthcare workers and non-medical lay people when compared with technologists or pathologists.

 

Methods

Study design

We performed a retrospective review of the technical evaluation database of the ICU, Chris Hani Baragwanath Academic Hospital, Johannesburg, SA. We obtained a convenience series of measurements from the database based on a POC analyser that was evaluated by multiple users during 2016.

Data collection

All blood samples from the ICU that had an FBC analysed during a technical evaluation of the QBC POC were included. The technical evaluation took place from 22 July -1 September 2016. The evaluation process included scientific, technical and practical training for all users. We included samples that were analysed by at least 3 users: doctor, clerk (2 ward clerk measurements were combined) and either a technologist or pathologist - used as the reference measurement.

QBC Autoread Plus Analyzer point-of-care measurement

During the evaluation, ICU staff underwent 4 hours of training (based on manufacturer's instructions) on the use of the POC analyser. ICU staff included a technologist, pathologist, doctor and 2 clerks. The methodology was based on electro-optical linear measurements of the discrete layers of packed blood cells in a microhaematocrit-type tube after high-speed centrifugation. Once blood was collected in a tri-potassium-ethylenediaminetetra-acetic acid (EDTA) tube, it required analysis within 90 minutes at room temperature. The parameters analysed were haematocrit (Hct), white cell count (WCC) and platelets (Pit). A calibration was performed prior to each user performing the measurements.

Sample size

Adequacy of sample size was based on the key research question to be answered, i.e. if there was agreement between non-laboratory and laboratory staff analysis using the Bland-Altman method. This method requires at least 60 comparisons for statistical accuracy^[6]

Statistical analysis

Spearman's correlation co-efficient was used, as the data were not normally distributed. We used Passing-Bablok fit to determine the significance of the correlation. The confidence intervals (CIs) for the χ intercept and the slope were calculated with the bootstrap method.^[7] The Bland-Altman method was used to assess for bias between users. The mean difference of the plot provided an estimate of the bias with 95% CIs. Data analysis was performed using Statistica version 13.3 (StatSoft, USA). The 5% significance level was used

Outcomes

The main outcome was a comparison of the FBC measurement performed by the doctor and clerk with that of either the technologist or pathologist (laboratory personnel as reference) on the QBC POC instrument.

Ethical approval

Ethical approval to conduct the study was obtained from the Human Research Ethics Committee, University of the Witwatersrand (ref. no. M180762), as well as from the Hospital Research Committee and management. Informed consent was waived by the Ethics Committee owing to the retrospective nature of the study.

 

Results

The minimum number of 60 comparisons between clerk and technologist or pathologist was found for Hct and Pit. For the WCC, 98 comparisons between clerk and technologist were available, but only 56 between technologist and pathologist. We included the latter despite not meeting the minimum of 60. As none of the comparative measurements performed by the doctor reached the minimum requirement of 60, these were excluded for the comparison using the Bland-Altman method.

The range of values for Hct, WCC and Pit that were tested are described in Table 1. This provides comparative median values between clerk and laboratory personnel and between 2 laboratory personnel considered as the reference (technologist and pathologist) for each of the 3 components of the FBC studied.

 

 

Correlation studies

The correlation between clerk and laboratory personnel (reference) using regression is provided in Table 2. Visualisation of the regression curves shows a linear relationship. All correlations were significant using Passing-Bablok fit, indicating the absence of both a constant and proportional difference between the clerk and laboratory personnel (Figs 1- 3). All correlations were >0.9.

 

 

 

 

 

 

Bland-Altman method

Using the Bland-Altman method, we calculated the mean bias of the measurements performed by the 2 clerks against that of the laboratory personnel. We then also calculated the mean bias of the measurements performed by 1 of the laboratory staff against the other (technologist v. pathologist) (Table 3 and Figs 4 - 9).

 

 

 

 

 

 

 

 

 

 

 

 

Discussion

Our main finding was that, given a short 4-hour training session on a moderately complex POC instrument, a non-medical, non-laboratory lay person was able to operate this diagnostic instrument with results similar to those of professionally trained laboratory personnel.

Overall, we found a significant linear correlation between the 2 clerks performing the FBC (Hct, WCC, Pit) and the laboratory staff. Furthermore, the bias of the clerks' measurements compared

with that of the laboratory personnel was similar to that between the 2 laboratory users. We tested a wide range of measurements in a group of critically ill patients, making the comparison relevant for a variety of clinical settings and disease processes. The QBC POC system has previously been technically evaluated against a laboratory haematology analyser, the Sysmex system (Sysmex, Germany).^[8]

While we have not found any published data on the direct comparison of different types of users in the critical care setting, there are some data comparing POCT to laboratory reference values.

With regard to Hct, the Siemens RAPIDPoint (Siemens, Germany) POC system demonstrated a bias of -1.1 compared with the reference value.^[9] In this study, nurses who were trained on the rapid point system performed the test in critically ill patients. Another POC instrument, the haemocytometer, was evaluated in an intensive care setting.^[10] Clinical staff were trained to use this instrument and observed the bias of Hct against the reference of 0.51. A study comparing the Hemoscreen (Pixcell Medical, Israel) POC instrument in a research setting, using a combination of flow cytometry and digital imaging, included testing for user friendliness by 6 lay people. However, these users did not perform the testing for the main comparison. All cell counts including Hct and Hb (excluding differential counts) had correlations >0.92.^[1l] A recent study in critically ill patients comparing a non-invasive POC pulse co-oximetry Hb/Hct to the reference method found a mean bias of 1.64.^[12] Overall, our study bias of -0.5 for Hct (clerk v. laboratory personnel) compares favourably with other POC comparisons in the literature.

Relevant data on WCC comparisons are sparse. The HemoCue system (HemoCue, SA), based on microscopical image detection (photomicroscope), is a POC system for WCC. Evaluation outside of a critical care setting revealed a difference of < 10% from the reference for 96% of the samples.^[13] Our study bias for WCC (clerk v. laboratory personnel) was +0.1. This translates into a <1% difference for a WCC at the threshold value of 12 χ 10VL, indicating comparative results between clerk and laboratory personnel.

Pit were also evaluated by the previously mentioned study using the haemocytometer.^[10] With this POC method, the mean bias was -10 χ 10⁹/L, which is similar to that in our study, where the Pit bias (clerk v. laboratory personnel) was 10 χ 10⁹/L.

Our findings suggest that POCT can be adequately performed by lay non-medical, non-healthcare users with appropriate and relatively brief training, even when the tests are moderately complex. These would traditionally be performed by trained laboratory personnel.

A recent market report indicated that POCT in SA is one of the most promising growth areas in the pharmaceutical industry owing to a growing demand for efficiency and accuracy. This has led to the increased use and emergence of the non-laboratory consumer of in vitro diagnostics.^[14] Given that it is relatively quick and simple to train lay people to accurately run a POCT, our focus should shift towards improving process and quality management of this expanding technology

Study strengths

Our study is the first to compare different levels of POCT users in the critical care setting. Moreover, instead of a simple POC analyser, we used a moderately complex analyser, making lhe results more robust.

Study limitations

We had to combine the results of 2 clerks to achieve the minimum number of comparisons required. It is obvious that this would result in greater variation, contributing lo the comparison in the former case. However, this allowed inclusion of a sufficient number of comparisons for evaluation of the 2 user groups. The effect of the combination of 2 clerks can especially be noted in the wider CIs of the bias of the Pit count. This is not surprising, as the coefficient of variation of the Pit count on the QBC POC analyser is the greatest (12.5%) of lhe 3 tests compared.^[14]

 

Conclusions

When given appropriate training, non-medical, non-healthcare lay users are able to perform a moderately complex diagnostic test on a POC inslrument wilh similar accuracy lo a laboralory- trained professional. More focus should be placed on equipment and quality management processes rather than on the medical/technical qualification of the user.

Declaration. None.

Acknowledgements. Data sharing is applicable to this article, as datasets from the ICU at Chris Hani Baragwanath Academic Hospital, Johannesburg, were generated or analysed during the current study

Author contributions. WL: assisted with the report design, literature review and analysis, article drafting, final approval and submission of article; SO: designed the study, performed the statistical analysis and interpretation, helped to draft and review the article, approved the final manuscript, and obtained hospital and ethics permission; RM: helped with the study design, data collection and review of the article.

Funding. None.

Conflicts of interest. None.

 

References

1. Stevens W, Gems N, Ford N. Scott LF. Feasibility of HIV point-of-carc tests for resource-limited settings: Challenges and solutions. BMC Med 2014:8:12. https://doi.org/10.1186/s12916-014-0173-7        [ Links ]

2. Price CP. Point of care testing. BMJ 2001:22(7297):1285-1288. https://doi.org/10.1136/bmi.322.7297.1285        [ Links ]

3. Centers for Disease Control and Prevention. Clinical laboratory improvement amendments, testing complexities. https://wwwn.cdc.gov/CLIA/Resources/TestComplexilies.aspx (accessed 7 February 2018).         [ Links ]

4. South African Health Products Regulatory Authority. https://www.sahpra.org.za/Publications (accessed 20 December 2018).         [ Links ]

5. Point-of-care testing (POCT) - requirements tor quality and competence, https://www.iso.org/obp/ui/#iso:std:iso:22870:ed-2:v1:en (accessed 7 February 2018).         [ Links ]

6. Li H, Han D, Hegener MA, Pauletti GM, Steckl Af. Flow reproducibility of whole blood and other bodily fluids in simplified no reaction lateral flow T assay devices. Biomicrofluidics 2017:11(2):024116. https://doi.org/10.1063/L4979815        [ Links ]

7. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986:1(8476):307-310.         [ Links ]

8. Erhabor OI, Richardson G, Mohammed I, et al. Evaluation of the QBC Star centrifugal three pari differential haematology system. Br J Biomed Sei 2013:70(2):67-74.         [ Links ]

9. Allardet-Servent J, Lebsir M, Dubroca C et al. Point-ot-care versus central laboratory measurements of hemoglobin, hematocrit, glucose, bicarbonate and electrolytes: A prospective observational study in critically ill patients. PLOS ONE 2017:12(1):e0169593. https://doi.org/10.1371/journal.pone.0169593        [ Links ]

10. Despotis GJ, Alsoufiev A, Hogue CW, et al. Evaluation of complete blood count results from a new, on-site hemocytometer compared with a laboratory-based hemocytometer. Crit Care Med 1996:24(7):1163-1167.         [ Links ]

11. Ben Yosef Y, Marom B, Hirshberg G, D'Souza C, Larsson A, Bransky A. The HenioScreen, a novel haematology analyser for the point of care. J Clin Pathol 2016:69(8):720-725. https://doi.org/10.1136/iclinpath-2015-203484        [ Links ]

12. Murphy SM, Omar S. The clinical utility of noninvasive pulse co-oximetry hemoglobin measurements in dark-skinned critically ill patients. Anesth Analg 2018:126(5):1519-1526. https://doi.org/10.1213/ANE.0000000000002721        [ Links ]

13. Osei-Bimpong A, |ury C, McLean R, Lewis SM. Point-of-carc method for total white cell count: An evaluation of the HemoCue WBC device. Int | Lab Hematol 2009:31(6):657-664. https://doi.org/10.1111/)1751553X.2008.01093.x        [ Links ]

14. South Africa Point of Care Testing (POCT) Market Report 2016 . QY Research. https://www.qyresearchreports.com/report/south-africa-point-ot-care-teslingpoct-market-report-2016.him (accessed 20 December 2018).         [ Links ]

 

 

Correspondence:
S Omar
shahedicu@gmail.com

Accepted 13 June 2019