SciELO - Scientific Electronic Library Online

vol.112 número10 índice de autoresíndice de assuntospesquisa de artigos
Home Pagelista alfabética de periódicos  

Serviços Personalizados



Links relacionados

  • Em processo de indexaçãoCitado por Google
  • Em processo de indexaçãoSimilares em Google


SAMJ: South African Medical Journal

versão On-line ISSN 2078-5135
versão impressa ISSN 0256-9574

SAMJ, S. Afr. med. j. vol.112 no.10 Pretoria Out. 2022 



Leveraging routine data in impact evaluation: Understanding data systems in primary healthcare prior to a matched cluster-randomised evaluation of adherence guidelines in South Africa



A N HuberI; S PascoeI; M P FoxII, III, IV; J MurphyV; M PhokojoeVI; M GorgensVII; S RosenVIII, IX; D WilsonX; Y PillayXI; N Fraser-HurtXII

IPhD; Health Economics and Epidemiology Research Office, Department of Internal Medicine, School of Clinical Medicine, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa
IIDSC; Health Economics and Epidemiology Research Office, Department of Internal Medicine, School of Clinical Medicine, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa
IIIDSC; Department of Global Health, Boston University School of Public Health, USA
IVDSC; Department of Epidemiology, Boston University School of Public Health, USA
VMPH; Health Economics and Epidemiology Research Office, Department of Internal Medicine, School of Clinical Medicine, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa
VIDivision of Health Systems and Public Health, Faculty of Medicine and Health Sciences, Stellenbosch University, Cape Town, South Africa
VIIThe World Bank Group,Washington DC, USA
VIIIMPH; Health Economics and Epidemiology Research Office, Department of Internal Medicine, School of Clinical Medicine, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa
IXMPH; Department of Global Health, Boston University School of Public Health, USA
PhD; The World Bank Group,Washington DC, USA
XIPhD; Division of Health Systems and Public Health, Faculty of Medicine and Health Sciences, Stellenbosch University, Cape Town, South Africa
XIIPhD; The World Bank Group,Washington DC, USA





BACKGROUND: An essential part of providing high-quality patient care and a means of efficiently conducting research studies relies upon high-quality routinely collected medical information
OBJECTIVES: To describe the registers, paper records and databases used in a sample of primary healthcare clinics in South Africa (SA) with the view to conduct an impact evaluation using routine data
METHODS: Between October 2015 and December 2015, we collected information on the presence, quality and completeness of registers, clinical stationery and databases at 24 public health facilities in SA. We describe each register and type of clinical stationery we encountered, their primary uses, and the quality of completion. We also mapped the ideal flow of data through a site to better understand how its data collection works
RESULTS: We identified 13 registers (9 standard, 4 non-standard), 5 types of stationery and 4 databases as sources of medical information within a site. Not all clinics used all the standardised registers, and in those that did, registers were kept in various degrees of completeness: a common problem was inconsistent recording of folder numbers. The quality of patient stationery was generally high, with only the chronic patient record being considered of varied quality. The TIER.Net database had high-quality information on key variables, but national identification (ID) number was incompletely captured (42% complete). Very few evaluation sites used electronic data collection systems for conditions other than HIV/AIDS
CONCLUSION: Registers, databases and clinical stationery were not implemented or completed consistently across the 24 evaluation sites. For those considering using routinely collected data for research and evaluation purposes, we would recommend a thorough review of clinic data collection systems for both quality and completeness before considering them to be a reliable data source



For treatment of chronic diseases, including antiretroviral therapy (ART) for HIV/AIDS, to be effective, patients must initiate treatment as early as allowed under prevailing guidelines, remain in care for long periods of time, consistently achieve high levels of adherence to their treatment regimen and, as a result, exhibit stable monitoring test results and/or treatment completion. In the case of HIV/ AIDS, treatment is lifelong and requires consistent, nearly complete adherence to sustain an undetectable viral load.[1] Numerous studies and reviews have indicated that suboptimal retention in care and adherence to ART in South Africa (SA) poses a serious threat to the long-term success of the national HIV response.[2-5] In order to improve patient care and reduce disengagement from care, there is a push for research studies to produce results quickly and affordably.

An effective strategy for efficient and economical research studies is to utilise routinely collected data. Routinely collected data through electronic systems are becoming more widely available, even in resource-limited settings. Where electronic record keeping may not exist, paper-based systems are typically the source for medical

information, often through standardised, government-issued registers and paper records used for patient care, referred to as the patient stationery. Such systems can provide invaluable information for evaluation of health interventions and policy changes if they are well rationalised, filled in completely and contain high-quality data. At the same time, too many registers and duplication of record keeping can overcomplicate data collection systems, frustrate staff responsible for completing those registers and reduce providers' capacity to deliver care for patients.[6-9] In addition, high-quality routine data allow for continuity of care, assessment of treatment compliance and targeting of appropriate interventions when treatment results are suboptimal. Ideally, record-keeping systems would be simple to use, require little time to complete and be kept as up to date as possible.

While having good data at primary healthcare (PHC) clinics is essential for patient care, there is little documentation on how data are collected and stored within health facilities, how much variation there is among public sector health facilities and how good the quality of data that are collected is. We sought to describe the registers, stationery and databases used in a sample of PHC clinics in SA. This evaluation was aimed to leverage existing secondary data while also learning about the routine systems in place.



Between 2015 and 2018, we conducted a study to evaluate the effectiveness of SA's National Department of Health (NDoH)'s National Adherence Guidelines for Chronic Diseases (HIV, TB and non-communicable diseases).[10] The guideline recommends a number of interventions to improve retention in care and adherence to therapy for all chronic conditions, with a focus on HIV care.

The evaluation had a matched cluster randomised design involving 24 clinics in 4 provinces in SA (Gauteng, Limpopo, North West and KwaZulu-Natal). Six clinics within 1 district in each of the 4 provinces were chosen that had >1 000 current ART patients, had no backlog on the electronic HIV patient database TIER.Net (Three Integrated Electronic Registers), and were not National Health Insurance pilot sites, or in any other adherence-related studies or pilots. A TIER. Net backlog was defined as more than 1 day behind in the capture of patient files. Results describing these outcomes have been presented elsewhere.[11-13]

Since the evaluation was based on routine data, it relied on a certain completeness and quality of data from multiple routine systems. Aditionally, the evaluation team wanted to avoid all contact with patients, as any attempt to interact with them could potentially influence patient retention in care. It was therefore important to understand the clinics' data flow and the presence and status of registers, clinical stationery and databases. In order to achieve this, we worked with each of the 24 clinics during the preparation phase of the evaluation, and assessed the diversity and quality of record-keeping systems during October to December 2015. We compiled a list of all registers, clinical stationery and databases that the sites used to track patient care and report on patient volume and outcomes. Based on those assessments, clinic-specific data enhancement plans were made so that the study could proceed. The plans intended to improve the quality of data collection at each site to ensure that adequate routine data would be available for the evaluation to enroll the correct patients and determine intervention outcomes.

We report here on those baseline assessments, to provide a full understanding of the diversity of registers and clinical stationery and variation in the ways that those registers and documents are used. Because our work was focused on four conditions (HIV, tuberculosis (TB), hypertension and diabetes), we focused on registers used to track patients for these conditions and for general clinic operations. We did not review registers and clinical stationery related to pregnant women and children, as they were not eligible for the study. Below we describe the registers and clinical stationery that we encountered, and their primary uses. To identify the diversity of registers, stationery and databases, we noted which were used at each site, and noted any site-specific registers. For site-specific registers, we determined the reasons for developing and using those registers, and documented any gaps where important information was not recorded. Nonstandard registers are those that differ across facilities and have been developed either by non-governmental organisation support partners or by the site staff themselves for the purposes of monitoring and reporting, to address gaps not covered by the standardised stationery. It was not a requirement that the operational manager had approved the register for it to be included in this assessment. We also looked for overlap between the registers to identify areas where they could be streamlined in order to provide more efficient record keeping. We assessed the completeness and the quality of each register, stationery file and database. We then looked for variations in the way each register or stationery file was interpreted and used within sites to better understand how sites used the data they collected. Finally, we mapped the ideal flow of data through a health facility to better understand how data collection works, and summarised the basic flow, documenting the use of paper registers, non-standard registers, electronic databases and clinical stationery.



Care within clinics is generally divided into chronic care (including HIV and TB), acute/minor illness care and preventive care (maternal, child and women's health (MCWH)). Information for HIV, TB, diabetes and hypertension patients was mainly found either in general chronic disease care registers (for diabetes and hypertension) or in disease-specific registers (for HIV and TB). We identified two main sources of information for tracking patients within a site: (i) registers and clinical stationery; and (ii) electronic databases. Below we discuss each of the data collection mediums and the diversity of those data sources.

Registers and clinic stationery

Facilities in SA use various paper registers to track vital information on patients within their clinics. Table 1 shows all the registers we identified at the site visits that were relevant to the four conditions. This list includes both standardised and non-standardised registers. Standardised registers are those developed by the NDoH and disseminated through national, provincial and district departments. These include a headcount register that captures the total number of patients seen on a daily basis, and a PHC comprehensive tick register that allows tracking of all healthcare services in acute, chronic and MCWH consulting rooms. These two registers are the main source for identifying the volume of services provided by the clinics, and are where much of the data on patients with conditions other than HIV and TB are recorded. HIV and TB both had a number of specific registers that tracked and reported on specific aspects of care (e.g. pre-ART v. ART care, and suspected TB v. a TB register for care of TB patients). We also noted that sites rely heavily on the lab specimen book to understand which specimens have been collected. This book records all specimens sent for processing at the National Health Laboratory Service (NHLS), SA's provider of laboratory services for the public sector. Examples of non-standardised registers included (i) HIV counselling session registers; (ii) patient tracing registers for patients who miss visits; and (iii) registers for patients who received targeted interventions such as those in the adherence guidelines (e.g. adherence club registers). The purposes of these registers vary: some are used for individual tracking (ensuring that patients are not lost or receive the care they should), while others are designed for aggregate reporting.

In addition to the registers, standard stationery was placed inside patient file folders to record patient specific information and more detailed visit information. We identified specific stationery for: ( i) HIV patients on treatment; (ii) chronic (non-HIV) patients; and (iii) TB screening and treatment patients. Sites also had non-standard appointment cards that patients were given with reminders for next appointments.


As with the registers and stationery, we found diversity in the electronic databases being used to capture information on patients, as shown in Table 2. The exception to this was the TIER.Net electronic HIV patient database. This system is used by all clinics to compile national and subnational data on the number of HIV patients in care, on HIV treatment, retained over time and with a suppressed HIV viral load. The data in this system are initially recorded on the HIV patient stationery, and then entered by site data capture staff into TIER.Net. The District Health Information System (DHIS) is the routine repository for aggregated health information in SA, but data were only directly entered into the system in about half of the study sites. All other study sites would report aggregated data to the District Department of Health, which were entered into the DHIS system for reporting.

While the NDoH has taken steps to ensure that electronic data are captured on all HIV patients through TIER.Net and the DHIS for the purposes of reporting, very few sites included in the evaluation collected electronic data on patients who were not HIV infected. The exception to this was Ekurhuleni District in Gauteng, where the eHealth database was being piloted in five of the six study sites for tracking names, date of birth, national identification (ID) number and type of client (acute, chronic, MCWH) for all patients in the district. In addition, LABTRAK, also known as TrakCare, is the web-based laboratory information system of the NHLS that enables clinical staff to access lab results for specific patients via smartphone or computer. This system was only used in four evaluation sites in two provinces. All the other sites received lab results via courier-delivered printouts or clinic-based SMS printers.

As Table 1 indicates, we assessed the quality of the TIER.Net and DHIS databases only. The TIER.Net system is easily accessible and has high-quality information on some of the key variables for tracking HIV patient care, such as ART start date, CD4 counts and viral load results. The DHIS was of mixed quality because its quality is dependent upon the quality of the aggregated data fed into the DHIS data collection system.

Gaps in recording information

Table 1 also describes the quality of completion of each system and the usefulness of each system or document in terms of being able to follow patient information. We found that the TB register and TB sputum register and the headcount register were of high quality at nearly all sites. The quality of the PHC comprehensive tick register was varied, with the biggest problem being the fact that the folder numbers were not consistently recorded. This meant that individual records for patients listed in the tick register cannot always be identified within the clinic. Of note, the pre-ART register was of poor quality, as data on HIV patients were largely not collected until a patient began HIV treatment. The non-standard registers were also of varied quality. Encouragingly, the quality of information completion on patient stationery was generally high, with only the chronic patient record being considered of varied quality.

One of the biggest insights from our baseline assessment was that not all clinics used all the standardised registers, and for those that did, they were kept in various degrees of completeness (Table 2). Of note, the headcount register was only used at three sites, while the PHC comprehensive tick register was used in all sites in three of the provinces, but was not used at all in KwaZulu-Natal Province. The HIV counselling and testing register, TB register and TB sputum registers were used in most sites, though only in KwaZulu-Natal and Limpopo provinces were these three registers used in all sites. The remaining standard registers were used sporadically, with the daily clinic register and the pre-ART register only being used in KwaZulu-Natal. As would be expected, there was little pattern of how and where the non-standard registers were used. Counselling session registers and tracing registers were most often used, but no province had all sites using all of the registers. In terms of patient stationery, all sites used the HIV patient record and the TB treatment record, while the chronic patient record was used in all provinces except Limpopo, and the TB screening tool was used at every site in Gauteng, but only in three other sites outside Gauteng.

While the registers described above allow for capturing the majority of information needed to track patients across the HIV and chronic disease care continuum, we identified some important gaps in what was captured. One key piece of data that was incompletely captured was the national ID number. Across all 24 study facilities, about 42% of active patients on TIER.Net had their SA ID number recorded. Of all TIER.Net records across the 24 facilities (105 307), 23% were active and recorded ID number. North West and Gauteng provinces had the lowest completion rate of active patients, with North West facilities recording <25%. This may be due to differing numbers of patients with national IDs at the sites (as non-South Africans would not have an ID). Lack of national ID number poses multiple challenges, such as difficulty in tracking patients transferring to a different clinic, in linking lab results to mobile patients, and cross-referencing mortality with the national death index, among others.

Ideal data flow

Given the numbers of registers, stationery and clinical databases, it is essential to understand how the data are generated and registers are completed. Fig. 1 demonstrates the ideal flow of data through a health facility. The general flow begins when the patient arrives: (s)he checks into reception, is counted in the headcount register and a file is pulled (existing patients) or created (new patients). The patient then moves to the vitals station where weight and blood pressure are measured, and then queues to be seen by a clinician. Acute patient data are recorded in the patient file and the PHC tick register (or similar daily clinic register in KwaZulu-Natal). If applicable, standard stationery (TB screening tool) and registers (HIV counselling and testing, TB suspect/sputum, NHLS specimen log) are completed. Chronic patient data are captured in the patient file and on disease-specific stationery (HIV patient stationery, TB treatment record, chronic patient record) and disease-specific registers (pre-ART, Adherence Club, TB treatment, chronic registers, isoniazid preventive therapy and counselling). Patients then return to reception to schedule another visit. After the visit is completed, all files are returned directly to the file room except for the files of HIV-positive patients, which are taken to a designated data capturer who enters visit information from the ART stationery and paper NHLS results in the electronic TIER.Net database. Since TIER.Net is the last step in the process, the quality of the information in the TIER.Net system is directly dependent on the quality of the recording in the stationery. The ART file is returned to the file room after entry into TIER.Net.



There are multiple advantages to using routine data for health monitoring and evaluations, including non-interference with patient care, faster production of results, capitalising on investments already made in data systems and saving costs through minimal primary data collection. One of the disadvantages is to have less control over data quality and which indicators are measured when using secondary data. We therefore assessed the registers, clinical stationery and databases used to track patients through stages of care and the flow of data through PHC clinics in four provinces in SA. We determined the quality of each of those sources as part of an effort to strengthen data collection in preparation for a trial evaluating the effectiveness of a national adherence strategy being rolled out by SA's NDoH.

One of the key findings has been in identifying the diversity of methods of keeping data on patients within the various clinics. As part of a focus from the NDoH to reduce reporting burden on clinic staff by rationalising clinic registers, two changes were made: (i) the TIER.Net system was introduced to reduce the number of registers and digitise reporting for those in HIV care;[14] and (ii) the PHC comprehensive tick register was introduced to capture PHC services delivered in acute, chronic, and MCWH consulting rooms.[15] While all clinics were using the TIER.Net system to report data on patients in the HIV care system, we found that the HIV counselling and testing register was still in use at a majority of sites, while the pre-ART register was used at a minority of sites. The PHC comprehensive tick register was in use in three provinces, and in its place the daily clinic register was used in five out of six facilities in KwaZulu-Natal. Numerous other methods, such as non-standard registers in personal notebooks, were being used to track key pieces of information on patients. These were completed with varying degrees of accuracy and completeness at the different sites.

The process of engaging with the facilities led to a number of key insights. We learned how the data flows work in the clinics in order for data to go from patient interactions to a completed database. The current system requires duplicating information in the clinical stationery, paper register and/or electronic database. Though the registers have been rationalised at a national level, clinicians, counsellors and data capturers still document information in multiple places, or have developed systems to meet their own record needs. Though the systems are not created for evaluation purposes, our findings are in line with conclusions reached by other researchers that the use of routine data sources for evaluation purposes is both inexpensive and convenient.[16,17]

Though completing unnecessary registers can be a burden on clinic staff, the presence of non-standard registers provides insight into gaps where recording information in a register is useful to the clinic staff or management in a way that was overlooked during that rationalisation process. We found that counsellors had created registers to document information on who had received counselling, and that outreach workers were using non-standard registers to record the tracing of clinic patients. This information could be used to inform future production of standard registers for these purposes.

As was the case for registers and databases, we found that the standard stationery for HIV and TB care was present and complete in the patient file at the majority of sites, but there was a lack of standard stationery for other chronic conditions. The chronic patient record is standard stationery that has been created for hypertension, diabetes and other chronic conditions. This stationery was in use at fewer facilities, and had varied quality, resulting in the potential for inconsistent recording of patient information and a lack of continuity in patient care.

Limitations exist for this description of registers. The parent study excluded pregnant women and children, and for that reason this article does not include records related to maternal or child health. Furthermore, we only collected information at a single point in time. After the study period, TIER.Net introduced a TB module, and the Health Patient Registration System (HPRS) expanded coverage. These, among other changes, are not included in this evaluation. Though there are limitations, we believe the study accurately represents the chronic and acute patient information tracking systems at the 24 PHC clinics across four provinces involved in this study.

Registers, databases and clinical stationery were not implemented consistently across the 24 clinics reviewed. For those considering using routinely collected data for research purposes, we would recommend a thorough review of clinic data collection systems for both quality and completeness, and potentially considering a specific period and support for data enhancement before considering them as reliable data sources for a research study.

Declaration. None.

Acknowledgements. None.

Author contributions. NF, SP, MP, MG, SR and MPF all contributed to developing the protocol. AH, JM, SW, MN and YP all contributed substantive changes to the protocol. ANH and MPF drafted the manuscript. All authors were involved in editing the final manuscript.

Funding. This work was supported by World Bank trust funds from several governments to finance the impact evaluation, data collection, cleaning and analysis components, and from the government of SA domestic health financing for the implementation of the adherence interventions that were evaluated in this impact evaluation. The World Bank trust fund donors supporting this evaluation had no role in study design, data collection, analysis or interpretation, or writing of the manuscript. The corresponding author had full access to all data in the study and had final responsibility for the decision to submit for publication. Part of AH's time was supported by the National Institutes of Health Fogarty International Center (#D43TW009340).

Conflicts of interest. None.



1. Cohen MS, Chen YQ, McCauley M, et al. Antiretroviral therapy for the prevention of HIV-1 transmission. N Engl J Med 2016;375(9):830-839.        [ Links ]

2. Fox MP, Rosen S. Retention of adult patients on antiretroviral therapy in low- and middle-income countries: Systematic review and meta-analysis 2008-2013. J Acquir Immune Defic Syndr 2015;69(1):98-108.        [ Links ]

3. Fox MP, Rosen S. Patient retention in antiretroviral therapy programs up to three years on treatment in sub-Saharan Africa, 2007-2009: Systematic review. Trop Med Int Health 2010;15(Suppl 1):S1-S15.        [ Links ]

4. Mugglin C, Estill J, Wandeler G, et al. Loss to programme between HIV diagnosis and initiation of antiretroviral therapy in sub-Saharan Africa: Systematic review and meta-analysis. Trop Med Int Health 2012;17(12):1509-1520.        [ Links ]

5. Kranzer K, Govindasamy D, Ford N, Johnston V, Lawn SD. Quantifying and addressing losses along the continuum of care for people living with HIV infection in sub-Saharan Africa: A systematic review. J Int AIDS Soc 2012;15(2):17383.        [ Links ]

6. Shihundla RC, Lebese RT, Maputle MS. Effects of increased nurses' workload on quality documentation of patient information at selected primary health care facilities in Vhembe District, Limpopo Province. Curationis 2016;39(1):1545.        [ Links ]

7. Schuster RC, McMahon DE, Young SL. A comprehensive review of the barriers and promoters health workers experience in delivering prevention of vertical transmission of HIV services in sub-Saharan Africa. AIDS Care 2016;28(6):778-794.        [ Links ]

8. Olaleye AO, Tsibolane Y Van-Turha L, et al. I don't know what I am doing because I am doing everything: Perceptions and experiences of nurses about HIV counselling and testing among children in Free State Province, South Africa. AIDS Care 2016;28(Suppl 2):S21-S28.        [ Links ]

9. Myburgh H, Murphy JP, van Huyssteen M, et al. Implementation of an electronic monitoring and evaluation system for the antiretroviral treatment programme in the Cape Winelands district, South Africa: A qualitative evaluation. PloS ONE 2015;10(5):e0127223.        [ Links ]

10. Fox MP, Pascoe SJ, Huber AN, et al. Assessing the impact of the National Department of Health's National Adherence Guidelines for Chronic Diseases in South Africa using routinely collected data: A cluster-randomised evaluation. BMJ Open 2018;8(1):e019680.        [ Links ]

11. Fox MP, Pascoe SJS, Huber AN, et al. Effectiveness of interventions for unstable patients on antiretroviral therapy in South Africa: Results of a cluster-randomised evaluation. Trop Med Int Health 2018;23(12):1314-1325.        [ Links ]

12. Fox MP, Pascoe S, Huber AN, et al. Adherence clubs and decentralised medication delivery to support patient retention and sustained viral suppression in care: Results from a cluster-randomised evaluation of differentiated ART delivery models in South Africa. PLoS Med 2019;16(7):e1002874.        [ Links ]

13. Pascoe SJ, Fox MP, Huber AN, et al. Differentiated HIV care in South Africa: The effect of fast-track treatment initiation counselling on ART initiation and viral suppression as partial results of an impact evaluation on the impact of a package of services to improve HIV treatment adherence. J Int AIDS Soc 2019;22(11):e25409.        [ Links ]

14. Osler M, Hilderbrand K, Hennessey C, et al. A three-tier framework for monitoring antiretroviral therapy in high HIV burden settings. J Int AIDS Soc 2014;17(1):18908.        [ Links ]

15. Mckenzie A, Schneider H, Schaay N, Scott V, Sanders D. Case study from South Africa PRIMARY HEALTH CARE SYSTEMS (PRIMASYS). Geneva: WHO, 2017. (accessed 21 January 2019).         [ Links ]

16. Kane R, Wellings K, Free C, Goodrich J. Uses of routine data sets in the evaluation of health promotion interventions: Opportunities and limitations. Health Educ 2000;100(1):33-41.        [ Links ]

17. Harries AD, Zachariah R, Maher D. The power of data: Using routinely collected data to improve public health programmes and patient outcomes in low- and middle-income countries. Trop Med Int Health 2013;18(9):1154-1156.        [ Links ]



A Huber

Accepted 13 June 2022

Creative Commons License Todo o conteúdo deste periódico, exceto onde está identificado, está licenciado sob uma Licença Creative Commons