RESEARCH

 

The other side of surveillance: Monitoring, application, and integration of tuberculosis data to guide and evaluate programme activities in South Africa

 

 

L J Podewils^I; L Bronner Murrison^{II, III}; C Bristow^{IV, V}; N Bantubani^{VI, VII}; L D Mametja^VIII

^IMS, PhD; Division of Tuberculosis Elimination, Centers for Disease Control and Prevention, Atlanta, Georgia, USA
^IIMPH, PhD; Division of Tuberculosis Elimination, Centers for Disease Control and Prevention, Atlanta, Georgia, USA
^IIIMPH, PhD; Global AIDS Program, Centers for Disease Control and Prevention, Pretoria, South Africa
^IVMSc, MPH, PhD; Global AIDS Program, Centers for Disease Control and Prevention, Pretoria, South Africa
^VMSc, MPH, PhD; Tuberculosis Research Unit, South African Medical Research Council, Durban, South Africa
^VIMPH; Global AIDS Program, Centers for Disease Control and Prevention, Pretoria, South Africa
^VIIMPH; Tuberculosis Research Unit, South African Medical Research Council, Durban, South Africa
^VIIIMPH; Department of Tuberculosis Control, National Department of Health, Pretoria, South Africa

Correspondence

 

 

---

ABSTRACT

BACKGROUND: The importance of using surveillance data to monitor and evaluate programme activities has been emphasised in international policies for tuberculosis (TB) control
OBJECTIVES: A survey was conducted to assess the use of TB surveillance data to monitor and guide TB programme activities in South Africa (SA
METHODS: As part of an evaluation of the SA national TB surveillance system, semi-structured interviews were conducted among TB staff at health facilities and offices in three provinces. At each site, all persons involved with TB care, management and surveillance were invited to participate
RESULTS: At least one person (range 1 - 4) was interviewed at 47/54 health facilities (87.0%), 11/13 subdistrict and district TB offices (84.6%), 2/3 provincial TB offices (66.7%), and at the national level (1/1, 100.0%). Of 119 TB staff, 64.7% recognised the purpose of TB surveillance as guiding programme planning, implementation and evaluation. However, only 16.0% reported using data to measure disease burden, 8.4% to monitor trends, and 9.2% to inform resource allocation. The majority reported using TB management tools provided by the national programme, but 44.5% also described using additional tools. Personnel mentioned the need for dedicated surveillance staff, training on recording and reporting, improved computer access, and methods to apply information from surveillance data to the programme
CONCLUSIONS: The majority of TB staff understood the purpose of surveillance but did not routinely use data to guide programme planning, implementation and evaluation. Training and supporting TB staff to utilise surveillance data will help improve the TB surveillance system

---

 

 

Core objectives of public health surveillance systems include informing and guiding public health action and programme activities.^[1] However, information is often recorded and reported without a comprehensive understanding of the importance and utility of accurate and reliable data.^[2] Consequently the data generated and reported may not represent the true burden of disease, which may lead to inability to appropriately monitor and evaluate public health programmes. Ultimately, the result may be suboptimal allocation and distribution of resources and failure to achieve public health disease targets. It is critical that all persons involved with management of patients and reporting and recording have a solid understanding of the purpose of surveillance and the importance of complete and accurate recording. Furthermore, the information derived from surveillance systems needs to be accessible and disseminated in a manner in which it can readily be applied to plan and implement programme activities.

As part of a systematic evaluation of the tuberculosis (TB) surveillance system in South Africa (SA),^[3] semi-structured interviews were conducted among healthcare workers and TB programme staff to determine knowledge, attitudes, and practices and the extent to which information from the system was used by TB facilities and the TB programme.

 

Methods

Study population

In brief, a retrospective evaluation of the SA TB surveillance system was conducted in three of the nine provinces of SA, randomly selected based on tertile of cure rate (low, medium, high)^[3] (and national data by province, 2008, provided by the National Tubercuclosis Programme (NTP) on 20 August 2009). The initial analysis aimed to assess the completeness and reliability of data on TB patients from multiple sources at the health facility, subdistrict, district and provincial levels and in the national TB programme offices. As part of the evaluation, healthcare personnel involved with TB care, management and surveillance were interviewed using a semi-structured questionnaire to assess knowledge of and attitudes towards the TB surveillance system, and practices pertaining to recording, reporting, monitoring, dissemination and use of TB surveillance data.

TB surveillance data structure

At health facilities, information on persons thought to have TB disease is captured on a paper TB Suspect Register. Information on patients confirmed to have TB disease based on a positive sputum smear for the presence of acid-fast bacilli is recorded on a paper TB Register. A TB Blue Card is established for each TB patient, which serves as the patient's primary TB medical file. Patient information on the TB Register is entered into the Electronic Tuberculosis Register (ETR) at the subdistrict level. ETR records are merged across subdistricts at the district TB office, and across districts at the provincial TB office. The NTP receives databases from all nine provinces, and uses the data to generate national reports and report information to the World Health Organization (WHO).

Data collection

Each participant was asked to provide information on their current job position and duties, education, and TB training and experience.

Participants were asked to describe the purpose of TB surveillance; responses were recorded verbatim. The participants were subsequently asked to provide a 'yes' or 'no' answer as to whether surveillance information was used for each of the following: managing TB patients, determining TB burden, resource planning, or reporting data. Respondents were asked to rate ease of use and usefulness of each surveillance tool (TB Suspect Register, TB Blue Card, TB Register, ETR) on a four-point Likert scale. Health facility personnel were asked about their experience with the paper tools (TB Suspect Register, TB Blue Card and TB Register); TB programme staff at the subdistrict, district, provincial and national levels were asked about the ETR.

Additionally, participants were asked to provide their opinion as to whether each tool contained adequate information for TB management. Information was also collected to quantify the number of persons responsible for reporting and recording TB data, modes of communicating data between facilities and programme levels, and how feedback on programme performance was communicated after data were reported to higher administrative levels. At the conclusion of the interview, participants were asked to provide general comments on the TB surveillance system and suggestions for improving TB surveillance data.

Analysis

Descriptive statistics were used to summarise the educational background, job titles, and TB-related job duties and training of the study population. Frequencies and percentages were calculated for responses to each question for the total sample and for each subgroup of respondents (health facility staff, subdistrict staff, district staff, and provincial and national staff).

Qualitative responses on the purpose of surveillance were categorised according to the elements contained in the US Centers for Disease Control and Prevention (CDC) definition of the purpose of public health surveillance:^[1] (i) for immediate public health action; (ii) to establish/monitor the burden of disease; (iii) to manage patients; (iv) to evaluate the programme; (v) to plan resources; and (vi) for research. Responses that did not fit any of the defined elements were organised into separate categories.

Ethical review

The study was reviewed and approved by the institutional review boards of the CDC and the South African Medical Research Council. All participants provided written informed consent prior to participation.

 

Results

At least one person (range 1 - 4) was interviewed at 47/54 health facilities (87.0%), 11/13 subdistrict and district offices (84.6%) and 2/3 provincial offices (66.7%), and at the national level (1/1, 100%). The total study population included 119 TB staff: 91 health facility staff, 10 subdistrict staff, 12 district staff, 4 provincial staff and 2 national surveillance staff. The number of questionnaires completed at the health facility level was similar across provinces, with 36 from Gauteng, 28 from KwaZulu-Natal and 27 from Mpumalanga.

Respondent characteristics

The majority of health facility staff were nurses (56.0%), had a college or university degree (77.5%), and had been working in TB for more than 4 years (44.0%) (Table 1). Over half of the health facility personnel had job-related duties involving reporting and recording on the TB Register (56.0%) and managing TB patients (53.9%). The majority of health facility staff had been trained in TB reporting and recording (80.2%) and TB management (60.4%).

Personnel interviewed at the subdistrict level included TB managers (40.0%), other co-ordinators/managers (30.0%), and data capturers (persons who abstract and enter data, 30.0%) (Table 1). Staff in the district and provincial/national offices mostly held management positions (75.0% and 83.3%, respectively). The majority of subdistrict, district and provincial/national level staff had university education (60.0%, 50.0% and 83.3%, respectively). Subdistrict and district level staff had less experience working in TB than provincial/national level employees. TB-related job duties among subdistrict and district staff included monitoring data, supervising, and conducting site visits (60.0% and 25.0%, respectively), and co-ordinating programme activities (40.0% and 50.0%, respectively). Provincial/national staff were primarily involved with data entry, merging and management (83.3%) and monitoring data, supervising, and conducting site visits (50.0%). TB staff had received training in TB reporting and recording, TB management, TB/HIV care, infection control and multidrug-resistant TB; however, fewer than one-third of the subdistrict, district and provincial/national level staff had received training on DOTS (directly observed therapy, short course) (Table 2).

Defining surveillance

The majority of health facility and TB programme staff indicated that the purpose of TB surveillance was to guide planning, implementation and evaluation of the programme (64.7%). Fewer than one-fifth of respondents identified surveillance data as a platform for measuring disease burden or identifying high-risk areas or populations (16.0%) or informing resource allocation (9.2%) (Table 2). Additionally, 19.3% of staff stated that the purpose of TB surveillance is to prevent or cure TB.

TB surveillance tools: utilisation, acceptability and ease of use

Almost all health facility personnel reported using TB Blue Cards (94.5%) and the TB Register (96.7%). Most health facility staff found the TB Blue Card and TB Register easy to use (97.7% and 89.7%, respectively), and believed that these tools contained adequate information for TB management (88.4% and 89.7%, respectively). Nearly half of the respondents described using other tools for TB management (46.2%), including TB diaries with key patient information (33.3%), other statistical spreadsheets or databases (16.7%), and log books to track daily appointments (14.9%). All subdistrict respondents reported that the health facilities were using the TB Register, and most believed that the TB Register provided adequate information for TB management (80.0%).

The majority of subdistrict, district, and provincial/national TB programme staff stated that they were using the ETR in all areas under their supervision (90.0%, 66.7% and 100.0%, respectively) and that they found it easy to use. TB personnel used the ETR to monitor programme performance (100.0%, 66.7% and 66.7%, respectively), determine TB burden (66.7%, 33.3% and 33.3%, respectively) and report to the next programme level as required by the department of health (33.3%, 50.0% and 16.7%, respectively). District and provincial/national level staff stated that the ETR was useful for understanding TB programme management, but 22.2% found the data fields in the ETR to be less than adequate. Approximately one-third of subdistrict staff (30.0%) and nearly half of the district and provincial/national TB staff (41.7% and 50.0%, respectively) reported using other tools to monitor the TB programme. These tools included a follow-up register for tracing and identifying patients who had missed visits or defaulted, a TB/HIV register for co-infected patients, the District Health Information System, and a register for patients with multidrug-resistant TB.

Overall, 61.5% of respondents stated that they had a method in place to verify the completeness of TB surveillance data and 51.8% had a method to monitor data accuracy, but the proportion of respondents reporting these practices varied across different levels of TB management (range 60.2 - 83.3% for completeness and 50.0 -70.0% for accuracy). Less than one-third of the respondents (28.4%) stated that there were defined deadlines for reporting data to the next level, and the majority (81.9%) reported making updates to information after it was sent to the next level of management.

Information dissemination and programme application

The majority of health facilities used the TB Register to summarise TB patient information (80.7%). The subdistrict, district, and provincial/ national level TB staff used the ETR to run reports summarising data on key TB indicators.

Most health facility, subdistrict, district, and provincial/national level TB staff received feedback on their respective level's performance from the TB manager or director at the next level of management. Feedback on programme performance was most commonly provided at subdistrict, district, provincial and national TB meetings within 1 - 3 months after data were sent to the next administrative level (77.5%). The majority (95.5%) of personnel found the feedback helpful.

Attitudes and suggestions for improvement

When asked to provide general comments on the TB surveillance system, over a quarter (25.2%) of all respondents mentioned the lack of staff and the need for dedicated TB personnel, with 10.9% specifying a need for data capturers (Table 3). Respondents also expressed a need for additional training on TB management and reporting and recording TB surveillance data (20.2%), improved computer access or systems (13.4%), and assistance in understanding the feedback provided (10.9%). Eleven respondents (9.2%) felt that the surveillance system helped to manage TB.

 

Discussion

This evaluation of knowledge, attitudes, and practices related to the TB surveillance system generated information that can be applied to guide and enhance current TB programme activities in SA.

Findings from this evaluation illustrate the importance of staffing, training and experience for understanding and utilising TB surveillance information. TB programme staff reported a wide range of responsibilities crucial to the success of the TB surveillance system, but were greatly limited in their ability to carry out these duties adequately and consistently owing to realistic time constraints and competing obligations. It is therefore not surprising that over a quarter of respondents indicated that lack of human resources was one of the challenges faced by healthcare and TB programme staff. Both healthcare and TB programme staff reported a variety of TB-related training; however, 20.2% mentioned the need for additional training in order to improve TB surveillance.

Previous research has demonstrated the importance of the 'human element' in information system design and maintenance and the need for local ownership of information and systems.^[2] It is essential that healthcare and TB programme staff understand the purpose of the TB surveillance system and how to apply the information to the TB programme in SA. The majority of staff interviewed in this study reported using the TB surveillance system to guide planning, implementation and evaluation of the TB programme. However, responses from health facility personnel suggest that additional efforts are needed to communicate how daily tasks contribute to the overall surveillance system. A critical component to providing personnel with a sense of ownership and improving their understanding of the purpose of surveillance is clear, prompt, accurate and consistent feedback on programme performance from the next level of management.

Overall, there was widespread acceptance of TB surveillance system tools by TB staff; the majority found the tools easy to use and to contain adequate information for management of TB patients. However, many healthcare and TB programme staff were also employing other tools for TB management across all levels of the NTP. Most of the additional tools involved documenting additional patient information, which may relate to the staff's perceived lack of training on the current tools available.

Practices monitoring data completeness, accuracy and quality differed between the various levels of the TB programme. Facilities and programmes that are not routinely monitoring data recording and reporting may lack adequate human or computer resources or may need additional training. These findings highlight the need to invest in measures to ensure the completeness and accuracy of the TB surveillance system across all facility and programme levels. Furthermore, the majority of TB programme staff were unaware of specific deadlines for reporting data to the next level of management, and most continued to update data after reporting, even though the NTP has established deadlines for data reporting and guidance for deadlines within various levels of TB management. The responses from this survey suggest a need for improved communication at all levels of the TB surveillance system to ensure complete, accurate and timely reporting.

Monitoring, dissemination and performance evaluation serve as key mechanisms to engage and motivate TB healthcare and programme staff.^[4-6] In this study, most communications regarding surveillance data occurred through site visits to the health facilities and lower programme levels or via TB programme meetings. While the practice of site visits is most pertinent at the health facility level, in order to involve practitioners in a discussion of their needs and performance, there is a similar necessity at higher levels to increase motivation and improve feelings of ownership, utility and usability of information.^[4-6] Timeliness of feedback is also essential, as it allows lower programme levels to adapt. Furthermore, site visits offer opportunities for training and transfer of knowledge, which could improve the overall performance of the NTP.

Study strengths and limitations

This study provides detailed information on understanding, acceptability and use of the TB surveillance system across all levels of TB care. Results from this evaluation can be applied to guide system-wide activities to improve TB surveillance data. However, the study has limitations. The interviews were conducted in the three provinces of SA that were selected for inclusion in the national evaluation, and the responses therefore may not be generalisable to all areas of the country. The sample size for staff at the higher levels of the TB programme was inherently small owing to the number of staff operating at this level, yet the information collected may serve as a baseline for the use and understanding of the TB surveillance system by TB healthcare and programme staff. It is also possible that respondents answered questions in the manner they felt would be most acceptable to the interviewer. However, the questionnaires were purposely designed as semi-structured to allow for open-ended responses and to minimise the extent of response bias. The information garnered in this evaluation is limited to an assessment of the tools and systems currently used for TB surveillance in SA. However, the methods used provide a framework that may be useful to other national programmes for systematically evaluating knowledge, attitudes and practices related to surveillance systems.

 

Conclusions

The value of surveillance is only realised through its application, without which it becomes solely a set of numbers. Routine surveillance data are often viewed as having suspect validity, requiring costly supplemental surveys to obtain a more accurate understanding of the TB burden and epidemiology in a given setting. International bodies, such as the WHO, have promoted initiatives to improve the accuracy and reliability of routinely collected programme data and to define clear objectives for the use of surveillance data. It is critical to dedicate resources to ensuring the integrity and use of surveillance information, and to empower healthcare workers and programme managers with the skills and knowledge to analyse and apply data to inform programme activities in SA.

Funding. This work was supported by the Centers for Disease Control and Prevention and the South African Medical Research Council (Cooperative Agreement 5 U51 PS000729-05, PAS07-006).

Author contributions. LJP, NB and LDM contributed to the development and design of the evaluation protocol. LJP, LBM, CB and NB developed and piloted all data collection instruments and forms, created and finalised standard operating procedures and the procedure manual, and trained field staff. NB and CB led study field teams and were responsible for verifying and monitoring all data collected and monitoring data entry. LJP, LBM and CB were responsible for data management and all statistical analyses. All authors provided assistance with interpretation of results. LJP, LBM and CB led the writing of the manuscript, and NB and LDM reviewed and made substantial edits and contributions to the final manuscript.

Disclaimer. The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the US Centers for Disease Control and Prevention.

 

References

1. Centers for Disease Control and Prevention. Updated guidelines for evaluating public health surveillance systems: Recommendations from the guidelines working group. MMWR Morb Mortal Wkly Rep 2001;50(RR-13):1-31. http://www.cdc.gov/mmwr/preview/mmwrhtml/rr5013a1.htm (accessed 10 March 2016).         [ Links ]

2. Wilkins K, Nsubuga P, Mendlein J, Mercer D, Pappaioanou M. The data for decision making project: Assessment of surveillance systems in developing countries to improve access to public health information. Public Health 2008;122(9):914-912. DOI:10.1016/j.puhe.2007.11.002        [ Links ]

3. Podewils LJ, Bantubani N, Bristow C, et al Completeness and reliability of the Republic of South Africa National Tuberculosis Surveillance System. BMC Public Health 2015;15(1):765. DOI:10.1186/s12889-015-2117-3        [ Links ]

4. Hotchkiss DR, Eisele TP, Djibuti M, Silvestre EA, Rukhadze N. Health system barriers to strengthening vaccine-preventable disease surveillance and response in the context of decentralization: Evidence from Georgia. BMC Public Health 2006;6:175. DOI:10.1186/1471-2458-6-175        [ Links ]

5. Reeder B, Revere D, Olson DR, Lober WB. Perceived usefulness of a distributed community-based syndromic surveillance system: A pilot qualitative evaluation study. BMC Res Notes 2011;4:187. DOI:10.1186/1756-0500-4-187        [ Links ]

6. Heidebrecht CL, Tugwell PS, Wells GA, Engel ME. Tuberculosis surveillance in Cape Town, South Africa: An evaluation. Int J Tuberc Lung Dis 2011;15(7):912-918. DOI:10.5588/ijtld.10.0296        [ Links ]

 

 

Correspondence:
L J Podewils
lpp8@cdc.gov

Accepted 7 January 2016.