SciELO - Scientific Electronic Library Online

 
vol.107 número2Is it possible to differentiate tuberculous and cryptococcal meningitis in HIV-infected patients using only clinical and basic cerebrospinal fluid characteristics? índice de autoresíndice de materiabúsqueda de artículos
Home Pagelista alfabética de revistas  

Servicios Personalizados

Articulo

Indicadores

Links relacionados

  • En proceso de indezaciónCitado por Google
  • En proceso de indezaciónSimilares en Google

Compartir


SAMJ: South African Medical Journal

versión On-line ISSN 2078-5135
versión impresa ISSN 0256-9574

SAMJ, S. Afr. med. j. vol.107 no.2 Pretoria feb. 2017

http://dx.doi.org/10.7196/samj.2017.v107i2.11331 

RESEARCH

 

Readiness to adopt a performance measurement system for substance abuse treatment: Findings from the Service Quality Measures initiative

 

 

B MyersI, II; P Petersen WilliamsI, II; K JohnsonIII; R GovenderIV; R ManderscheidV; J R KochVI

IPhD; Alcohol, Tobacco and Other Drug Research Unit, South African Medical Research Council, Cape Town, South Africa
IIPhD; Department of Psychiatry and Mental Health, Faculty of Health Sciences, University of Cape Town, South Africa
IIIMA; Alcohol, Tobacco and Other Drug Research Unit, South African Medical Research Council, Cape Town, South Africa
IVPhD; Department of Sociology, Faculty of Humanities, University of Cape Town; and Violence, Injury and Peace Research Unit, South African Medical Research Council, Cape Town, South Africa
VPhD; National Association of County Behavioral Health and Developmental Disability Directors, Washington DC, USA
VIPhD; Department of Psychology, Virginia Commonwealth University, Richmond, VA, USA

Correspondence

 

 


ABSTRACT

BACKGROUND. A performance measurement system - the Service Quality Measures (SQM) initiative - has been developed to monitor the quality of South Africa (SA)'s substance abuse treatment services. Identifying factors associated with readiness to adopt this system may inform strategies to facilitate its robust implementation.
OBJECTIVE. To examine factors associated with readiness to adopt a performance measurement system among SA substance abuse treatment providers.
METHODS. We surveyed 81 treatment providers from 13 treatment sites in the Western Cape, SA. The survey examined awareness, resources, organisational climate, leadership support and readiness to adopt the SQM system. Regression analysis was used to identify factors associated with readiness to adopt this system.
RESULTS. Readiness to adopt the SQM initiative was high (M=5.64, standard deviation 1.63). In bivariate analyses, caseload size (F=3.73 (degrees of freedom (df)=3.70), p=0.015), awareness (r=0.78, p<0.0001), leadership support (r=0.70, p<0.0001), resources (r=0.65, p<0.0001), openness to change (r=0.372, p=0.001), and external pressure to change were associated with readiness to adopt the SQM. In multivariate analyses, only awareness of the SQM initiative (B=0.34, standard error (SE) 0.08, t=4.4, p<0.0001) and leadership support (B=0.45, SE 0.11, t=4.0, p<0.0001) were significantly associated with readiness to adopt this system.
CONCLUSION. While treatment providers report high levels of readiness to adopt the SQM system, findings show that the likelihood of adoption can be further increased through improved provider awareness and enhanced leadership support for this health innovation.


 

 

The prevalence of substance use disorders is high in South Africa (SA), with an estimated 13% of the adult population meeting the Diagnostic and Statistical Manual of Mental Disorders, 4th edition criteria for a lifetime diagnosis of a substance use disorder.[1] Although SA has an established substance abuse treatment system, concerns have been raised about the quality and impact of treatment. Several studies have shown that a large proportion of people with untreated substance use disorders feel that treatment is of limited effectiveness.[2] These negative beliefs are a major barrier to treatment initiation.[3]

Although information about the quality of substance abuse treatment can address these negative beliefs, the quality and effectiveness of treatment have not been routinely monitored by SA treatment providers.[4,5] To address this gap, we developed a performance measurement system for SA's substance abuse treatment services, i.e. the Service Quality Measures (SQM) initiative. This performance measurement system consists of three tools: the SA Community Epidemiology Network on Drug Use (SACENDU) admission form that collects sociodemographic and substance use history information of patients enrolled into substance abuse treatment and is completed by programme administrators; the SA Addiction Treatment Services Assessment (SAATSA), which collects patient-reported data on perceived outcomes, quality and accessibility of treatment and is completed by patients after 2 - 3 weeks of treatment;[6] and a discharge form, which counsellors complete after their patients are discharged from or leave treatment. This information collects administrative data on the type of services received and the patient's response to treatment.[7] The developmental process[6,7] and the process evaluation of this system's initial implementation have been reported elsewhere.[8]

Despite a robust developmental phase, questions remain about the extent to which treatment facilities will adopt the SQM system. As the implementation of this system creates some additional paperwork for treatment counsellors and support staff, and the system draws conclusions about the quality of services they provide, treatment staff may feel threatened by adopting it.[9] Organisations generally value stability and may resist adoption of the SQM, as it entails a measure of transitional discomfort from the current status quo of minimal monitoring. Understanding the factors associated with readiness to adopt a performance measurement system may provide insights into the development of interventions to enhance the uptake of the SQM system.[9]

To the best of our knowledge, there has been little examination of factors associated with readiness to adopt performance measurement systems for substance abuse or mental health services, either locally or globally. Although studies conducted in high-income countries have shown that substance abuse treatment providers' readiness to adopt evidence-based practices varies as a function of leadership, staff skills and attributes, organisational resources, organisational openness to change, and internal and external pressures for change,[9,11] the extent to which these factors are relevant to the adoption of performance measurement systems in the SA context remains uncertain. To address this gap, we explored factors associated with readiness to adopt a performance measurement system among SA substance abuse treatment providers.

 

Methods

This article presents data from a cross-sectional survey of treatment providers involved in the implementation of the SQM system.

Procedures

In 2014, the SQM initiative, a performance measurement system developed for SA's substance abuse treatment services, was first implemented in 13 treatment facilities (3 residential and 10 outpatient) in the Western Cape, SA. The Western Cape was purposively selected as an implementation site, as it is the most well-resourced province in terms of publicly funded substance abuse treatment programmes. These programmes serve diverse population groups, which exhibit a broad range of substance use disorders. As part of the evaluation of the pilot implementation of this system[17] staff at participating facilities completed a survey examining the extent to which the SQM initiative had been adopted by their respective organisations and the characteristics of their organisation that may be associated with system adoption. At each participating facility, maximum variation sampling techniques were used to ensure that we included participants with a diverse range of roles and responsibilities, such as administrative personnel, counsellors, nurses and service managers. We considered this important for the evaluation of the initiative, as both support and clinical staff members were involved in the implementation of this performance measurement system. Those who agreed to participate were asked to provide written informed consent prior to completing the questionnaire. Participants were given a ZAR50 gift voucher to thank them for their participation. The final sample comprised 81 participants with a diverse range of roles, including programme directors (n=7), clinical supervisors (n=3), counsellors (n=46), and support staff who conduct clinical intakes (n=26). The mean age of participants was 38.8 years (standard deviation (SD) 11.0) and 71% were females.

Survey instrument

The survey questionnaire explored demographic characteristics of participants, as well as awareness of the SQM system, readiness to adopt the system, and a range of other organisational factors associated with adoption of innovations that have been identified in previous research.[10,11] The items were explored in focus groups with providers and were shown to have good face validity.[6]

Demographic questions. Such questions assessed age and gender.

Job-related items. These items assessed participants' job roles (programme director, clinical supervisor, counsellor, or support staff); years of experience (<1 year, 1 - 3 years, >3 years); time in current job (<1 year, 1 - 3 years, >3 and <5 years, >5 years); and number of clients on caseload (1 - 10, 11 - 20, 21 - 30, >30).

Awareness of the SQM initiative. Participants used a 7-point Likert scale to rate the extent to which treatment staff members were aware of the SQM initiative. Responses ranged from 1 (not at all) to 7 (completely).

Readiness to adopt. Participants used a 7-point Likert scale to rate the extent to which they agreed that the organisation would readily adopt this performance measurement system. Responses ranged from 1 (strongly disagree) to 7 (strongly agree). This was the primary outcome variable.

Resources. Participants used a 7-point Likert scale to rate the extent to which they agreed that their facility had sufficient resources to implement the SQM system. Responses ranged from 1 (strongly disagree) to 7 (strongly agree).

Leadership support. Participants used a 7-point Likert scale to rate the extent to which they agreed that the SQM initiative had support from their facility's leadership. Responses ranged from 1 (strongly disagree) to 7 (strongly agree).

Openness to change. Participants used a 7-point Likert scale to rate the extent to which they thought their facility was open to changing their programme to improve the quality of treatment. Responses ranged from 1 (not at all) to 7 (completely).

Pressure for change. The pressure for change subscale from the organisational readiness for change (ORC) scale was used to assess staff perceptions of external pressures for change and adoption of new practices. The scale comprises 7 items, with responses on a 5-point scale ranging from 1 (disagree strongly) to 5 (agree strongly). The ORC has previously been used in SA.[12] Internal consistency was excellent for this subscale (α=0.92).

Analysis

Bivariate analyses were conducted, examining the relationship between all of the variables listed above and the dependent variable (readiness to adopt the SQM initiative). Next, a multivariate linear regression analysis was conducted, entering into the model all variables significantly related to readiness to adopt the SQM initiative in bivariate analyses at p<0.01, to be conservative. All analyses were done in SPSS version 24 (IBM Corp., USA).

 

Results

Sample characteristics

Table 1 presents the full sample characteristics (n=81). Approximately half of the staff (54%) were employed as counsellors, 51% had been in the field for at least 3 years, and just over half (52%) had been in their current job for >3 years. Almost a third of the sample (30%) had >30 patients on their caseload. Descriptive data of all individual- and organisational-level variables are given in Table 1. Readiness to adopt the SQM initiative was high in this sample, with an obtained mean (SD) score of 5.64 of a possible 7 (1.63).

 

 

Bivariate analyses

At the bivariate level, the only individual-level factor related to readiness to adopt the SQM measure was individual caseload (F=3.73 (degrees of freedom (df)=3.70), p<0.05). Organisational-level variables related to readiness to adopt were greater awareness of the SQM initiative (r=0.78, p<0.001), more leadership support for the implementation of the SQM initiative (r=0.70, p<0.001), more resources within the organisation to support adoption of the SQM initiative (r=0.65, p<0.001), greater organisational openness to change (r=0.372, p<0.001), and more external pressure to change as measured by the TCU ORC (TCU Institute of Behavioral Research, USA) pressure to change subscale (r=0.23, p<0.05). All bivariate relationships with readiness to adopt the SQM initiative are given in Table 1.

Multivariate model

Variables associated with readiness to adopt the SQM system at the bivariate level at p<0.01 were included in the testing of a multivariate regression model. There was little impact of collinearity among the independent variables in this model, with all variance inflation factors <2.5. Only greater awareness of the SQM initiative (B=0.34, standard error (SE) 0.08, t=4.4, p<0.001) and more leadership support were significantly associated with greater readiness to adopt the SQM initiative (B=0.45, SE 0.11, f=4.0, p<0.001). Multivariate linear regression results are shown in Table 2.

 

 

Discussion

This study examined readiness to adopt a performance measurement system among treatment providers from publicly funded substance abuse treatment facilities in the Western Cape, SA. Both individual-and organisational-level factors related to greater readiness to adopt this system were explored. At the individual level, the only factor related to greater readiness to adopt the SQM system was the size of the staff member's patient caseload. Staff members with more patients on their caseload reported greater readiness to adopt this system. As providers with large caseloads in low-resourced services often feel overburdened and have few resources to meet each patient's needs,[2,5] providers with large caseloads may have been more concerned about the quality of treatment than those with smaller caseloads. Alternatively, providers with large caseloads may have been more ready to adopt this system because they expected it to streamline their administrative duties and reduce their paperwork.[6]

At the organisational level, greater awareness of the SQM system among staff at the treatment facility, more support from facility leadership for the implementation of the system, more resources available to support adoption of the system, an organisational climate that was more open to change and the adoption of new practices, and greater perceived pressure for change were also related bivariately to high levels of readiness to adopt the SQM performance measurement system. In the multivariate model, the only factors associated with readiness to adopt this system were awareness of the SQM initiative among facility staff and leadership support for the implementation of the system. After controlling for the potential impact of limited resources and organisational climate, treatment facilities with leadership that actively endorsed and supported the SQM initiative, and staff who were highly aware of the purpose and benefits of this system, were more ready to adopt the system than those with poor leadership support and staff who were less aware of the system.

The finding in this study that treatment providers' awareness of the SQM initiative was significantly associated with readiness to adopt, even after controlling for the potential influence of other organisational characteristics, is not particularly surprising, given evidence from other studies that staff awareness of a new health system innovation is highly predictive of its adoption.[13] In addition to providing staff with training in the implementation of the SQM system,[8] we provided facility-wide training and information on the purpose and benefits of performance measurement as well as the organisation's need for such a system. Building awareness of the organisation's need for a performance measurement system and the associated benefits to the individual and organisation, arguably enhanced treatment providers' readiness to adopt the system.[9,13] Beyond training, community engagement and marketing activities also may have enhanced treatment providers' awareness and acceptance of the initiative. For instance, the pilot implementation of the SQM system was preceded by an intensive community engagement phase that included a series of focus group discussions with treatment providers, presentations at treatment facilities to discuss the need for such a system, and marketing by means of posters and information brochures.[6-8] Collectively, these activities could have enhanced awareness and acceptance of the SQM system among treatment staff, partially accounting for the high degree of readiness to adopt the SQM initiative observed among this sample of treatment providers.

We also found that leadership support for the implementation of the SQM initiative was strongly associated with readiness to adopt; this effect prevailed even after controlling for other potential influences on system implementation. This finding is in keeping with evidence that suggests that it is difficult to promote changes to routine practices within health services without the support of decision leaders.[13,14] Leaders' attitudes and actions influence how staff members within their respective organisations perceive proposed innovations and help to create organisational climates that are open to change. As such, leaders can either encourage or inhibit adoption of the proposed change to routine practice.[19,12-14] Interestingly, organisational resources and openness to change, which were associated with readiness to adopt in bivariate analyses and in other studies,[14,15] were no longer associated with adoption when leadership support was added to the multivariate model. While this finding requires further investigation and confirmation, the result highlights the importance and centrality of treatment leaders for establishing an organisational climate supportive of and responsive to the adoption of new systems.

Findings from this study confirm that building substance abuse treatment managers' and other decision-makers' support for the SQM initiative is key to facilitating the routine implementation of the system. To enhance leadership support for this initiative, the SQM team needs to demonstrate to treatment leaders how the information generated from this system can be used as a managerial support tool to inform programmatic decision-making and to secure additional resources for their organisations by proving to funders that their services are effective.[9] Furthermore, as earlier studies have shown that many substance abuse treatment leaders do not have the requisite skills to fully utilise management information systems,[6] some treatment leaders may need further training in the analysis and interpretation of their data to secure support for the ongoing implementation of the SQM system. However, we were unable to establish the exact nature of leadership support or managerial styles that enhanced readiness to adopt the SQM initiative. The implementation science literature distinguishes between transformational and transactional leadership styles, and notes the differential impact of these leadership styles on adoption of evidence-based practices.[15] Future research that produces a more nuanced understanding of how different types of leadership are associated with organisational readiness to adopt the SQM initiative may lead to the identification of strategies to increase the likelihood of system adoption.

While this study identified several ways in which readiness to adopt the SQM initiative could be enhanced, findings should be considered in the light of some study limitations. The main limitation was the relatively small sample size, which had various implications for the analysis. Firstly, the sample comprised predominantly counsellors and support staff. While staff members in these roles were directly responsible for implementing the SQM system, the very small proportion of programme directors and treatment managers did not permit a fuller examination of the key variables as differentiated by job role. Such analysis may have highlighted further avenues for increasing readiness to adopt the SQM system, particularly as other studies have found that perceptions of organisational readiness to change vary as a function of such job roles.[12,15] Similarly, the relatively small sample size prevented stratification of findings by level of treatment provided. It is quite likely that resource challenges, pressures for change and staff attributes may have differed across facilities as a function of their not-for-profit/profit status and with regard to whether they provided inpatient or outpatient care. Additional studies, with larger sample sizes comprising greater representation of job roles and facility types, are needed to address these questions. Third, given the purposive nature of the sampling and because the study was limited to treatment providers from facilities located in the Western Cape, findings may not be generalisable to facilities located in other provinces, where the context of substance abuse treatment service delivery may differ. As the SQM initiative is expanded into other provinces, it is important to identify drivers of adoption in these regions so that contextually appropriate interventions to enhance the likelihood of sustained implementation can be developed and implemented.

Despite these limitations, this study yielded valuable information that can be used to enhance substance abuse treatment providers' readiness to adopt the SQM performance measurement system. It is noteworthy that there were relatively high levels of readiness to adopt the system within this sample of treatment providers. In part, this reflects the largely positive perceptions that treatment providers have of the SQM initiative,[7] which can be attributed to both the simplicity and ease of use of the system, and the community engagement activities that the SQM team undertook to ensure that treatment providers had a role in the development of the sys-tem.[5, 6] Related to this, as findings show that enhanced provider awareness of the SQM initiative is associated with greater readiness to adopt this performance measurement system, community engagement activities and facility-wide training with regard to the need for and benefits of performance measurement may be of critical importance for enhancing organisational support for adoption. Findings also indicate that building leadership support for the initiative is critical to ensuring organisational readiness to adopt this performance measurement system. In addition to the need to develop a better understanding of the role and impact of different leadership styles, training of leaders in the need for performance measurement systems, demonstrating how performance data might be used to their organisation's benefit, and developing treatment leaders' capacity to interrogate their own data are some strategies for ensuring leaders' support for adoption.

Acknowledgement. This article was supported by the Western Cape Department of Social Development.

Disclaimer. The findings and conclusions are those of the authors and do not necessarily represent the official position of the Western Cape Department of Social Development.

 

REFERENCES

1. Herman AA, Stein DJ, Seedat S, Heeringa SG, Moomal H, Williams DR. The South African Stress and Health (SASH) study: 12-month and lifetime prevalence of common mental disorders. S Afr Med J 2009;99(5):339-344.         [ Links ]

2. Myers B, Fakier N, Louw J. Stigma, treatment beliefs, and substance abuse treatment use in historically disadvantaged communities. Afr J Psychiatry 2009;12(3):218-222. http://dx.doi.org/10.4314/ajpsy.v12i3.48497        [ Links ]

3. Myers B, Louw J, Pasche S. Inequitable access to substance abuse treatment services in Cape Town, South Africa. Subst Abuse Treat Prev Policy 2010;5(1):28. http://dx.doi.org/10.1186/1747-597X-5-28        [ Links ]

4. Myers B, Burnhams NH, Fakier N. Monitoring and evaluation of substance abuse services in South Africa: Implications for policy and practice. Int J Ment Health Addiction 2010;8(4):557-565. http://dx.doi.org/10.1007/s11469-009-9232-2        [ Links ]

5. Myers B, Govender R, Manderscheid R, Williams PP, Johnson K, Loch JR. Need for and readiness to implement a performance measurement system for South Africa's substance abuse treatment services. Int J Ment Health Addiction, early online (6 October 2016). http://dx.doi.org/10.1007/s11469-016-9706-y        [ Links ]

6. Myers B, Govender R, Koch JR, Manderscheid R, Johnson K, Parry CD. Development and psychometric validation of a novel patient survey to assess perceived quality of substance abuse treatment in South Africa. Subst Abuse Treat Prev Policy 2015;10(1):44. http://dx.doi.org/10.1186/s13011-015-0040-3        [ Links ]

7. Myers B, Petersen Z, Kader R, et al. Identifying perceived barriers to monitoring service quality among substance abuse treatment providers in South Africa. BMC Psychiatry 2014;14(1):2-10. http://dx.doi.org/10.1186/1471-244X-14-31        [ Links ]

8. Myers B, Williams PP, Johnson K, Govender R, Manderscheid R, Koch JR. Providers' perceptions of the implementation of a performance measurement system for substance abuse treatment: A process evaluation of the service quality measures initiative. S Afr Med J 2016;106(3):308-311. http://dx.doi.org/10.7196/SAMJ.2016.v106i3.9969        [ Links ]

9. Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F. Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, 2005.         [ Links ]

10. Simpson DD. A conceptual framework for transferring research to practice. J Subst Abuse Treat 2002;22(4):171-182. http://dx.doi.org/10.1016/S0740-5472(02)00231-3        [ Links ]

11. Simpson DD, Dansereau DF. Assessing organizational functioning as a step toward innovation. Sci Pract Perspect 2007;3(2):20-28. http://dx.doi.org/10.1151/spp073220        [ Links ]

12. Bowles S, Louw J, Myers B. Perceptions of organizational functioning in substance abuse treatment facilities in South Africa. Int J Ment Health Addiction 2011;9(3):308-319. http://dx.doi.org/10.1007/s11469-010-9285-2        [ Links ]

13. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Quart 2004;82 (4):581-629. http://dx.doi.org/10.1111/j.0887-378X.2004.00325.x        [ Links ]

14. Garland A, Bickman L, Chorpita B. Change what? Identifying quality improvement targets by investigating usual mental health care. Adm Policy Ment Health 2010;37(1-2):15-26. http://dx.doi.org/10.1007/s10488-010-0279-y        [ Links ]

15. Aarons GA, Hurlburt M, Horwitz S. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health 2011;38(1):4-23. http://dx.doi.org/10.1007/s10488-010-0327-7        [ Links ]

 

 

Correspondence:
B Myers
bmyers@mrc.ac.za

Accepted 26 October 2016

Creative Commons License Todo el contenido de esta revista, excepto dónde está identificado, está bajo una Licencia Creative Commons