On-line version ISSN 2071-0763
Print version ISSN 0258-5200
SA j. ind. Psychol. vol.35 n.1 Cape Town Jan. 2009
Carin Marais; Karina Mostert; Sebastiaan Rothmann
WorkWell Research Unit for Economic and Management Sciences, North-West University, South Africa
ORIENTATION: The investigation of the psychometric properties of translated versions of a burnout measure.
RESEARCH PURPOSE: The purpose of this study was to examine the psychometric properties of translated versions of the Maslach Burnout Inventory - General Survey (MBI-GS) and the Cognitive Weariness Scale.
MOTIVATION FOR THE STUDY: To adhere to Section 8 of the Employment Equity Act, Act No. 55 of 1998 (p. 7), which stipulates that 'Psychological testing and other similar assessments are prohibited unless the test or assessment being used (a) has been scientifically shown to be valid and reliable, (b) can be applied fairly to all employees, and (c) is not biased against any employee or group.'
RESEARCH DESIGN, APPROACH AND METHOD: The current study follows the quantitative research tradition. This study was conducted with a convenience sample of members of the South African Police Service (SAPS; N = 685). The questionnaires were translated into Afrikaans and Setswana and were administered together with the original English version.
MAIN FINDINGS: The results indicated that a four-factor model, consisting of Exhaustion, Cynicism, Professional Efficacy, and Cognitive Weariness or Burnout, fitted the data best. The scales did not show measurement invariance for Afrikaans, Setswana and English samples. The reliabilities of the Exhaustion and Professional Efficacy subscales were acceptable in the three samples.
PRACTICAL/MANAGERIAL IMPLICATIONS: The implication of the results of the study under review is that the adapted MBI-GS scores obtained by the English, Afrikaans and Setswana home language speaking participants were not comparable in terms of the different translations of the instrument. Therefore, separate norms have to be developed to assess the extent of burnout experienced by the English, Afrikaans and Setswana home language speaking members of the SAPS who participated in the study. As the translated versions of the adapted MBI-GS are clearly not equivalent, more studies are needed to minimise the semantic differences existing between the different translations of the instrument.
CONTRIBUTION/VALUE-ADD: To date, no research regarding burnout has been conducted in South Africa within the different language groups by means of translated measuring instruments.
Keywords: burnout; construct validity; construct equivalence; reliability; police officers
Over the last decade, various cultures have demanded recognition in their own right and the multicultural nature of populations has become more apparent in many countries, including especially South Africa (Van de Vijver & Rothmann, 2004). Since 1994, South Africa has undergone many changes aimed at promoting the uniqueness of distinctive cultures. Such changes continue to influence the different aspects of South African society, particularly those that can be subjected to psychological testing. Section 8 of the Employment Equity Act, Act No. 55 of 1998 (p. 7) stipulates that 'Psychological testing and other similar assessments are prohibited unless the test or assessment being used (a) has been scientifically shown to be valid and reliable, (b) can be applied fairly to all employees, and (c) is not biased against any employee or group.' The requirements of the Act make the South African research community responsible for investigating, validating and standardising those measuring instruments that will ultimately be used to test and assess the members of all cultures and language groups scientifically and fairly.
Multicultural assessment is an important branch of the tree of psychological assessment. Van de Vijver and Rothmann (2004) report that the advent of multicultural assessment is mainly inspired by a growing societal need. It is a response to the perceived need to deal with a multitude of cultures in assessment without the a priori designation of a single culture as the target or model for other cultures. Cross-cultural studies, therefore, aim at establishing how similar to, and/or different from one another cultural groups are. Such studies rely on relatively structured means of data collection, such as standardised inventories (Van de Vijver & Leung, 2001). In the absence of cross-cultural comparisons, psychological theory would be confined within its own cultural boundaries. A blind 'exportation' of Western instruments to other cultures, without concern for the appropriateness of the measures, is also unlikely to lead to major theoretical advancements (Van de Vijver & Leung, 2001).
One such western psychological theory pertains to burnout, which is a psychological syndrome that involves experiencing a prolonged response to stressors in the workplace. The nature of burnout is currently being studied worldwide by various professions in different cultural settings (see Burke & Greenglass, 2001; Roelofs, Verbraak, Keijsers, De Bruin & Schmidt, 2005; Schaufeli, Martínez, Pinto, Salanova & Bakker, 2002). In South Africa, studies regarding burnout have been conducted across various samples, including among members of the South African Police Service (SAPS) (Storm & Rothmann, 2003), academic staff (Coetzee & Rothmann, 2004), educators (Jackson & Rothmann, 2005) and emergency workers (Naudé & Rothmann, 2006). Several of the researchers involved have confirmed the construct validity and equivalence of the Maslach Burnout Inventory (MBI). Even though the MBI appears to be a psychometrically valid, equivalent and reliable instrument for application in the South African context, an apparent limitation still exists - employees from different language groups have had to complete the original English version of this instrument.
South Africa is a multicultural society with 11 different national languages, of which English is one. Only 8.3% of South African citizens speak English at home (Statistics South Africa, 2001). Studies that have been conducted in the country in the past have generally reported that issues of race, education and language, including the comprehension of English, are the main factors that impact on the construct and item comparability of psychometric tests (Meiring, Van de Vijver, Rothmann & Barrick, 2005). Such findings have confirmed the need to translate measuring instruments, including the MBI, into the home language of those who undergo testing. If such translation does not take place, language-related issues might be a potent source of bias in cases where those tested differ in their proficiency in the language in which the test is conducted. Such bias is not uncommon in multicultural studies, in which a test is administered in the second or third language of the participants (Van de Vijver & Rothmann, 2004). If no attention is paid to such language differences, invalid conclusions regarding the constructs being studied could be drawn, with serious consequences for culturally diverse settings, such as those to be found in South Africa.
To date, no research regarding burnout has been conducted in South Africa within the different language groups by means of translated measuring instruments. In the case of the current study, the decision was made to translate the measuring instruments to be used into Setswana and Afrikaans, with the North West province being selected as the most appropriate region for data gathering. According to Burger (2005), Afrikaans and Setswana are the two main languages used in the region. Setswana home language speakers comprise 8.2% of the South African population, with 65.2% of such speakers residing in the North West province (Statistics South Africa, 2001). Furthermore, 7.4% of the North West province population are Afrikaans home language speakers, whereas only 1.2% can be identified as English home language speakers (Statistics South Africa, 2001). The diverse cultural composition of the SAPS, as well as its members' experience of burnout (Storm & Rothmann, 2003), make its members who live in the North West province an appropriate population for the current study.
Recently, clinical observations indicate that chronic burnout is associated with impaired cognitive functioning (Sandström, Rhodin, Lundberg, Olsson & Nyberg, 2005). However, few studies have included cognitive weariness as a burnout construct (Coetzee & Rothmann, 2004; Van Horn, Taris, Schaufeli & Schreurs, 2004). The present study, therefore, also included cognitive weariness as an additional dimension of burnout.
Based on the above arguments, the objective of the current study was to determine the construct validity, measurement invariance and internal consistency of the translated MBI-GS. The results were intended to reveal the above-mentioned limitations in the existing research into the experience of burnout in South Africa. Furthermore, it was anticipated that the results could possibly convey a more accurate view of burnout among Afrikaans and Setswana home language speaking SAPS members.
The Maslach Burnout Inventory - General Survey (MBI-GS)
The MBI was originally designed for use with human services and health care employees, such as teachers, nurses, and social workers. However, given the recent increase of interest in burnout within occupations that are other than traditionally people orientated, the MBI-GS was developed for use with employees in all occupational fields.
Maslach (2003) defines job burnout as a psychological syndrome that involves a prolonged response to stressors in the workplace. Specifically, such burnout results from chronic strain caused by an incongruence, or misfit, between the worker and the job. Burnout, as measured in occupations outside the human services, consists of three key dimensions: an overwhelming feeling of exhaustion, feelings of cynicism and detachment from the job, and a sense of ineffectiveness and lack of accomplishment.
The three-factor structure of the MBI-GS has been confirmed both internationally (Roelofs et al., 2005; Schaufeli et al., 2002) and in studies conducted in South Africa (Jackson & Rothmann, 2005; Storm & Rothmann, 2003). However, a new development has recently occurred in research into burnout. According to Schaufeli (2003), the MBI-GS, despite being an effective instrument from a clinical point of view is rather limited in scope and requires supplementation by a scale that is capable of assessing cognitive weariness. As a result, recent research into the phenomenon of burnout has employed a newly developed scale that enables greater focus to be lain on the concept. The alternative exhaustion scale, which was developed by Van Horn et al. (2004), is intended to assess levels of cognitive weariness, meaning the lack of capacity to absorb new information, as well as to concentrate.
Van Horn et al. (2004), in a study into the psychological well-being of teachers, tested and confirmed four different models: a one-factor model, in terms of which all facets of occupational well-being load on the one underlying factor; a five-factor orthogonal model, representing the five uncorrelated factors, as described in Van Horn et al.'s (2004) model of occupational well-being; a five-factor oblique model, representing the five correlated factors, as described in their model of occupational well-being; and a five-factor orthogonal model, with a second-order overall factor, on which the five first-order factors loaded. The final model confirmed a new five-factor structure of burnout, with the factors concerned being exhaustion, cognitive weariness, cynicism, professional efficacy, and psychosomatic complaints.
Coetzee and Rothmann (2004) attempted to verify the comparable model of Van Horn et al. (2004) by testing the following three default models: A one-factor model, in terms of which exhaustion, cognitive weariness, cynicism and professional efficacy load on one single factor; a three-factor model, in terms of which exhaustion and cognitive weariness form one factor, and cynicism and professional efficacy form the other two factors; and a four-factor model, in terms of which each of the four different scales represents a separate factor. Their results confirmed the proposed four-factor model of burnout for employees at a higher education institution in South Africa. Based on their results, it was hypothesised that a four-factor structure, in which the factors consist of exhaustion, cognitive weariness, cynicism and professional efficacy, would fit the data best (Hypothesis 1).
Various studies regarding the MBI-GS confirmed the soundness of the psychometric properties of the instrument (including variance of factor loadings), as well as its reliability and validity (Leiter & Schaufeli, 1996; Roelofs et al., 2005; Taris, Schreurs & Schaufeli, 1999). Schaufeli et al. (2002) found that the Dutch, Portuguese and Spanish translations of the three-factor MBI-GS were not equivalent across all the samples. Instead, different patterns of equivalence were found between the languages concerned, with efficacy items showing the most consistent pattern of equivalence of factor loadings. However, Storm and Rothmann (2003) confirmed the construct equivalence for the original three-factor MBI-GS for SAPS members. Jackson and Rothmann (2005) also found that the construct equivalence of the original three-factor MBI-GS was acceptable for Afrikaans and African home language speakers present in a sample of educators. The current study hypothesises that the English, Afrikaans and Setswana translations of the MBI-GS will be invariant across all three language groups (Hypothesis 2).
Internationally, internal consistencies of the MBI-GS have been found to be satisfactory, ranging from 0.73 (Cynicism) to 0.91 (Exhaustion). Acceptable Cronbach's alpha coefficients were also reported in the relevant South African studies. Coetzee and Rothmann (2004) found the four burnout scales to be reliable (Exhaustion: α = 0.85; Cynicism: α = 0.70; Professional Efficacy: α = 0.66; Cognitive Weariness: α = 0.76). Therefore, the present study expects that the four factors will all show acceptable internal consistencies (Hypothesis 3).
The current study follows the quantitative research tradition. A cross-sectional survey design was used in the study. Questionnaires were used to gather primary data by means of a non-random field survey.
Research participants and procedure
A convenience sample (N = 685) was drawn from SAPS members working in police stations situated in the North West province. After a presentation of the proposed research was given to the Provincial Head of Psychological Services, permission was granted to incorporate the research into the unit's ongoing research programme. The management, including the Station Commissioner and the Human Resource Management Head, of each of the identified stations in the province was contacted to explain the rationale behind, and the procedures involved in, the proposed study. English, Afrikaans and Setswana home language speaking police officers from randomly selected stations received paper-and-pencil questionnaires, which were distributed by members of the Psychological Services Unit. A letter explaining the rationale of the study, as well as giving assurances of the anonymity and confidentiality with which responses would be treated, accompanied each questionnaire. The relevant members were given two to four weeks in which to complete the questionnaire, on completion of which period the members of the Psychological Services Unit personally collected the completed questionnaires from each station. The response rate was approximately 35%. Table 1 presents some of the characteristics of those who completed their questionnaires.
The MBI-GS (Maslach, Jackson & Leiter, 1996) was used to measure the degree of burnout experienced by the participants in the study. The MBI-GS consists of 16 items and has three subscales, which are Exhaustion (five items, e.g. 'I feel used up at the end of the workday'); Cynicism (four items, e.g. 'I have become less enthusiastic about my work'); and Professional Efficacy (seven items, e.g. 'In my opinion, I am good at my job'). All items are scored on a seven-point frequency rating scale, ranging from 0 (never) to 6 (every day). The three components of the Burnout construct are conceptualised in broader terms relating to the job, rather than just to the personal relationships that might form part of the job (Maslach, Schaufeli & Leiter, 2001). Together, the subscales of the MBI-GS provide a three dimensional perspective on burnout.
In addition to the MBI-GS, the Cognitive Weariness Scale (CWS) (five items) was administered. The CWS was developed by Van Horn et al. (2004) to measure the level of cognitive well-being. Though the scale initially consisted of seven different items, the developers recommended that, due to the high internal consistency of items 3 and 7, such items be dropped from the general seven-item version of the scale. The scale, as it is now, tests the capacity to absorb new information and to concentrate at work, with items such as 'I have trouble concentrating'. It is scored on a seven-point frequency scale, ranging from 0 (never) to 6 (every day). Van Horn et al. (2004) reported a Cronbach's alpha coefficient of 0.92.
A Biographical Questionnaire was used to ascertain the biographical characteristics of the SAPS members concerned. The questionnaire investigated such dimensions as background information (age, race, gender, and educational information), job situation (rank and years in service), and the number of hours worked.
The measuring instruments were translated from English into Afrikaans and Setswana. The latter two languages are the home language of most of the inhabitants of the North West province, with 65.4% of the resident population being Setswana home language speakers and 7.5% being Afrikaans home language speakers (Burger, 2005). The Afrikaans and Setswana versions of the measuring instrument were then translated back into English. The original English language measuring instrument was then compared with the retranslated measuring instrument. The same procedure was followed when questionable items were found in the retranslated measuring batteries, in order to find a better fit for the items. All translating was undertaken by four independent translators, who were registered with the South African Translator's Institute. The questionnaires were then finalised and distributed to the participants concerned, with each receiving a questionnaire in their home language.
The statistical analysis was performed with the aid of the SPSS (SPSS, 2008) and AMOS programs (Arbuckle, 2006). Firstly, the construct validity and equivalence of the adapted and translated MBI-GS was examined. Structural equation modelling (SEM) methods were used to test the factorial models for the translated MBI-GS. Comparing three models in terms of the relationships among the 21 items tested the construct validity (measurement invariance) of the translated MBI-GS. Multi-group confirmatory factor analysis (CFA) was used to test the construct equivalence of the factor structure and the equivalence of parameter estimates (in terms of factor loadings, factor covariances and item error variances) for the different language versions.
To test for the measurement invariance of the adapted MBI-GS, the χ2 statistic was used to determine the difference in statistical fit between the unconstrained and constrained models. The null hypothesis (H0) postulates that the specification of the factor loadings, factor variances/covariances, and error variances for the model under study are valid, with the χ2 test simultaneously testing the extent to which such a specification is true. The probability value associated with χ2 represents the likelihood of obtaining a χ2 value that exceeds the χ2 value when H0 is true. Thus, the higher the probability associated with χ2, the closer is the fit between the hypothesised model (under H0) and the perfect fit (cf. Byrne, 2001). Due to the sensitivity of the likelihood ratio test (LRT) to sample size and it being based on the central χ2 distribution, several other goodness-of-fit indices are used to summarise the degree of correspondence between the implied and observed covariance matrices (see Byrne, 2001). Such matrices include: a) χ2/df ratio; b) the Goodness-of-Fit Index (GFI); c) the Incremental Fit Index (IFI); d) the Tucker-Lewis Index (TLI); e) the Comparative Fit Index (CFI); and f) the Root Mean Square Error of Approximation (RMSEA). Values of 0.90 and higher signify acceptable fit for GFI, TLI and CFI, whereas values of 0.08 and lower indicate acceptable fit for RMSEA (Byrne, 2001).
Descriptive statistics were used to analyse the data that were collected in the study. Cronbach's alpha coefficients were used to assess the reliability of the scales, with the product-moment correlations being used to determine the relationship between the dimensions.
Firstly, the construct validity of the adapted MBI-GS was tested. It was decided not to include item 13 of the MBI-GS in the analysis, since previous studies in this regard had shown that the item lacks validity (for reviews, see Jackson & Rothmann, 2005; Schutte, Toppinen, Kalimo & Schaufeli, 2000; Storm & Rothmann, 2003). In the hypothesised models, each of the 20 observed variables loads onto only one factor. The indicator variables in the model were treated as continuous variables. Errors of measurement associated with each of the observed variables were uncorrelated.
The following hypothesised models of the adapted MBI-GS were tested:
Model 1: A one-factor model, consisting of one first-order latent factor (namely Burnout).
Model 2: A three-factor model, consisting of one third-order latent factor, namely Burnout; three second-order latent factors, namely Fatigue, Cynicism, and Professional Efficacy; and two third-order latent factors, namely Exhaustion and Cognitive Weariness, which made up Fatigue.
Model 3: A four-factor model, consisting of one second-order latent factor, namely Burnout, and four first-order latent factors, namely Exhaustion, Cognitive Weariness, Cynicism, and Professional Efficacy.
The fit indices for models 1 and 3 are provided in Table 2. Model 2 resulted in negative variances, indicating that the model fit was poor. No further analyses were undertaken regarding Model 2.
The results reported in Table 2 indicate that Model 1 did not fit the data, since the results did not fall within the acceptable boundaries of goodness-of-fit indices, as indicated in the literature (χ2 = 1360.96; GFI, IFI, TLI and CFI < 0.90 and RMSEA > 0.08). Model 3, which distinguished between the four proposed dimensions of Burnout, explained the associations among the items significantly better than did the other competing models (Model 3 vs. Model 1: Δχ2 = 908.24; Δdf = 4; p < 0.001).
The third model hypothesised that the adapted MBI-GS consists of four latent first-order factors, namely Exhaustion (5 items), Cognitive Weariness (5 items), Cynicism (4 items), and Professional Efficacy (6 items), and one latent second-order factor, namely Burnout. It was assumed that the errors of items were uncorrelated. The model was over-identified, yielding 210 distinct sample moments, 40 distinct parameters to be estimated, and 170 degrees of freedom.
The statistically significant χ2 value of 452.72 (df = 166; p < 0.01) revealed a poor overall fit of Model 3. However, the sensitivity of the likelihood ratio test (LRT) to sample size and its basis on the central χ2 distribution, which assumes that the model fits perfectly in the population, have been reported as leading to problems of fit. Jöreskog and Sorbom (1993) point out that the use of χ2 is based on the assumption that the model holds exactly in the population, which is a stringent assumption. A consequence of the assumption is that the models that hold approximately in the population will be rejected in a large sample. Regarding the acceptability of Model 3 from a practical perspective, acceptable fit is evident from the achievement of GFI, TLI and CFI values higher than 0.90, as well as from the RMSEA value, which is lower than 0.08. Since the model fit was acceptable and the results agreed with the theoretical assumptions underlying the structure of the adapted MBI-GS, no further modifications of the model were deemed necessary.
Cynicism had the highest loading on Burnout (β = 0.95, p < 0.01), followed by Cognitive Weariness (β = 0.83, p < 0.01), Exhaustion (β = 0.83, p < 0.01), and Professional Efficacy (β = -0.51, p < 0.01). The squared multiple correlation (R2) for Cynicism was 0.90, indicating that the predictors of Cynicism explain 80% of its variance, while its error variance was 10%. Furthermore, the R2 values for Cognitive Weariness, Exhaustion and Professional
Efficacy were 0.69, 0.59, and 0.26 respectively, indicating error variances of 31%, 41%, and 45% for the three respective factors. Hypothesis 1, which postulates that burnout can be characterised as a four-dimensional construct that distinguishes between Exhaustion, Cognitive Weariness, Cynicism, and Professional Efficacy, is thus accepted.
Measurement invariance of the MBI-GS
Next, the hypotheses relating to the measurement invariance of factor loadings, factor variances and covariances of the four-factor structure of the adapted MBI-GS (Hypothesis 2) were tested at the statistical level. The measurement invariance of the adapted MBI-GS was tested, using the LRT (Vandenberg & Lance, 2000). The LRT assesses the difference in chi-square (i.e. Δχ2) per degree of freedom between the initially developed model and a more restricted model, including equality constraints. Using the LRT, a non-significant Δχ2 indicates that the parameters constraining equality are not significantly different across groups. All the tests of invariance across the samples began with a global test of the equality of their covariance structures (Jöreskog, 1971). In testing for such equivalencies, the sets of parameters are tested in a logical order and by increasing the restrictions in every step. The sets of parameters that are of most interest regarding group variances are: (a) the factor-loading paths; (b) the factor variances/covariances; and (c) the structural regression paths. The results of the multi-group analyses to assess the measurement invariance of the adapted MBI-GS for the Afrikaans, English and Setswana home language speaking groups are reported in Table 3.
Table 3 indicates that a χ2 value of 889.25 (df = 492) was obtained for the baseline (unconstrained) model in the three samples. In the first step of the multi-group analysis, the measurement weights of the adapted MBI-GS were equally constrained. The change in χ2 was statistically significant (Δχ2 = 96.77, Δdf = 32, p < 0.01). In subsequent steps, the measurement weights, structural weights, and structural covariances were equally constrained between the three samples. Statistically significant changes in χ2 values were found. This finding confirms the lack of measurement invariance of the adapted MBI-GS for the three samples.
Given that the factor structure of the adapted MBI-GS was not invariant for the three samples, it was decided to investigate the standardised regression coefficients of the 20 items of the adapted MBI-GS in the unconstrained model (see Table 4).
The most notable differences between the standardised regression coefficients in the three groups were evident for CW3 (β = 0.16, Setswana home language speakers sample), CW1 (β = 0.39, Afrikaans home language speakers sample), and MBI-GS5 (β = 0.22, Setswana home language speaking sample). It was then decided to re-specify the measurement model after the three items had been deleted. Table 3 shows the results of the multi-group analyses for the Afrikaans, English and Setswana home language speaking groups.
Table 3 indicates that a χ2 value of 707.62 (df = 345) was obtained for the baseline (unconstrained) model in the three samples, with the three items deleted. In the first step of the multi-group analysis, the measurement weights of the adapted MBI-GS were equally constrained. The change in χ2 was statistically significant (Δχ2 = 78.11, Δdf = 26, p < 0.01). In subsequent steps, the measurement weights, structural weights, and structural covariances were equally constrained between the three samples. Statistically significant changes in χ2 values were found. This finding confirms that the measurement invariance of the adapted MBI-GS for the three samples is still not acceptable, even after the three items that were notably invariant were removed.
Such results do not support the measurement invariance of the adapted MBI-GS across the three language groups concerned. The construct equivalence of the adapted MBI-GS for the English, Afrikaans and Setswana home language speaking SAPS members is not confirmed. Hypothesis 2 is therefore rejected.
Descriptive statistics, reliability and the relationship between the burnout constructs
Given the lack of measurement invariance of the adapted and translated MBI-GS, it was necessary to compute descriptive statistics, reliability, and correlations for the English, Afrikaans and Setswana home language speaking samples.
The descriptive statistics and alpha coefficients of the adapted and translated versions of the MBI-GS for the three language groups are reported in Table 5.
Table 5 shows that, in the Afrikaans home language speaking sample, all four scales were found to be reliable (α > 0.70; Nunnally & Bernstein, 1994). In the English home language speaking sample, three scales, namely Exhaustion, Cognitive Weariness, and Professional Efficacy, showed acceptable reliabilities. However, the alpha coefficient of Cynicism was just below the recommended value of 0.70 (Nunnally & Bernstein, 1994). In the Setswana home language speaking sample, only two scales, namely Exhaustion and Professional Efficacy, had higher alpha coefficients than 0.70. Such results provide partial support for Hypothesis 3.
The product-moment correlations between the scales of the adapted and translated versions of the MBI-GS for the three language groups are reported in Table 6.
The results in Table 6 indicate that Exhaustion is positively and statistically significantly related to Cognitive Weariness (large effects) and Cynicism (large effects) in the English and Afrikaans samples. Furthermore, Cynicism is also positively statistically significantly related to Cognitive Weariness (large effects) in three samples. Professional Efficacy is negatively and statistically significantly related to Cognitive Weariness (medium effects) and Cynicism (medium effects) in the English and Afrikaans home language speaking samples. In the Setswana home language speaking sample, Exhaustion is statistically and practically significantly related to Cognitive Weariness and Cynicism (medium effects).
The objective of the study described in this article was to investigate the construct validity, measurement invariance and internal consistency of the translated MBI-GS and the CWS. The results support the construct validity of an adapted and translated MBI-GS for SAPS members. The adapted and translated MBI-GS, however, showed a lack of measurement invariance for participants who completed the English, Afrikaans and Setswana versions of the instrument. The reliabilities of the Exhaustion and Professional Efficacy scales were acceptable for the three language groups. However, the reliabilities of the CWS and Cynicism scale were not acceptable.
The first step in achieving this was to determine the construct validity of the adapted burnout model, including exhaustion, cognitive weariness, cynicism and professional efficacy. This model was tested by comparing three models using SEM. The results revealed that burnout can be characterised as a four-dimensional construct that distinguishes between the four dimensions, supporting previous research undertaken regarding the proposed four-structure model of the MBI-GS (Coetzee & Rothmann, 2004; Van Horn et al., 2004). Cynicism had the highest loading on Burnout, followed by Cognitive Weariness, Exhaustion, and Professional Efficacy. The results showed that the predictors of Cynicism explained 80% of its variance, while 69%, 59% and 26% of the variances of Cognitive Weariness, Exhaustion and Professional Efficacy respectively explained their predictors. The results support the previous research undertaken regarding the psychometric properties of the instrument (Leiter & Schaufeli, 1996; Roelofs et al., 2005; Storm & Rothmann, 2003; Taris et al., 1999).
Next, the hypotheses relating to the measurement invariance of the adapted and translated four-factor MBI-GS were tested at the statistical level (using χ2 statistics). The results did not support the measurement invariance of the adapted and translated MBI-GS. The measurement weights were equally constrained between the English, Afrikaans and Setswana home language speaking samples, and produced a statistically significant difference from that obtained in the observed unconstrained model. The same tendency was found when structural weights and structural covariances were equally constrained between the three language groups. It could, therefore, be presumed that, even though the translations and backward translations were accurate, the different cultures could still ascribe different meanings to individual or collective items. The construct being investigated might, therefore, assume a different meaning in the respective cultures, since the cultures in question differ from that in terms of which the instrument had originally been developed. Although a valid and reliable translation or adaptation of a measuring instrument would stand to combine high levels of linguistic, psychological and statistical equivalence, attaining such equivalence is not always possible, so that the development of a new and culturally specific instrument is called for.
Analysis of the standardised regression coefficients showed that the three translated items made a notable contribution to the lack of measurement invariance. Such items include CW3 ('I have trouble concentrating') in the Setswana sample; CW1 ('I find it difficult to focus my attention on my job') in the Afrikaans sample; and MBI-GS5 ('I can effectively solve the problems that arise in my work') in the Setswana sample. However, after these items had been removed from the analysis, the measurement invariance for the three language groups was still not regarded as acceptable.
Although it can be concluded that the factor structures of the adapted and translated MBI-GS are acceptable in the English, Afrikaans and Setswana home language speaking groups, the measurement invariance of the measure is not acceptable. The lack of measurement invariance can be attributed to one of the following: (1) semantic differences, resulting from the translation of the original English text into Setswana and Afrikaans; (2) the cultural differences between the three different language groups; (3) both the semantic and cultural differences existing between the three different language groups. However, it is more plausible that measurement invariance in the adapted and translated MBI-GS is the result of semantic differences associated with the former cause. Researchers often make use of translation and back-translation procedures; however, despite the usefulness of back-translating, such a procedure does not guarantee the achievement of valid results, since it is difficult to assess the quality of translations (Cheung & Thumboo, 2006). Zhang's (2008) experiment showed that cultural factors only play a minor role in determining people's lexicon; with linguistic elements, such as the phonological structure, the syntactic category, the morphological structure and the presence of semantically related words, being the leading dominants. Storm and Rothmann (2003) showed the measurement invariance of the MBI-GS when it was administered in English to samples of SAPS members from different cultural groups.
The reliability analyses showed that Exhaustion and Professional Efficacy, being two of the four scales of the adapted and translated MBI-GS, are sufficiently reliable for English, Afrikaans and Setswana home language speakers. Such results support the findings made in various international and South African studies (e.g. Coetzee & Rothmann, 2004; Leiter & Schaufeli, 1996; Roelofs et al., 2005; Storm & Rothmann, 2003; Taris et al., 1999). However, the results showed that the CWS was not sufficiently reliable for the Setswana home language speakers, and that the reliability of the Cynicism scale was lower than the recommended guideline for both the English and Setswana home language speaking groups.
The results show that the relation between the experience of Cynicism and Cognitive Weariness are the only constructs that are shared between the three samples. Furthermore, very high correlations were found between Exhaustion, Cognitive Weariness and Cynicism in the English and Afrikaans home language samples, compared to the correlations between these constructs within the Setswana home language speaking sample. Such findings were in accordance with a study conducted by Schaufeli, Leiter and Kalimo (1995), in which the researchers found that exhaustion was associated with mental strain. To a certain extent, the results of the study discussed in this article are similar to those of Coetzee and Rothmann (2004), who found that Afrikaans and English home language speakers reported experiencing higher levels of exhaustion and cognitive weariness than did the indigenous language groups. The interviews conducted with the Afrikaans home language speaking members of the SAPS revealed that they regarded the following factors as increasing the levels of burnout that they experienced: their lack of promotion due to affirmative action; the demanding workload; the lack of adequate resources; and the generally negative view of the public towards them.
Limitations and recommendations
The study described in the current article had various limitations. Firstly, the participants in the study consisted only of some SAPS members working in the North West province. More research is needed into the psychological well-being of employees from other organisations. Secondly, the sample sizes were not optimal. Most notably, the Setswana home language speaking sample (N = 166) was relatively small, which could have impacted on the results obtained in the initial construct validity study. Also, the sample size in general was relatively small, given the assumptions of the statistical analysis technique (SEM) that was employed.
The implication of the results of the study under review is that the adapted MBI-GS scores obtained by the English, Afrikaans and Setswana home language speaking participants were not comparable in terms of the different translations of the instrument. Therefore, separate norms have to be developed to assess the extent of burnout experienced by the English, Afrikaans and Setswana home language speaking members of the SAPS who participated in the study. As the translated versions of the adapted MBI-GS are clearly not equivalent, more studies are needed to minimise the semantic differences existing between the different translations of the instrument. The use of linguistic validation interviews, in which a group of native speakers of the target language assess the new translation and discuss their understanding of it, will help (Cheung & Thumboo, 2006).
As the study described in the current article focused only on SAPS members in the North West province, additional studies with the original, as well as with the translated, instruments should be carried out in the other eight South African provinces. The results obtained in such studies could then be compared with those obtained in the present study. Furthermore, the study reviewed in the current article translated the two instruments concerned into only two of the 11 official languages of South Africa. Translating the MBI-GS into the other eight official languages as well would promote the in-depth investigation of burnout across cultures in South Africa.
Since the present study was conducted among a relatively homogenous group, other studies, exploring a greater variety of demographic characteristics, should serve to enhance the reliability and usefulness of the MBI-GS. Furthermore, it is recommended that external variables, such as job characteristics, ill health and personality variables be investigated by means of additional research into the phenomenon of burnout in South Africa.
The material described in the current article is based upon work supported by the National Research Foundation, under reference number TTK2005081000025. The research was carried out as part of Carin Marais' doctoral studies, which were completed at North-West University.
Arbuckle, J.L. (2006). Amos 7.0 user's guide. Chicago: SPSS Inc. [ Links ]
Burke, R.J., & Greenglass, E.R. (2001). Hospital restructuring and psychological burnout in nursing staff. Equal Opportunities International, 20(1/2), 61-71. [ Links ]
Byrne, B.M. (2001). Structural equation modelling with AMOS: Basic concepts, applications and programming. Mahwah: Erlbaum. [ Links ]
Cheung, Y.B., & Thumboo, J. (2006). Developing health-related quality-of-life instruments for use in Asia: The issues. PharmacoEconomics, 24(7), 643-650. [ Links ]
Coetzee, S.E., & Rothmann, S. (2004). An adapted model of burnout for employees at a higher education institution in South Africa. South African Journal of Industrial Psychology, 30(3), 29-40. [ Links ]
Jackson, L.T.B., & Rothmann, S. (2005). An adapted model of burnout for educators in South Africa. South African Journal of Education, 25(2), 100-108. [ Links ]
Jöreskog, K.G. (1971). Simultaneous factor analysis in several populations. Psychometrika, 36, 409-426. [ Links ]
Jöreskog, K.G., & Sorbom, D. (1993). LISREL 8: Structural equation modelling with the SIMPLIS command language. Hillsdale: Lawrence Erlbaum Associates. [ Links ]
Leiter, M.P., & Schaufeli, W.B. (1996). Consistency of the burnout construct across occupations. Anxiety, Stress, & Coping, 9, 229-243. [ Links ]
Maslach, C. (2003). Job burnout: New directions in research and intervention. Current Directions in Psychological Science, 12, 189-192. [ Links ]
Maslach, C., Jackson, S.E., & Leiter, M.P. (1996). Maslach Burnout Inventory manual (3rd edn.). Palo Alto: Consulting Psychologists Press. [ Links ]
Maslach, C., Schaufeli, W.B., & Leiter, M.P. (2001). Job burnout. Annual Review of Psychology, 52, 397-422. [ Links ]
Meiring, D., Van de Vijver, A.J.R., Rothmann, S., & Barrick, M.R. (2005). Construct, item, and method bias of cognitive and personality tests in South Africa. South African Journal of Industrial Psychology, 31(1), 1-8. [ Links ]
Naudé, J.L.P., & Rothmann, S. (2006). Work-related well-being of emergency workers in Gauteng. South African Journal of Psychology, 36, 63-81. [ Links ]
Nunnally, J.C., & Bernstein, I.H. (1994). Psychometric theory (3rd edn.). New York: McGraw-Hill. [ Links ]
Republic of South Africa (RSA). (1998). Employment Equity Act, Act No. 55 of 1998. Cape Town: Government Printers. [ Links ]
Roelofs, J., Verbraak, M., Keijsers, G.P.J., De Bruin, M.B.N., & Schmidt, A.J.M. (2005). Psychometric properties of a Dutch version of the Maslach Burnout Inventory General Survey (MBI-DV) in individuals with and without clinical burnout. Stress & Health, 21 (1), 17-25. [ Links ]
Sandström, A., Rhodin, I.N., Lundberg, M., Olsson, T., & Nyberg, L. (2005). Impaired cognitive performance in patients with chronic burnout syndrome. Biological Psychology, 69, 271-270. [ Links ]
Schaufeli, W.B. (2003). Past and future perspectives of burnout research. South African Journal of Industrial Psychology, 29(4), 1-15. [ Links ]
Schaufeli, W.B., Leiter, M.P., & Kalimo, L. (1995). The General Burnout Inventory: A self-report questionnaire to assess burnout at the workplace. Paper presented at Work. Stress and Health '95: Creating Healthier Workplaces, Washington, D.C., September 14-16, 1995. [ Links ]
Schaufeli, W.B., Martínez, I.M., Pinto, A.M., Salanova, M., & Bakker, A.B. (2002). Burnout and engagement in university students: A cross-national study. Journal of Cross-Cultural Psychology, 33, 464-481. [ Links ]
Schutte, N., Toppinen, S., Kalimo, R., & Schaufeli, W.B. (2000). The factorial validity of the Maslach Burnout Inventory - General Survey (MBI-GS) across occupational groups and nations. Journal of Occupational and Organizational Psychology, 73, 53-66. [ Links ]
SPSS (2008). SPSS 16.0 for Windows. Chicago: SPSS Inc. [ Links ]
Storm, K., & Rothmann, S. (2003). A psychometric analysis of the Maslach Burnout Inventory-General Survey in the South African Police Service. South African Journal of Psychology, 33, 219-226. [ Links ]
Taris, T.W., Schreurs, P.J.G., & Schaufeli, W.B. (1999). Construct validity of the Maslach Burnout Inventory - General Survey: A two-sample examination of its factor structure and correlates. Work & Stress, 13, 223-237. [ Links ]
Vandenberg, R.J., & Lance, C.E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-69. [ Links ]
Van de Vijver, F.J.R., & Leung, K. (2001). Personality in cultural context: Methodological issues. Journal of Personality, 69, 1007-1031. [ Links ]
Van de Vijver, F.J.R., & Rothmann, S. (2004). Assessment in multicultural groups: The South African case. South African Journal of Industrial Psychology, 20(4), 1-7. [ Links ]
Van Horn, J.E., Taris, T.W., Schaufeli, W.B., & Schreurs, P.J.C. (2004). A multidimensional approach to measuring teacher wellbeing. Journal of Occupational and Organizational Psychology, 77, 365-375. [ Links ]
Zhang, B. (2008). Different types of word links in the mental lexicon. College Student Journal, 42(2), 431-439. [ Links ]
Postal address: North-West University, Potchefstroom
Private Bag X6001 (Internal Box 114) Potchefstroom
Received: 10 July 2009
Accepted: 07 Sept. 2009
Published: 21 Oct. 2009
This article is available at: http://www.sajip.co.za
© 2009. The Authors. Licensee: OpenJournals Publishing. This work is licensed under the Creative Commons Attribution License.