On-line version ISSN 2076-3433
S. Afr. j. educ. vol.29 n.3 Pretoria Aug. 2009
The advocacy of an appraisal system for teachers: a case study
Tom BisschoffI; Annah MathyeII
ISenior Lecturer at the University of Birmingham and Emeritus Professor from the University of Johannesburg, is the co-author of seven books, of which the most recent is Mestry R & Bisschoff T 2008, Financial School Management Explained. Cape Town: Pearson. E-mail: firstname.lastname@example.org
IIAnnah Mathye is Senior Manager in the Limpopo Department of Education and holds a doctorate from the University of Johannesburg
Education systems all over the world, like all other organisations, have certain organisational goals that they set and wish to achieve. It is argued that for increased pupil performance, in the case of education systems, teachers must work harder and smarter. A performance system is regarded as part of the process to achieve this organisational goal. Some prominent researchers, however, forecast a movement away from performance management as we know it. In this paper we use insights from a specific school district in South Africa to highlight the need for effective advocacy of a performance system to ensure a measure of success. One of the major findings is that although advocacy in its narrow sense is a process to make stakeholders aware of a new policy, the data collected reveal two broad themes, namely, the process itself but also the issues about the content of that which should be advocated. Findings on the process itself focus on sufficient funding, effective training, reconsideration of the cascading mode of delivery, clarity on the roles of different structures, official sources of information, anticipated reaction of teachers and effective monitoring of the implementation process. Findings on the content of IQMS focus on the conceptual framework of IQMS, clarity of the content and contextual factors to be considered.
Keywords: human resource management in education; Integrated Quality Management System (IQMS); performance management in education
What led to this inquiry?
During the 2008 Commonwealth Council for Educational Administration and Management (CCEAM) held in Durban in South Africa two of the keynote speakers spoke about a post performance management era that schools and education systems are moving into. Andy Hargreaves coined it the "fourth way" and Tony Townsend called it the "fifth 'S'-curve". From questions and remarks from the delegates (predominantly practitioners from the schooling sector in South Africa) it was clear that the papers were seen as not applicable to South Africa. Some delegates insisted that we (South Africa) were still in the inspection phase and needed it to get rid of incompetent teachers or force them to work harder. "We are now in the accountability phase and need an inspection system to get the lazy teachers to work or out of the system" (CCEAM, 2008:1).
Some delegates insisted that their teachers need an inspection system from a developmental point of view. These remarks are interesting in the light of Foskett and Lumby's comment on the myth that appraisal systems improve teachers' performance (2003:74). In many countries the management of teachers' performance has negative connotations as is the case in South Africa (Makau & Coombe, 1994). Managing performance is associated with a punitive and often an inefficient inspection system. In considering how to support teachers to achieve a satisfactory or better performance, governments have striven to move forward by balancing the strong accountability element of previous inspection regimes with a more positive determination to provide support.
With the IQMS, the department seeks to provide a framework to ensure that each teacher's individual contribution contributes to the effectiveness of the system. The individual performance must lead to the achievement of the school system's goal as an organisation. Such a framework can only work if staff know what is required of them, receive feedback on how they are doing, and are supported to achieve expectations (Middlewood & Lumby, 1998; Middlewood & Cardno, 2001). The way the performance system is advocated is therefore paramount to its success.
This triggered us to look at the thesis, submitted by the second author under the first author's supervision in 2007, on the advocacy of the Integrated Quality Management System (IQMS) implemented in South Africa recently. The research questions of the thesis were: how did we advocate IQMS to the school-based teachers (SBTs) in the province and what were the perceptions of teachers on the advocacy process? These two questions also form the driving force for this article.
If it is so, that we in South Africa need to move beyond a teacher performance system as we know it, then it is logical that a new system is imminent. Therefore the study is relevant and timely to ensure the effective advocacy of such a new system.
The importance of effective advocacy of policy was accepted more than two decades ago in the economic and developmental public sectors but it appears that, in the case of education policy changes and implementation, we are somewhat behind in the light of the information collected from the teachers (Peattie & Aldrete-Haas, 1981). Teachers seem to resist what they perceive as attempts by government to further restrict their professional freedom. Regardless of teachers' perceptions, government carries on regardless with its reform agenda. Despite evidence of initiatives that fail governments remain stubbornly surprised when their plan remains unimplemented and move swiftly on to the next change (Coombe, 1997; Foskett & Lumby, 2003; Timperley & Robinson, 1997). Foskett and Lumby conclude with the words: "Nowhere is this scenario seen more acutely than in the area of managing teacher performance" (2003:75).
As far back as 20 years ago, Sabatier (1988:129) wrote about an advocacy coalition framework of policy change in the field of policies that would curb pollution. His advocacy framework focuses on the belief systems of advocacy coalitions within policy systems as the critical vehicle for understanding the role of policy analysis in policy-oriented learning and the effect, in turn, of such learning on changes in governmental programmes. The work done by Cohen and his colleagues also provides us with a better understanding of advocacy as an essential part of implementing policy (Cohen, De la Vega & Watson, 2001. See also Jansson, 2007). The line between policy implementation and policy advocacy is a thin line especially at the beginning of the implementation phase. For the purposes of this study, advocacy must be understood as an integral part of policy implementation.
The so-called teacher performance management trajectory in South Africa went through a number of phases after the 1994 elections in South Africa. After getting rid of any remnants of the inspection structures that remained from the previous regime, the department started with a Developmental Appraisal System (DAS), moved towards a Whole School Development System (WSD), then towards a Whole School Evaluation System (WSE), which was further developed into a Performance Measurement system (PM), which culminated in the Integrated Quality Management System (IQMS) and is practice in South African schools. DAS aims at appraising individual teachers in a transparent manner whereby a teacher evaluates him/herself and discusses the outcomes with the Development Support Group at every school. WSE evaluates the overall effectiveness of a school while PM evaluates individual teachers for salary progression, grade, appointment affirmation, rewards and incentives (ELRC Collective Agreement 8, 2003: Section A). All three of these policies (DAS, WSE and PM) were faced with implementation problems in the relevant province owing to the manner in which they were advocated to school-based teachers (Daniels, 2007:5. See also De Clerq, 2008, on the implementation readiness of the majority of schools). When IQMS was advocated, it was a way of stepping back and reflecting collectively on the enacted policies. The education department thought that for quality to exist in the system, different structures needed to be in place as a way of ensuring continuous improvement. For IQMS to be successfully implemented, there are structures that should be introduced in schools, like the School Development Teams (SDT) and Development Support Groups (DSG) (ELRC Collective Agreement 8, 2003: Section 3).
Our inquiry attempts to explore and describe the advocacy phase in the implementation of the IQMS in one of the districts in one of the provinces of South Africa. The research questions are: how did we advocate IQMS to the School Based Teachers (SBTs) and what are the perceptions of teachers on the advocacy process? The first research question could only be addressed by looking at the official documentation that was available within the head office and district office and only provided the so-called official perspective on the advocacy process. In answering the second research question a more in-depth and fuller picture of the process becomes evident.
The inquiry started off by identifying different types (rural, semi-rural, urban, and township) of secondary schools in the district in one of the provinces of South Africa and having focus group interviews with teachers of the schools. Each of the four schools represented the initial criterion for type of school but the final choice was more a matter of convenience and accessibility to the researchers. The emphasis on the different types of school was done to get "maximum variation" in the data collected. This is in accordance with the spirit of qualitative research and its quest for explanations which encompass complexity, subtlety and even contradictions (Denscombe, 2003: 26). A focus group interview has advantages (generates richer responses by allowing participants to challenge one another's' views) but in the context of this study it also created a problem in the sense that "quieter" people in the group did not express their views or only a somewhat moderated view that more reflected the "acceptable view" of the group (Denscombe, 2003:168).
Ethical clearance was obtained from the relevant education department as well as from the Faculty: Ethics Committee of the university. In total four schools were identified and the groups were made up of post level one teachers (to ensure that no power relations existed within the groups) who underwent training in the IQMS.
The interviews were audio-taped and transcribed. Those sections that were in one of the local languages (Xitsonga or TshiVenda) were translated. The interview protocol used was a simple question on the participants' perceptions of how the IQMS was advocated to them as school-based teachers with a series of follow-up questions (Arksey & Knight, 1999:97; Litosseliti, 2003: 55). The follow-up questions followed a "funnel approach" and were aimed at increasing the richness and depth of responses (Stewart & Shamdasani, 1990: 76; Patton, 2002:372). Trustworthiness was enhanced by recording an audit trail, analysing in collaboration and doing regular member checks (Maykut & Morehouse, 1994:146-147; 152).
The analysis was done in a more formal way and not the 'other way' so clearly put by Denscombe:
One way of doing this (qualitative data analysis) is to become immersed in the data, looking at them many times and then making an intuitive attempt to identify the key categories and connections on the basis of knowing the data so well that insights simply come to the researcher almost as a matter of inspiration (2003:270).
Meticulous attention was paid to the so called cut-and-paste method (Barry, 1998), which ensured a more systematic procedure in the analysis process. We regard this section as very important because the process of analysing the data is often overlooked in reports of qualitative data. Whereas an explanation of statistical techniques and data manipulation always accompanies the use of quantitative data in research this is not the case with qualitative reports (Denscombe, 2003:277).
The four transcribed interviews went firstly through an analytic coding phase. The raw data were scanned for particular ideas that could form a unit of analysis. Units of data — bits of information — are literally sorted into groupings that have something in common. A unit of data is any meaningful (or potentially meaningful) segment of data. Lincoln and Guba (1985; cf. Merriam, 1998:179) say a unit must meet two criteria. First, it should be heuristic — i.e. the unit should reveal information relevant to the study (research questions) and stimulate the reader to think beyond the particular bit of information. Second, the unit should be "the smallest piece of information about something that can stand by itself — i.e. it must be interpretable in the absence of any additional information other than a broad understanding of the context in which the inquiry is carried out (Lincoln & Guba, 1985:345). In the spirit of open coding, time was not wasted on why certain ideas were chosen but more on the discovering, naming and categorising the components of the concept of advocacy (cf. Strauss & Corbin, 1990:181).
In the second phase early categories and sub-categories were reflected on and compared so as to refine them to better explain the concept of advocacy. The researchers were constantly aware of the guidelines Merriam state for the efficacy of categories, namely, they must reflect the purpose of the research; they must be exhaustive; mutually exclusive; sensitising and conceptually congruent (1998:183-184). Tables 1 and 2 are attempts to do just that. These categories were reflected on by both authors and discussed in depth. Although the aim was not to develop grounded theory the analysis was done according to what Glaser and Strauss (1967) call the constant comparative method of data analysis. The basic strategy of the method is to do just what its name implies, namely, to constantly compare. The initial categories and constant comparing led to the refinement of the categories (Merriam, 1998:159).
Findings and discussion
The two broad themes that came to the fore were issues with regard to the process itself and issues with regard to the content of IQMS. First the process will be discussed and then the content.
The broad categories and sub-categories in Table 1 were identified from the four focus interviews. All of these relate to the IQMS process of advocacy.
The table will be discussed in detail to understand the shortcomings in the advocacy process but also to suggest some guidelines that may support the advocacy process in the future.
Sufficient funds needed for the advocacy process
From the data collected from the four interviews it is evident that there were not sufficient funds allocated for the process. Participants' complaints on this matter were clear in the following statements:
That is why it (training workshops TCB & AM) was switched from three days to one day.
Because of financial constraints we ended up holding a half day workshop.
We received notices that due to financial constraints the workshop has been cancelled.
More effective training of SBT to be provided
From the data collected it appeared that there was a problem with the duration and quality of the workshops.
It was done haphazardly in a very short space of time
All these things were done in one day.
those who had to help us were trained for a day
Reconsideration of the use of the cascading mode of information dissemination
The IQMS advocacy process included the school principal, one other member of the school management team, and one teacher representing the staff. These invited members were expected to cascade down the information they obtained from the workshops to all staff members. The other member of the school management team and the teacher representing the staff were waiting for the principal to disseminate the information to the staff but that was delayed due to a lack of confidence (by the principals) and uncertainty on when they were suppose to advocate the IQMS to all staff members. When eventually the dissemination took place it was based on a weak understanding of the system and loss of memory due to the time lapse between the workshop and actual cascading of the information. In the dissemination the negativity of the principal about the system came out very clearly and influenced the perceptions of staff at that specific school.
Weeks after the workshop my principal started off the staff meeting with — " guys I cannot remember everything they told us at the workshop but here is the gist "
Our principal was negative about the whole thing — he said they just want to change for the sake of change.
More clarity on the roles of different structures in the advocacy process
The whole process of the advocacy of the IQMS is set out in the ELRC Collective Agreement (2003: Section A). The composition of the two key groups, namely, the School Developmental Team (SDT) with the principal and democratically elected member of staff including a beginner teacher and the Development Support Group (DSG), made up of the teacher who is undergoing appraisal, an immediate senior and a peer (who must be a teacher). A peer is selected on the basis of appropriate phase/Learning Area expertise (ELRC Collective Agreement 8, 2003: 13 — Section A). The size of the SDT is decided by the school as well as the duration of the term of office. According to the ELRC Collective Agreement the following are the functions of an SDT, namely, that they should:
- Ensure that all teachers are trained on the procedures and processes of the IQMS
- Co-ordinate activities pertaining to staff development
- Prepare and monitor the management plan of the IQMS
- Facilitate and give guidance on how DSGs have to be established
- Link the developmental appraisal to the school improvement plan
The functions of the DSGs are to:
- Mentor and support the teachers
- Assist teachers to compile their Personal Growth Plans (PGP) (a PGP is a record of needs and progress of an individual teacher).
The participants were adamant that the unions should play a more prominent role.
and also from unions
But at least we got more information from unions than from the department
I attended a union work session where I got something about IQMS
Need for more official and secure sources of information for SBT
From the remarks made by the participants there was total confusion on the sources of information with regard to IQMS. Some teachers received a circular (official letter that is written and distributed to various schools by the department and can be via the circuit office, district office or direct from the provincial head office) which invited them to a former College of Education building; some were however only verbally informed.
they wrote us a letter which invited us to a common venue
Some government officials told us verbally that we are going to start with IQMS.
it was verbal and then followed up by written information
More thorough investigation into what reactions of SBT on the IQMS process could be expected
It was clear that the department did not anticipate many negative reactions from the SBT because from the interviews it was clear that their concerns were not addressed in the workshops. Some teachers were surprised by the new policy since DAS, WSE, and PM in their eyes have failed.
this is a new monster of a thing that is going to fail
A more positive reaction was that they felt that they would receive more money after the appraisal process.
my reaction is that inspection is coming back in another form
actually, I am surprised of what was going to come I am suspicious of the success of the whole thing
the news I exciting because it was just about money
More effective monitoring of the implementation process of IQMS
Part of the process was also to monitor the implementation of the advocacy process. Schools were notified that departmental facilitators will do school visits to help schools if they experience challenges during the implementation of IQMS. The following statement is self evident:
I only received a circular indicating that from such a date up to such a date they will be moving around, but none of them ever came to our school.
These facilitators lack knowledge on IQMS.
Some schools were sampled, others were left out and this did not give the department a true reflection of what was happening in schools.
On the issue of the content Table 2 will be discussed.
Table 2 will now be discussed in detail to understand the shortcomings in the content of IQMS as it relates to the advocacy of the system but also to suggest some guidelines that may support the advocacy in future.
Conceptual framework of IQMS
From the interviews it is clear that there was a lot of confusion on the protocol of the IQMS as set out in the conceptual framework. The protocol sets out step-by-step the process that must be followed during the observation of a teacher in his/her practice. The framework stresses that for any teacher to be evaluated, the policy should have been advocated to him/her prior to the observation. The DSG (Development Support Group) should have been formed and it should include the teacher's immediate senior and a peer who is selected on the basis of Phase or Learning Area expertise. The DSG is there to mentor and support the candidate. The practice as described by the participants point towards the signing of IQMS process forms by the DSG without any consultation with the candidate on his/her submitted Personal Growth Plan. Due to the violation of the conceptual framework and the associated protocol participants were sceptical about the content of IQMS.
The papers must go somewhere the next thing we are supposed to get money.
The practicality and cost-effectiveness of the IQMS content will now be discussed. The IQMS content includes the completion of several forms, which request personal details, self-evaluation forms, the PGP and the School Improvement Plan (SIP). Reference to the Performance Standards when a teacher is observed in his/her practice is also required. All this leads to a paper- driven system that does not appear practical (schools with few resources) or cost-effective, considering that all this will lead to a 1% progression in the salary of the successful candidates.
we are only going to get 1%
Part of the problem related to the conceptual framework was the lack of understanding of the content and the lack of constructive and professional engagement by teachers. Because of the unsuccessful advocacy of the IQMS instrument it was not understood and that resulted in teachers not being empowered to manage the flow chart (as set out in the conceptual framework) to complete the PGP and SIP. The teachers who attended the IQMS advocacy sessions did not understand its content. The following comments were made:
is a clear indication that they do not understand this
Most of the questions he (facilitator during the workshop) did not answer.
The participant did not experience the IQMS design as being constructive in its design. It did not, according to the participants, improve the competence level of the teaching fraternity. On the contrary, it led to teachers cheating and threatening the DSG. The following comments stress the above conclusion:
.. It is very difficult to prove whether somebody has cheated or what
That is why we are going to cheat because we want those points (to get more money)
I think you should always be there for a friend. How can you down grade a friend?
obviously, my friend is not going to reveal that information
The above comments and conclusion challenge the instrument on two levels namely, is it constructive and is it professional? Simply, cheating and threatening fellow colleagues results in tension and hinders constructive and professional engagement in the IQMS. The success of any performance system is based on a sense of ownership by those affected, coupled with a degree of honesty and integrity from all involved.
Clarity of the IQMS content
For the successful advocacy of an appraisal system clarity of the content is paramount. From the participants it seems that the IQMS facilitators were unsure of the IQMS content. They did not demonstrate an understanding of the content. The participants make the following damning comments on this aspect:
even those people who conducted the advocacy-training to our principals and those educators who went to workshops were not 100% sure of certain issues related to IQMS
He (facilitator) was not clear because we went out of that workshop like the way we went in
the people who were training us were not knowledgeable
The participants were adamant that the IQMS is flawed in its content because it does not consider the contextual factors that impact on the performance of a teacher. The one recurring factor was the one on pupil-teacher ratios. On the one hand the department views the benchmark of 1:35 (for secondary schools) and 1:40(for primary schools) as the reality at the chalk-face level but it is clearly not the case:
you get 50 learners in one classroom
having up to 94 learners in a class is more difficult for an educator than having 35
Can you imagine being evaluated in a class of 80 learners?
In the same vein participants referred to the general lack of resources in most township and rural schools that affect the appraisal of teachers in that school.
if you put Mr X (an award winning teacher) in my school he will not be as successful
In conclusion we can only concur with Foskett and Lumby (2003:85) that any notion that improving teachers' salaries and conditions or managing their performance more tightly will improve the experience and outcomes for learners is far too simplistic. First, motivation and development are intensely personal and individual; and no formal process will succeed in improving performance if the individuality of each member of staff is not recognised as an essential element. The multi-lingual and multi-cultural context only serves to exacerbate this point in the South African context. Second, appraisal and review give an account of performance as a starting point for development. The latter involves that teachers learn in its broadest sense, increase knowledge, skills and attitudes. The comments on policy changes made by Sabatier are also very relevant here:
understanding the role of policy-oriented learning requires a time perspective of a decade or more (and) public policies (or programmes) can be conceptualised in the same manner as belief systems, namely, as sets of value priorities and causal assumptions about how to realise them (Sabatier, 1988:131). See also Middlewood & Cardno (2001).
Managing performance is, therefore, a holistic process that should be advocated effectively and implemented as a part of managing teachers to enhance learning in the school. If leaders in education have the courage to do this (see the findings and discussion in this paper) and customise best practices in the world for local conditions they could well be providing a model of lifelong learning for teachers and in that way sustain improvement in the delivery of the organisational goal of education systems.
Arksey H & Knight P 1999. Interviewing for social scientists — An introductory resource with examples. London: SAGE. [ Links ]
CCEAM 2008. CCEAM 2008. Conference Report. Johannesburg: Conference Committee. [ Links ]
Cohen D; De la Vega R & Watson G 2001. Advocacy for social justice: A global action and reflection guide. Sterling: Kumarian Press. [ Links ]
Coombe C 1997. Unleashing the power of Africa's teachers. International Journal of Educational Development, 17:113-117. [ Links ]
Daniels L 2007. Pandor pursues plan to introduce performance rewards for teacher. Cape Times, 27 February. [ Links ]
De Clerq F 2008. Teacher Quality, Appraisal and Development: The Flaws in the IQMS. Perspectives in Education, 26:7-18. [ Links ]
Denscombe M 2003. The Good Research Guide, 2nd edn. Maidenhead: Open University Press. [ Links ]
ELRC Collective Agreement 8 2003. Integrated Quality Management System. Polokwane: Review Printers. [ Links ]
Foskett N & Lumby J 2003. Leading and managing Education — International Dimensions. London: SAGE. [ Links ]
Glaser BG & Strauss AL 1967. The Discovery of Grounded Theory. Chicago: Aldine. [ Links ]
Jansson BS 2007. Becoming an effective policy advocate: From policy practice to social justice. Pacific Grove: Brooks/Cole Publishing. [ Links ]
Lincoln YS & Guba EG 1985. Naturalistic Inquiry. Thousand Oaks: SAGE. [ Links ]
Litosseliti L 2003. Using focus groups in research. London: Continuum Publishers. [ Links ]
Makau B & Coombe C 1994. Teacher Morale and Motivation in Sub-Saharan Africa: Making practical Improvements. London: Commonwealth Secretariat. [ Links ]
Mathye AK 2007. The perceptions of school-based educators on the advocacy of Integrated Quality Management System (IQMS) in selected public secondary schools in Giyani. Unpublished DEd thesis. Johannesburg: University of Johannesburg. [ Links ]
Maykut P & Morehouse R 1994. Beginning Qualitative Research — A philosophical and practical guide. London: The Falmer Press. [ Links ]
Merriam SB 1998. Qualitative research and case study — applications in education. San Francisco: Jossey-Bass. [ Links ]
Middlewood D & Cardno C (eds) 2001. Managing Teacher Appraisal and performance: A Comparative Approach. London: Routledge. [ Links ]
Middlewood D & Lumby J 1998. Human Resource management in Schools and Colleges. London: Paul Chapman. [ Links ]
Patton MQ 2002. Qualitative research and evaluation methods. 3rd edn. Thousand Oaks: SAGE. [ Links ]
Peattie L & Aldrete-Haas JA 1981. "Marginal" Settlements in Developing Countries: Research, Advocacy of Policy, and Evolution of Programs. Annual Review of Sociology, 7:157-175. [ Links ]
Sabatier PA 1988. An advocacy coalition framework of policy change and the role of policy-oriented learning therein. Policy Sciences, 21:129-168. [ Links ]
Steward DW & Shamdasani PN 1990. Focus groups — Theory and Practice. Newbury Park: SAGE. [ Links ]
Strauss AL & Corbin J 1990. Basics of Qualitative Research: Grounded Theory Procedures and Techniques. London: SAGE. [ Links ]
Timperley H & Robinson V 1997. The problem of policy implementation: the case of performance appraisal. School Leadership and Management, 17:333-345. [ Links ]