Entrenching performativity or enhancing pedagogy: Addressing the challenge of assessment policy and practice

 

 

Anil Kanjee^I; Nicky Roberts^II; Osman Sadeck^III; Jeanette Ramollo^IV

^IDepartment of Primary Education, School of Education, Tshwane University of Technology, Pretoria, South Africa anil.kanjee@gmail.com https://orcid.org/0000-0003-0161-0233
^IICentre for Education Practice Research, Faculty of Education, University of Johannesburg, Johannesburg, South Africa nroberts@uj.ac.za https://orcid.org/0000-0002-1910-0162
^IIIDepartment of Research and Postgraduate Studies, Faculty of Education, Cape Peninsula University of Technology, Cape Town, South Africa osadeck@gmail.com https://orcid.org/0000-0002-4887-6744
^IVDepartment of Primary Education, School of Education, Tshwane University of Technology, Pretoria, South Africa RamolloJK@tut.ac.za

 

 

---

ABSTRACT

The focus of the South African Education Research Association's Assessment and Testing Special Interest Group (SIG) is to contribute to current initiatives and debates pertaining to the development and implementation of assessment systems for improving learning and teaching. In particular, the SIG's members aim to address dominant performativity discourses impacting schools and universities by (1) providing a common understanding of the purpose and use of assessment, (2) locating the different assessment applications across the broader system within which learning and teaching occurs, and (3) highlighting recent initiatives impacting on assessment policy and practices. We think it essential to highlight critical policy and practice questions, while simultaneously acknowledging ongoing challenges for implementing enabling assessment systems that support the specific pedagogical needs of learners, teachers, students, and lecturers. Notwithstanding the complexities of effecting change, increasing discourse on, as well as relevant critique of, policies and practices that fail to improve learning and teaching, enhances possibilities for implementing enabling assessment policy and practice that seek to address the elusive challenge of equity and quality within the education system.

Keywords: assessment systems, performativity, formative assessment, assessment for learning

---

 

 

Introduction

A key output of the South African Education Research Association's Assessment and Testing Special Interest Group (SIG) has been the publication of a special issue to provide an additional platform for debate and to discuss the range of critical matters that have been raised by SIG members over the years. One underlying theme across different SIG sessions has focused on the tension between performativity requirements on the one hand, and the provisioning of effective support to improve learning and teaching within the schooling and higher education systems on the other. In particular, we sought to address dominant discourses that privileged performance over learning, and its implications on the expertise and experience of educators, teacher educators, policy makers, researchers, and scholars in their understanding (and effective use) of assessment evidence for addressing the challenge of quality and equity in schools and universities.

A review of the seminal assessment policy (Department of Education [DoE], 1998) implemented in the postapartheid era reveals that the underlying approach advocated was a shift from the dominant high-stakes examinations-based system in which performance and test scores were privileged over learning and teaching. However, several scholars have argued that this policy intent failed to materialise, and have highlighted the negative impact of performativity and test-based accountability policies and practices impacting on teaching and learning in South African schools (Chisholm & Wilderman, 2013; Jansen, 2001; Kanjee & Sayed, 2013). Jansen (2001) noted that these performance-based pedagogies privilege external behaviours that are easily codified into discrete outcomes that lead to the fragmentation of knowledge into that which can be measured. More critically, Jansen (2001) argued that these performance regimes also undermine fundamental commitments to equity with greater negative implications for the no-fee schools that serve the majority of learners from poor and marginalised backgrounds in South Africa.

In their review of assessment reform in the postapartheid era, Kanjee and Sayed (2013) reported that while notions of assessment for learning (AfL) and/or formative assessment have been promoted across different versions of the policy, the current system is strongly dominated by policies and practices that are assessment focused and measurement driven, citing the dominance of school-based testing and the national matriculation examinations. Chisholm and Wilderman (2013) highlighted similar concerns regarding the potential abusive use, and consequences, of both local and international standardised tests. Focusing on the curriculum reform process Muller and Hoadley (2019) reported that the current Curriculum and Assessment Policy Statements represent a performance-based curriculum that is highly prescriptive regarding the subject content to be covered, as well as the sequencing and pace of coverage. More recently, Bertram et al. (2021) also questioned the use of the Annual Teaching Plans for supporting curriculum implementation, arguing that this approach would result in teachers focusing on content coverage over learners' understanding of the content.

The introduction of the Foundations for Learning Campaign (DoE, 2008) and the Annual National Assessments (ANAs; Department of Basic Education [DBE], 2012) provided further evidence of accountability-based national programmes for improving the quality of education in South African schools (Kanjee & Sayed, 2013). Kanjee (2021) revealed that between 1994 and 2019, 95 large-scale assessment studies were implemented in South Africa for monitoring and evaluating the education systems or intervention programmes (see Figure 1).

These studies include international and regional studies (i.e., TIMSS¹, SACMEQ² ), national and provincial assessments (Grades 3 and 6 systemic evaluations, Western Cape systemic evaluations, ANAs) as well as studies by governmental organisations (e.g., Quality Learning Project 2004, Early Grade Reading Assessment). Further entrenching the dominant performativity regimes impacting on learning and teaching in schools was the introduction of Action Plan 2014³ (DBE, 2012), which prescribed the attainment of 13 key outputs, measured by 18 indicators, of which 12 were derived from national examinations as well as national and international learner assessment surveys.

Significantly, the National Assessment Directorate introduced the National Integrated Assessment Framework (DBE, 2017) to address several key limitations impacting on assessment policies and practices. These include (1) the dominant role of examinations, (2) the limited use of (AfL) approaches, (3) the poor quality of school-based assessment, and (4) the lack of a reliable indicator of system performance (Chetty, 2019; DBE, 2017; Mweli, 2018). Notwithstanding the challenges of effective implementation, the renewed emphasis on improving the assessment system, in particular the implementation of AfL approaches, is a positive step that has the potential to significantly impact improvements in teaching and learning. However, to date, there has been limited information on how the proposed framework will be implemented.

To contribute to current initiatives and discussions, while also accounting for the unprecedented effect of the Covid-19 pandemic, the following title was identified for this special issue: "Entrenching Performativity or Enhancing Pedagogy: Unpacking the Impact of Assessment Systems on Equitable and Quality Education." The intention is to stimulate debate around the follow areas:

• The understanding and use of assessment to improve pedagogical practices within the schooling sector, as well as in Initial Teacher Education Programmes (ITEPs)

• Innovative pedagogies that integrate the use of assessment and which challenge current performativity regimes

• The impact of the Covid-19 pandemic on assessment in schools and ITEPs.

The papers presented in this special issue make important and timely contribution, especially given current trends impacting on changing contexts of education systems, nationally and internationally, regarding learning, learning outcomes, and especially the impact of the Covid-19 pandemic. In contextualising the contributions submitted to this special issue, we highlight three key assessment themes. These themes, which also require further deliberation and debate, have a significant impact on teaching, learning, policy, and research, generally and particularly within schools and initial teacher education programmes:

1. Arriving at a common understanding of the purpose and use of assessment within the education sector.

2. Locating the different assessment applications within and across the broader system and subsystems within which learning and teaching occurs across schools and initial teacher education programmes.

3. An overview of recent initiatives, primarily by the DBE in South Africa, aimed at addressing several challenges impacting on assessment policy and practices.

 

Common understandings: Assessment purpose and applications

A review of current policy documents as well as relevant literature reveals a wide range of different terms used when discussing issues of assessment. These include summative assessment, formative assessment, formal assessment, informal assessment, continuous assessment, diagnostic assessment, dynamic assessment, assessment for learning, assessment of learning, assessment as learning, national assessment, regional assessment, international assessment, large-scale assessment, school-based assessment, classroom assessment, learner assessment, teacher assessment, systems assessment, and so forth. Given the different purposes and contexts within which these concepts are applied, the range of definitions provided, as well as the conflation of these concepts, it is not surprising that a range of different understandings and interpretations exists regarding the concept of assessment.

It is critical to acknowledge that current understandings and interpretations of the concept of assessment is highly contested. Moreover, its application and value within any education setting must account for both the specific contexts in which assessment policies and practices are applied as well as the worldviews and underlying rationales and assumptions of those who undertake the assessments and use their results. It is equally critical to ensure that current understandings and interpretations of the concept of assessment, as well as its application, not only portray specific meanings and significance but also add value to, and promote, relevant practices that address aims and objectives of education systems. Within education systems, assessments are intricately linked to pedagogy, and comprise a critical component of teacher and teacher educator daily practices.

It is within this context that we provided the following understanding and interpretation of assessment and its application in the education sector:

The primary purpose of assessment within the education sector is to obtain evidence for use in enhancing decisions regarding learning and performance. The use of this evidence, however, depends on the specific teaching and learning related decisions that the evidence is intended to support.

Across the education system, assessment evidence serves two primary purposes. In practice, assessments undertaken at the end of a unit of learning or course are known as summative assessments, also referred to as "assessment of learning" (AoL), with the purpose of producing a shared meaning among all users (Christodoulou, 2017). Thus, results emanating from such assessments can be used to (1) monitor and evaluate progress of learners or students, (2) inform learners/students, teachers/lecturers and other key actors of what learners/students have achieved, (3) certify completion of a course of programme, and (4) select learners/students (e.g., into the next education level, or for employment). In addition, along with other information, summative assessment results can also be used to monitor and evaluate the performance of institutions and organisations, for example, schools, districts, or teacher education programmes.

Assessment evidence used for improving teaching and learning, referred to as "assessment for learning" (AfL), comprises (1) formative assessment and (2) formative use of summative results (Kanjee & Bhana, 2020). In practice, the purpose of applying AfL is to produce an actionable consequence for learners/students and/or teachers/lecturers (Christodoulou, 2017). In highlighting the critical role of formative assessment, Black and Wiliam (2009) provided the following definition:

Practice in a classroom is formative to the extent that evidence about student achievement is elicited, interpreted, and used by teachers, learners, or their peers, to make decisions about the next steps in instruction that are likely to be better, or better founded, than the decisions they would have taken in the absence of the evidence that was elicited. (p. 9)

Several key characteristics of formative assessment that need to be highlighted are that it involves both learners/students and teachers/lecturers, it is conducted during the lesson, does not comprise the awarding of any marks, and is used to address specific learning needs of learners/students during the lesson. Importantly, Black and Wiliam (2009) argued that the failure to use or effectively apply assessment evidence to address specific needs of learners/students cannot be considered as assessment for learning. We note that these types of assessment refer to "assessment for nothing" or AfN. The introduction of this concept and acronym is intended to at least convey a message that collecting assessment evidence and failing to use this evidence to inform and improve teaching or learning processes amounts to a waste of time, and should be actively discouraged, especially given teachers' and teacher educators' large and often challenging workloads.

A related set of practices that has gained popularity in recent times pertains to what is known as "assessment as learning" (AaL), which Earl (2012) defined as the process where students act as active agents in the development and improvement of their own learning. In practice, AaL can be considered as an extension of the formative assessment process where learners take the primary responsibility to monitor what they are learning, to identify information they can use to improve their learning, and to use this information as feedback for themselves to improve their knowledge, understanding, and skills.

The formative use of summative assessments is defined as the process in which assessments conducted for summative purposes are also used to obtain evidence that can be used to improve the teaching and learning process. In practice, this means that results from summative assessments (e.g., class tests, final examinations, or marks on a project) are analysed and the evidence used to identify, and thereafter address, specific learning and teaching needs of learners/students and teachers/teacher educators. For example, a teacher educator would analyse the results of semester exams to identify areas in which students performed poorly and would use this information to (1) identify what they as the lecturer should do to improve how they teach the specific topic, and (2) provide feedback to students that address their specific learning needs.

 

Overview of an integrated assessment system

An integrated assessment system can be defined as a group of interrelated policies, practices, structures, and processes implemented by stakeholders at the different levels of the education system to obtain evidence about learner performance (Kanjee, 2008). In the context of the schooling system, Kanjee (2008) noted that an integrated national assessment system generally comprises four components (as noted in Figure 2).

Evidence of learners' learning comprises the core (denoted by the triangle) of the system and indicates the purpose of undertaking assessment within each of the four components. Emanating outward from the core, the first component, classroom assessment (denoted by the innermost circle) has a direct impact on the teaching and learning process and comprises formative assessments, class tests, projects, and assignments as well as internal examinations. The second component comprises external examinations followed by the third component, school evaluations, and the fourth component, large-scale assessments that manifest as provincial, national, regional, as well as international assessments. Each of these three components addresses specific assessment issues that have an indirect impact on the teaching and learning process, and their impact on learners, teachers, parents, education officials, and other education actors varies depending on how assessment evidence is applied. Furthermore, national policies, structures, and systems that define the education sector and the socioeconomic and cultural context in which learning and teaching take place within the sector also impacts on the assessment system. The different components of an assessment system are all interrelated and dialectically impact on one another. Consequently, they should not be viewed as discrete components that function independently and in isolation of other components in the assessment system as well as in the overall education system. The integrated assessment system can also be applied to higher education institutions, albeit within a different context.

Component 1: Classrooms

Classrooms comprise the first level of this system, denoted by the inner circle, in which assessments directly impact on pedagogical practices. Typically, assessment evidence is obtained by teachers and learners within the context of classroom-based activities and can be used for both formative and summative purposes. Formative assessment activities include teacher and learner questioning, classroom discussions, class work, as well as peer and self-assessment (Wiliam & Thompson, 2007). Regular summative assessment activities include assignments, projects, homework exercises, class tests, and internal examinations (which are linked to the second component and discussed below). The primary purpose of evidence obtained from both formative and summative assessments is to provide verbal and written descriptive feedback and/or to modify teaching practices to address specific learning needs of learners. The classroom level is the only component in which formative assessments can be implemented, together with the formative use of summative assessment evidence. In practice, assessment at this level is usually high stakes for learners and parents given the use of assessment evidence for addressing learners' learning gaps or selection into the next grade, and low stakes for teachers, school leaders, and education officials. Assessment activities at the classroom level apply equally to teachers in school classrooms and lecturers in universities or colleges.

Component 2: Examinations

The second component of an integrated assessment system comprises examinations, which have a direct, and often significant, impact on teaching and learning practices in classrooms. Examinations refer to the assessment taken under standardised conditions for determining a learners' mastery of relevant knowledge and skills covered. By definition, the function of examinations is to discriminate learners who are deemed to have or have not mastered the content being examined. In the education sector, two types of examinations are usually conducted: internal and external.

Internal examinations are conducted by the teachers, and are usually developed by teachers within a school, or in the case of common examinations, provided by education officials within a district or province. The primary purpose of internal examinations is to monitor and report on progress of learners as well as to select learners for promotion into the next grade. Internal examinations are also administered and graded by teachers within a school, reported on at the learner level, and are usually conducted on a quarterly, half-yearly, and/or annual basis. Common examinations, which have been introduced recently within many provinces and districts, also comprise a form of internal examinations.

External examinations are conducted by bodies outside the school to certify completion of a specific phase of education, for example, the National Senior Certificate in South Africa that certifies successful completion of 12 years of schooling or the Basic Education Certificate Examination in Ghana, which certifies successful completion of Form 3 (ninth grade). Key distinguishing factors of external examinations include setting of papers as well as marking, moderation and reporting undertaken by external bodies, usually national examinations boards, for example, Umalusi in South Africa or the West African Examinations Council in Ghana. Assessment evidence obtained from classroom-based summative assessments is also used to contribute to the final grade of an external examination across many countries. Given the high stakes nature of external examinations for all education stakeholders, and the use of examination results for accountability purposes, the form and format of modern day (external) examinations have taken a very different character that has had a profound impact on equity within the education system of many countries. In the higher education context, with regards to external examinations, across many disciplines, students are required to take specific credentialing examinations offered by different professional bodies, for example, Medical and Dental Professions Board or Law Society of South Africa.

Component 3: School evaluations

School evaluations comprise the third component of this system and refer to the process of gathering evidence about the effectiveness and efficiency of a school for improving learning and teaching (Kanjee, 2008). School evaluations are typically undertaken as part of the quality assurance processes within many education systems and are based on site visits conducted to review school polices, practices, and challenges as well as the performance of teachers, school management, and school governing bodies, for example, Whole-School Evaluation in South Africa or the School Evaluation Programme in Vietnam. In practice, a key indicator for school evaluations is learner performance, data that are usually obtained from assessment evidence that is already available (e.g., examination or test results), although it is also possible to undertake learner assessments as part of this evaluation process. Evidence obtained from school evaluations is usually used to determine whether schools need additional support or to identify additional interventions to support schools to improve teaching and learning. Within the context of most education systems, school evaluation activities have a critical impact on whether and how assessments are implemented, what gets assessed, who is involved in the assessments, and even the amount of funding available for key activities that determine how learning and teaching takes place, including the nature, content, and target audience of training and support in schools.

In essence, the school evaluation system as currently implemented can serve as a key lever for enhancing assessment systems and practices within the education system. For teachers, school leaders, and education officials, school evaluations are extremely high stakes given that such information is often used to identify poor performance and thus linked to sanctions or special interventions. School evaluations are similar to university institutional audits that are usually undertaken by teams of external evaluators on behalf of specific governing bodies, for example, the Council on Higher Education in South Africa.

Component 4: Large-scale assessment studies

The fourth component of the integrated assessment system comprises large-scale assessment studies (LSAS). Large-scale assessment studies refer to the process of obtaining evidence from an education system (or part thereof) on the performance of learners and other role-players (such as teachers, principals, education officials, parents) as well as on the functioning of structures and programmes within that system (Kanjee, 2007a). The primary purpose for undertaking LSAS is to obtain information to monitor and evaluate the functioning of various aspects within an education system, to make decisions about the need for interventions and for resource allocation, for enhancing public awareness, and for accountability purposes (Kanjee, 2007a). In practice, LSAS are usually conducted at the provincial and national levels within countries, or at the regional and international levels across countries. These assessments comprise the administration of standardised tests and questionnaires to learners and/or teachers, and may include questionnaires, site visits, and interviews with key education officials including the school principal, ministry officials, and other role-players in education. In practice, these assessments are administered to a sample of schools and learners or, in some countries, on an entire population of interest (census-based studies). However, several countries have also conducted national assessment studies in which data is only collected on learner performance, usually referred to as national assessments or national learning assessment studies. For education officials, results of LSAS can be extremely high stakes given the political implications as well as public responses to poor performance of the education system.

The Ghana National Education Assessment and the Grades 3, 6, and 9 Systemic Evaluation in South Africa as well as the regional and international assessment are all examples of a sample-based LSAS. Examples of census-based national assessments include the Annual National Assessments undertaken in South Africa, the System for the Measurement of Quality in Education (SIMCE) conducted in Chile, and the Systemic Tests implemented by the Western Cape Education Department. Regional LSAS currently conducted include the Southern Africa Consortium for Monitoring Educational Quality, the Program for the Analysis of Educational Systems of the CONFEMEN (Conference of the Ministers of Education of French-Speaking Countries), and the Latin-American Laboratory for Assessment of the Quality of Education (LLECE). International LSAS that many countries have participated in include the Trends in International Mathematics and Science Study, the Progress in International Reading Literacy Study, and the Programme for International Student Assessment.

Considering the various audiences, types and uses identified in Table 1 there are corresponding stakes for each component of the integrated assessment system, as then shown in Figure 3.

 

Impact of Covid-19

Globally and nationally, issues of assessment have featured prominently in reform initiatives across many education systems both prior, and in response, to the Covid-19 pandemic. This manifested primarily in pedagogical responses aimed at mitigating the impact of the pandemic on teaching and learning across the schooling and higher education sectors. Although the Covid-19 pandemic has exacerbated existing disparities impacting schools across the poverty quintile categories, it has also provided opportunities for key decision makers to address long-standing challenges that have plagued assessment systems for decades. Interestingly, responses in both Chile and South Africa have sought to address several challenges impacting on the assessment system while, in Ghana, no specific education initiatives were implemented that focused on the assessment system.

In South Africa, the key initiative within the DBE pre-Covid focused on the implementation of a national integrated assessment framework for improving the assessment system in the country. Proposals for such a framework called for introducing three complementary tiers within the General and Further Education and Training Band: systemic evaluation, examinations, and school-based assessment (Chetty, 2019; DBE, 2017; Mweli, 2018). The introduction of this framework is intended to address several limitations that hinder the effective use of assessment data for improving learning and teaching. These include the dominant role of examinations, the limited use of assessment for learning (AfL) approaches, the poor quality of school-based assessment, and the lack of a reliable indicator of system performance. The renewed emphasis on improving the assessment system, in particular enhancing the implementation of AfL approaches, is a positive step that has the potential to significantly impact improvements in teaching and learning

In their response to the Covid-19 pandemic, the DBE has implemented several reform initiatives to mitigate its impact on teaching and learning in schools. With regards to assessment, these methods have long been regarded as 'sacred' and 'untouchable.' Specifically, the new measures called for (1) limiting the number of summative assessments, including cancelling examinations for Grades 4 to 11, (2) greater use of diagnostic assessments, and (3) the implementation of formative assessment. More importantly, the DBE (2020b) have also provided detailed guidelines on the use of formative assessment, addressing a long-standing gap in the new assessment policies implemented in the postapartheid era. Not only do these reform measures suggest ways in which decision makers have begun to think differently about long-standing challenges that have plagued the system for decades, but they also have the potential to challenge and transform current systems and practices intended to entrench performativity regimes in the education sector. However, limited information is available on the impact on teaching and learning across schools in the different poverty quintiles.

Similarly in Chile, responses to the Covid-19 pandemic generated substantial revisions to their current assessment policy and practice. Critically, in a country that is known for its census-based high-stakes testing system (SIMCE), this assessment system was postponed after 30 years of continual operation (LLECE, 2020). Across several schools, this change created opportunities for pedagogical innovation allowing teachers to experiment with interdisciplinary work as well as for the application of formative assessment practices that were more connected to the daily lives of students and their families, (Mesa Social Covid-19 Educación, 2021). Moreover, a national report produced by a group of academics and teachers provided guidelines to support schools to work with formative assessment in the context of the crisis (Mesa Social Covid-19 Educación, 2021). This report also highlighted the possibility that the pandemic should be understood as a turning point in the way in which assessment policies and practices are conceived in Chile, with the aim of moving towards a system where a formative purpose is at the core of teaching and learning. Within this context, the Ministry has alluded to the possibility of implementing a low-stakes sample-based national curriculum assessment system aimed at informing policy and modifying the consequences and characteristics of the current quality assurance system. However, discussions are still ongoing, and there is no certainty about the direction that political debate will take in connection to assessment policies in the future.

In South Africa, within initial teacher education programmes, little is known about the impact of the pandemic on the assessment practices of teacher educators, especially given their dual roles of using assessment as lecturers/teachers as well as modelling and supporting students to develop their theoretical knowledge and practical skills in the effective use of assessment as future teachers. Across both the schooling and higher education sectors, calls for online or blended learning, have added an additional level of complexity to the assessment debates in how learners and students should be assessed. An additional challenge for many teachers and teacher educators pertains to the disparity in access to facilities and resources and, in particular, devices and the internet for learners and students from poor and marginalised communities. The articles in this special issue that focus on initial teacher education provide a starting point for ongoing reflections and debates.

 

The contributions in this special issue

The articles presented in this special issue need to be read within the context outlined above. They cover a range of issues impacting on the different components of the assessment system and address critical implications in the various applications and specific uses of assessment evidence. Although we present each article as providing an example of addressing key issues that focus on different components of the integrated assessment system (see Figure 4), we also acknowledge that in practice, these components are integrally intertwined and impact differentially on specific actors across the assessment system.

Integrated Assessment System: Components 1, 2, 3, and 4

The submission by le Grange, Simmonds, Maistry, Blignaut, and Ramrathan is a rhizome-article that is an assemblage of five heterogeneous essays intended to problematise standardisation, measurement, quantification and other technologies of performativity that dominate contemporary assessment practices in schools and universities. The article offers critiques of assessment practices in various sites in South African schools and higher education undergraduate and postgraduate programmes. The discussions span different grain sizes from high-stakes assessment such as the national senior certificate examination, to small examples of particular assessment criteria. The authors describe its form as an assemblage of five short essays on assessment that invokes two rhizomatic principles: connection and heterogeneity, and multiplicity. They argue that the rhizome figuration allows for reflections about assessment as not being fixed but rather in constant state of becoming. In so doing, they invite readers to challenge the current dominant performative nature of assessment and to re-imagine assessment possibilities for more humanising processes that invoke social justice. A key intention listed by the authors is to explore how to "open up avenues to move assessment more in a direction to accommodate equity aspirations in a structurally and socially unequal society such as South Africa?" Although four of the five essays focus on assessments in the higher education sector, and the fifth focuses on fairness in assessments, the issues discussed impact on all components of the assessment system.

In the first essay, Maistry, focusing on higher education, argues that a performativity discourse prevails and permeates pedagogy and assessment practices with the consequence that "assessment regimes degenerate into protocols that measure economic utility-producing knowledge and skills." He reflects on the assessment predicaments that presented in the abrupt transition to online higher education programme delivery, making two claims. First, that neoliberal ideology at work in South African higher education enabled the transition to the online space, which has nudged assessment systems in the direction of assessing what is easily measurable. Second, the neoliberal surveillance blueprint (of "monitoring, tracking, tracing, sorting, checking and systematic watching" of students) has morphed into "liquid surveillance" in the online space. He envisages response to these two changes to turn to the emerging body of literature of students as partners in higher education to find constructive ways in which faculty and students construct the teaching, learning, and assessment enterprise. He argues that this holds potential for reframing assessment towards more socially just, inclusive, and democratic practices in higher education.

In the second essay, Simmonds also reflects on assessment practices in higher education from her new position as "online lecturer." She frames her essay in relation to the inequities in the South African education and socioeconomic system, which have been exacerbated by the Covid-19 pandemic-evident in digital and resource divides as well as psychosocial impacts. Simmonds notes that the Covid-19 pandemic provides an opportunity to critically engage with new learnings and the complexity of learning, teaching, and assessment. She observes that institutions using, and students participating in, measurement-driven assessment hold assessment at a distance from teaching, learning, and the world to which this could relate. She documents her experiences as an online lecturer in an education system entrenched in performativity that is not premised on the ideals of social justice but, rather, on how her performance is being measured, how many of her students will pass, and on her meaningful use of assessment for learning and teaching. She comments on students having to take on full- and part-time employment alongside their full-time studies with the result that students do not participate in all learning opportunities but merely complete the assessments needed to pass the module. As an appropriately mindful response to these challenges, Simmonds offers the notion of diffraction-a critical practice of engagement with differences, how they matter, and for whom. Her hope is a shift from assessment as a tool to uncover preexisting disciplinary facts towards assessment with the generative potential to develop different kinds of knowledge such as trans-disciplinary knowledge. Simmonds argues that assessment, when viewed as "a complicated conversation," is more humane and socially just because it provides a productive space to ask difficult questions about assessment practices: how might they (1) be imagined as vectors of escape from dominant thinking, (2) embrace difference to open up alternative pathways for students to become, and (3) include more collaborative approaches contributing to knowledge-making that is socially just, affective, and mindful?

In a third essay, Blignaut draws on his experience as a lecturer of honours and master's students. He argues that students' conceptions of assessment lead to shallow learning, which is informed by their instrumental/technicist notions of learning in which they regard themselves as clients paying for a service. Students seem to be neither curious for learning, nor invest the time needed to read, and are not willing to engage in assessment as learning processes. Noting the increasingly narrow focus on assessment for qualification in higher education, Blignaut questions whether the tests are fair given the very different obstacles that students have had to overcome to sit for these tests. In response to these observations of students' views on assessment, he argues for an extended notion of epistemological access that provides for all knowledge forms so that epistemological access subsumes epistemic/cognitive justice.

Next, Ramrathan reflects on his work in a higher education institution trying to shift away from an audit culture. Here, he conceptualises an audit culture where assessments are used for grading and ranking for external stakeholders towards using assessment to provide feedback to students on their progress, and to provide staff feedback on the effectiveness of their teaching programmes. Ramrathan first exemplifies this with an attempt to offer students three possible levels of attainment on their teaching practice: fail, pass, or pass with distinction. After initial uptake there was resistance from students and from the university assessment capture system, with both advocating for grades as percentages. His second example makes explicit the expected differences between undergraduate and postgraduate assessment practices: from exposure to the knowledge base and established processes of inquiry to inquiry-based processes that generate knowledge. Yet, postgraduate students found it difficult to accept a proposed lack of particle marking for referencing (either they can reference or they can't). As a proposed response to the quantification of assessment, Ramrathan offers assessment as learning. He suggests that the determination of pass or fail could include a quantitative and a qualitative element, proposing a gradual shift over time from the former to the latter.

Finally, le Grange focuses his reflection on fairness, noting that fairness has received less attention than reliability and validity in assessments. He argues that fairness ought not to be viewed simply in technical terms relating to assessment design-standardising particular aspects such as administration processes and assessment criteria-but rather, be viewed as ethical practice that can contribute to social justice concerned both with what precedes the assessment as well as its consequences. Preceding assessments are the educational opportunities and resources together with their out-of-school environments, and the consequences are assessments that depend on how assessments are interpreted, and their effect on learners' life chances. Noting the bi-modal distribution of South African National Senior Certificate results, le Grange argues that the prior experiences of learners should be taken into account to allow for a more nuanced interpretation of results (particularly for university entrance) where currently, principles of equality (rather than equity) are applied.

Component 1: Classrooms

Three articles discuss an aspect of the core component of the integrated assessment system- classroom evidence of learning. All three submissions address a critical issue impacting the education sector raised by Beets (2007), who argued for reframing assessment to support efficient learning so that:

developing the ability to use useful and timely feedback in order to understand and recognise quality with the aim of improving performance and fostering self-regulation-will open up possibilities for lecturers to develop a better understanding of students' learning and to reflect on the effectiveness of the teaching practice involved. (p. 581)

Nakidien, Sayed, and Sadeck reflect on the efficacy of a continuous professional development (CPD) programme implemented to enhance teachers' use of formative assessment in no-fee schools. These authors provide valuable insights for consideration as the DBE implements its Assessment for Learning pedagogical strategy in South African schools (DBE, 2021). The authors note that the CPD programme not only served as an alternative to the prevailing dominant views in which assessment is reduced to a process of testing, but also sought to be developmental and transformative. Thus, the programme was based on the principles of participant involvement and engagement that sought to promote reflective and reflexive practices. The CPD programme was implemented in a cyclical process in which teachers were provided with both opportunities and support to try out what they had learnt at the workshops in their classes then assess and reflect on its implementation. This process allowed participants to determine what, how, and why the model worked so that they could make informed decisions on their own adoption and implementation.

Moreover, to enhance effective development and implementation of the assessment strategies and techniques, and to promote sustainability, the participants included members of the school management team as well as district officials responsible for supporting teachers; materials were specifically designed to promote the model of action-reflection. The authors note that their study represents schools functioning in "fragile contexts," which required that the CPD be both practical and sustainable. Notwithstanding the positive response from participants, the authors note several challenges impacting the programme and highlight the importance of accounting for the context within which learning and teaching occurs, especially for schools in fragile contexts.

Enu and Ngcobo locate their study within the realm of classroom practice, focusing specifically on formative assessment in the context of higher education. The authors report on mathematics teacher educators' conceptions of assessment knowledge and understanding, and how this translates to practice when engaging their students. The authors highlight the value and merits of the assessment policy in Ghana, noting its emphasis on the use of formative assessment over summative assessment and, in particular, foregrounding feedback for improving learning and teaching. However, drawing attention to the disjuncture between policy and practice, the authors acknowledge that examination-orientated assessment practices dominate current practice in Ghana, and they raise specific concerns about such practices in the teacher colleges. Aptly presenting exemplars coupled with detailed explanations, Enu and Ngcobo conclude that the participants' considered effective feedback as a means to highlight knowledge gaps, and that their feedback practices were premised on the development of procedural knowledge at the expense of supporting students to self-regulate their learning and thereby become owners of their own learning. The findings of their study have serious implications for initial teacher education programmes, and the extent to which teacher educators are able to support student teachers develop the requisite practical and theoretical knowledge and skills to effectively contribute to the schooling system beset by a myriad of challenges, foremost of which pertains to supporting all learners to address their specific learning needs.

In their submission, Roberts and Maseko also focus on teacher educators' assessment practices, located at the classrooms level within the higher education context. However, in contrast to Enu and Ngcobo, their study explores the formative use of summative results to improve the teaching and learning process. Roberts and Maseko argue that there is a paucity in meaningful reporting and effective utilisation of evidence from assessments administered in initial teacher education programmes. The authors demonstrate how data intended for use in measuring the impact of an intervention programme could be applied to determine the specific learning needs of student teachers.

Similar to Enu and Ngcobo, Roberts and Maseko utilise exemplars and detailed explanations that demonstrate how the results obtained from an online summative assessment were applied to identify students' level of proficiency. These results also served as evidence for determining specific content areas in which students demonstrated improvement, and areas in need of assistance. Of specific interest was the finding indicating that the majority of students did not use calculators when responding to specific items even though the online assessment was not implemented under standardised examinations conditions. At a time when the use of online learning and assessment has received sustained focus with ranging views on issues of validity and reliability of results as well as privacy, security, and rights of students dominating current discourse (see submissions by Maistry and by Simmonds in this issue), this finding has added significance, not only for teacher educators but for instructional and assessment practice within the entire higher education sector.

Components 3 and 4: Large-scale assessment and school evaluations

Flórez Petour reflects on the policy process impacting on the national assessments in Chile, arguing for a reconceptualising of policy to account for the multiple voices of key actors in the system to ensure the assessment system focuses on improving learning and teaching. Interestingly, the Chilean System for the Measurement of Quality in Education (SIMCE) provides an example in which assessment evidence obtained from a census-based national assessment system is used for monitoring and evaluating the functioning schools as well as for rewarding or sanctioning schools and teachers. The article by Flórez Petour offers much food for thought for the South African context. The South African pivot-away from annual national assessments towards a greater focus on assessment for learning (DBE, 2017, 2021)-offers a counter trajectory in the national assessment system seen in Chile. Flórez Petour reflects critically on the assumption that large-scale assessment systems necessarily have a positive impact on teaching and learning and, therefore, on the quality of education as a whole. She argues that research evidence about the effect of these assessment systems on the improvement of learning results is not conclusive and, therefore, questions the assumption that accountability mechanisms (based on standardised assessments) drive increased learning. Flórez Petour notes the numerous negative impacts familiar to the South African audience such as teaching to the test, narrowing of the curriculum, teacher burnout and stress for school communities, discrimination and gaming practices in schools, stigmatisation of schools that serve more socioeconomically disadvantaged students, the lack of motivation towards pedagogical innovation and the search for success formulas, and the feelings of frustration and demotivation in school communities with low results.

The Chilean situation sees high-stakes accountability mechanisms in place with schools categorised in relation to levels of performance, publicly ranked, performance-based funding allocations, and even teacher salaries or employment being linked to assessment outcomes. South African readers ought to take note that the market-orientated coupling of standardised national assessments with funding mechanisms at school and individual teacher level are functional in Chile. Flórez Petour notes the significant disconnect between policy design (by "expert" policy actors) and policy implementation (by apparently "less expert" teachers). Her analysis highlights distance and lack of dialogue between these actors and posits that intelligent accountability, where principles such as trust and mutual responsibility replace the top-down and control-oriented approaches, would be preferable. A similar analysis of the assessment policy environment in South Africa-considering basic education and the initial teacher education policy frameworks-would be worth conducting. As in the Chilean case, we must ask how we can move to greater parity of participation where "better and healthier connections between large-scale assessment systems and the enhancement of pedagogy" may be more likely.

Component 4: Assessment surveys

The final contribution focuses on large-scale assessments, demonstrating how the formative use of summative evidence is applied to address issues of equivalence and fairness. Roux, van Staden, and Pretorius investigate the use of differential item functioning (DIF) techniques to determine linguistic and culture equivalence of results when reporting results of international assessments involving different subgroups. This article is an important and timely contribution, especially within the South African context, given the current implementation of Systemic Evaluation studies by the DBE (2017). Previously, Roberts and Barmby (2016) have argued for strengthening the design of Annual National Assessment, posing the question: How can investments in large-scale assessment be leveraged to focus attention on better learning, better teaching, and better assessment? It is precisely these types of questions that need to be addressed by the DBE in their Systemic Evaluation studies.

Given increasing concerns raised regarding issues of fairness and social justice in recent years (see article by le Grange in this issue), DIF studies have taken on greater significance given the heterogenous nature of student populations participating in the assessments. In the context of the limited number of similar studies undertaken within the South African education context (Bansilal, 2015; Kanjee, 2007b; Mtsatse & van Standen, 2021), this paper provides a stark reminder of the value of DIF studies. Moreover, given the significant impact of language on learning and learner performance, especially within the context of South Africa, it is critical to ensure that assessments in which learners take part do not unfairly advantage any one sub-group. A key point to note is that results of DIF studies only indicate instances where test items are functioning differentially across two or more sub-groups compared, and cannot detect bias as such (Kanjee, 2007b). However, the potential for using similar DIF analyses to explore underlying reasons for differential performance may provide invaluable insights that impact on how support is provided to learners for whom meaning and interpretation of text may differ as a function of language, culture, or gender.

 

Conclusion

Developing and implementing enabling assessment systems that provide valid and reliable information for use in improving learning and teaching has been an ongoing challenge that many countries have yet to attain. Several factors continue to impact on how assessment policies and practices across and within educations systems are understood and implemented. Key among these include the tension between performativity requirements on the one hand, and the provision of effective support to address the challenge of equity and quality on the other. In practice, this often leads to performance being privileged over learning, given the dominant discourse of curriculum coverage as learning, and the assessment-focused, measurement-driven policies that privilege examination and test results over learning. Other factors include inadequate professional development opportunities for developing the requisite assessment knowledges and skills of key actors across schooling and higher education systems, the lack specialised courses on assessment (in particular, those focusing on the technical areas of testing and measurement) within initial teacher education programmes, and the absence of a national assessment framework or policy guidelines.

However, issues of assessment have featured prominently in national responses aimed at mitigating the impact of the Covid-19 pandemic on teaching and learning across both the higher education and schooling sectors. Although the Covid-19 pandemic has exacerbated existing disparities impacting schools, it has also provided opportunities for key decision makers to address long-standing challenges that have plagued assessment systems for decades. Notwithstanding the complexities of effecting change in education and assessment systems, increasing discourse on, as well as challenging policies and practices that fail to support, the learning needs of all learners and students bodes well for developing enabling assessment systems that seek to address the elusive challenge of equity and quality within the education system.

 

References

Bansilal, S. (2015). A Rasch analysis of a Grade 12 test written by mathematics teachers. South African Journal of Science, 111(5/6), 1-9. https://doi.org/10.17159/sajs.2015/20140098        [ Links ]

Beets, P. A. D. (2007). (Re)positioning assessment in higher education: The case of geography in South Africa. South African Journal of Higher Education, 21(4), 577-584.         [ Links ]

Bertram, C. A., Mthiyane, C. C. N., & Naidoo, J. (2021). The tension between curriculum coverage and quality learning: The experiences of South African teachers. International Journal of Educational Development, 81(10), 1-8. https://doi.org/10.1016/j.ijedudev.2021.102353        [ Links ]

Black, P., & Wiliam, D. (2009). Developing the theory of formative assessment. Educational Assessment, Evaluation and Accountability, 21(1), 5-31.         [ Links ]

Chetty, M. (2019, July 29-30). The Department of Basic Education 's perspective on GET assessment [Paper presentation]. Mpumulanga Department of Education School-Based Assessment Lekgotla, Witbank, South Africa.

Chisholm, L., & Wildeman, R. (2013). The politics of testing in South Africa. Journal of Curriculum Studies, 45(1), 89-100. https://doi.org/10.1080/00220272.2012.755710        [ Links ]

Christodoulou, D. (2017). Making good progress? The future of assessment for learning. Oxford University Press.

Department of Basic Education. (2012). Action Plan to 2014: Towards the realisation of Schooling 2025-Full version.

Department of Basic Education. (2017). The development of a systemic evaluation model for the basic education sector.

Department of Basic Education. (2020a). Action Plan to 2024: Towards the realisation of Schooling 2025-Full version.

Department of Basic Education. (2020b). National Assessment Circular 3 of2020: Implementation of formative assessment in the General Education and Training (GET) band. https://irp-cdn.multiscreensite.com/c0cc1c10/files/uploaded/NA%20Circular%2003%20on%20Formative%20Assessment.pdf

Department of Basic Education. (2021). DBE and the PEDs place school principals at the forefront in implementing the Three-year Curriculum Recovery Plan.https://www.education.gov.za/ThreeYearRecoveryPlan0421.aspx

Department of Education. (1998). Assessment policy in the General Education and Training phase: Grades R-to 9 and ABET (Government Gazette No. 19640).

Department of Education. (2008). Foundations for Learning Campaign 2008-2011 (Government Gazette No 30884).

Earl, L. M. (2012). Assessment as learning: Using classroom assessment to maximize student learning. Corwin Press.

Jansen, J. D. (2001). On the politics of performance in South African education: Autonomy, accountability and assessment. Prospects, 31(4), 553-564.         [ Links ]

Kanjee, A. (2007a). Improving learner achievement in schools: Applications of national assessments in South Africa. In S. Buhlungu, J. Daniel, R. Southall, & J. Lutchman (Eds.), State of the nation: South Africa 2007 (pp. 470-499). HSRC Press.

Kanjee, A. (2007b). Using logistic regression to detect bias when multiple groups are tested. South African Journal of Psychology, 37(1), 47-61.         [ Links ]

Kanjee, A. (2008, November 19-21). A framework for developing national education assessment systems: Challenges and prospects [Paper presentation]. The National Policy Dialogue Forum on Developing Effective Assessment Systems, Human Sciences Research Council, Pretoria, South Africa.

Kanjee, A. (2021, May 26). Reflections on the journey for implementing Assessment for Learning in South African schools [Master's class seminar]. Department of Education, University of Oxford.         [ Links ]

Kanjee, A., & Bhana, J. (2020). Using formative assessment to improve learning and teaching: Practical guidelines for teacher during the COVID-19 pandemic. Oxford University Press.

Kanjee, A., & Sayed, Y. (2013). Assessment policy in post-apartheid South Africa: Challenges for improving education quality and learning. Assessment in Education: Principles, Policy &Practice, 20(4), 442-469. https://doi.org/10.1080/0969594X.2013.838541        [ Links ]

Latin American Laboratory for Assessment of the Quality of Education. (2020). Latin American education systems in response to COVID-19: Educational continuity and assessment. UNESCO. https://unesdoc.unesco.org/ark:/48223/pf0000374018

Mesa Social Covid-19 Educación. (2021). Recomendación para una evaluación pertinente en tiempos de crisis [Recommendations for pertinent assessment in times of crisis]. University of Chile. http://www.ie.uchile.cl/download.php?file=noticias/00_1625517751.pdf

Mtsatse, N., & van Staden, S. (2021). Exploring differential item functioning on reading achievement: A case for isiXhosa. South African Journal of African Languages, 41(1), 1-11.         [ Links ]

Muller, J., & Hoadley, U. (2019). Curriculum reform and learner performance: An obstinate paradox in the quest for equality. In N. Spaull & J. D. Jansen (Eds.), South African schooling: The enigma of inequality (pp 109-125). Springer Nature.

Mweli, M. (2018, April). Improving assessment practices [Presentation]. The Director General's meeting with the Kwazulu-Natal Provincial Education Department.

Roberts, N., & Barmby, P. (2016, January 12-15). Analysing Annual National Assessments to improve both teaching and assessment practices [Paper presentation]. The 24th Annual Conference for the Southern African Association for Research in Mathematics, Science and Technology Education (SAARMSTE), Tshwane University of Technology, Pretoria, South Africa.

Wiliam, D., & Thompson, M. (2007). Integrating assessment with learning: What will it take to make it work? In D. Dwyer (Ed.), The future of assessment: Shaping teaching and learning (pp. 53-82). Lawrence Erlbaum Associates.

 

 

Received: 25 May 2022
Accepted: 6 July 2022

 

 

1 Trends in International Mathematics and Science Study (https://www.timss-sa.org/).
2 Southern and Eastern Africa Consortium for Monitoring Educational Quality (http://www.sacmeq.org/).
3 Updated to Action Plan 2024 (DBE, 2020a) though the goals and the indicators remain unchanged.