GENERAL ARTICLES

 

Possibilities for long-term shifts in higher education assessment praxis: reflecting on COVID-19 as a stimulus for change

 

 

L. Dison^I; K. Padayachee^II

^ISchool of Education, Faculty of Humanities. University of the Witwatersrand, Johannesburg, South Africa. https://orcid.org/0000-0002-1626-4954
^IIScience Teaching and Learning Unit, Faculty of Science. University of the Witwatersrand, Johannesburg, South Africa. http://orcid.org/0000-0001-7015-5962

 

 

---

ABSTRACT

It is widely recognized that assessment impacts on the process and behavior of learning. In this article, we, as academic staff development professionals in two faculties at a research intensive South African university, explore the assessment challenges, processes and behaviours that emerged in the context of Emergency Remote Teaching and Learning during the COVID-19 pandemic. We argue that an analysis of changes in assessment culture and behaviour point to possibilities for a shift from the pre-COVID-19 dominance of the "assessment of learning" paradigm, to an orientation of assessment where both "assessment of learning" and "assessment for learning" are more equitably balanced, with potentially profound implications for shaping the ways students construct their understandings and succeed academically.

Keywords: assessment for learning, COVID-19, assessment culture, assessment paradigms, access, success

---

 

 

INTRODUCTION

It is widely recognized that assessment impacts on the process and behavior of learning. The move away from conventional concerns of measuring achievement and assuring assessment standards in recent years has highlighted the crucial role of assessment in facilitating students' active and critical engagement with epistemologies in their fields of study. Assessment scholars have increasingly (Sambell, McDowell, and Montgomery 2013; Carless 2015) asserted the place of assessment in the core business of teaching and learning and have gone further to counter traditional grading systems (Blum 2020) and emphasize the building of students' capability to make informed evaluative judgments (Boud and Falchicov 2007). Of the many challenges faced by lecturers and students during the shift to emergency remote teaching (ERT) and learning during COVID-19, there is growing evidence to suggest that assessment has been the most significant pedagogical driver of effective online learning. In this article we, as academic staff development professionals in two faculties, explore the assessment challenges experienced by lecturers and students during ERT, as well as the assessment innovations that emerged during this period. We argue that an analysis of changes in assessment culture and behavior point to possibilities for a shift from the pre-COVID-19 dominance of the "assessment of learning" paradigm, to an orientation of assessment where both "assessment of learning" and "assessment for learning" are equally addressed and more equitably balanced, with potentially profound implications for shaping the ways students construct their understandings and succeed academically.

This study aims to investigate the constraints and enablements of implementing assessment characterized by better balance and more alignment between assessment for and assessment of learning during and beyond COVID-19. It develops an integrative approach for understanding the principles of these assessment shifts and for identifying the conditions under which sustainable and transformative assessments can be achieved. Harari (2018) argues that it is no longer about digesting large quantities of information in the knowledge economy but rather the development of critical thinking skills that enable students to engage thoughtfully and creatively with knowledge, and work with knowledge in new and socially relevant just ways.

 

ASSESSMENT AS A SOCIAL SYSTEM

Butler Shay (2004) argues that assessment is a socially situated interpretive act involving multiple stakeholders. These stakeholders may be internal to the institution (students, lecturers, teaching assistants and administrators) or externally located, e.g., funders, prospective employers and, to some extent, parents. At institutional level, assessment is also regulated by national and institutional policies and frameworks. There thus exists a culture of assessment that goes beyond a shared community of meaning in which different individuals and groups within the system draw and elaborate upon components of this system in line with their vested interests and projects.

A cultural system is usually multifaceted and dynamic, and it is generally difficult to identify when or how stakeholders (cultural agents) are able to act to bring about change within it. The result is an unavoidable conflation of actors, their behaviours and actions, and the various structural factors that influence these. Archer (2013) offers the social realist conception of "analytical dualism" as a potential solution to this conflation, theoretically separating the "parts" and the "people" even though, in practice, they do not act independently. Archer argues that in order to understand the interplay between the "parts" and the "people" each has to be accorded distinct properties and powers which need to be understood in order to understand the dynamics of the larger cultural system.

In viewing assessment as a cultural system, it is necessary to define the parts as well as the people, in order to understand the complex interplay between these, and how this interplay manifests in assessment behaviour and outcomes. The parts may be defined by the structures such as national and institutional policies, examination procedures and processes around which assessment practices are organised, and the requirements of various external accreditation bodies. These structures are however, based upon the underlying views of policy makers and society at large. We posit that the traditional view of assessment has been one in which assessment is viewed as an end point in the curriculum, to determine how well students have developed particular competencies or achieved particular outcomes. From this view point, assessment structures would be crafted around the measurement - an assessment paradigm referred to as "assessment of learning" (Boud and Falchikov 2007).

 

RECONCILING ASSESSMENT PURPOSES

The paradigm of assessment of learning is one that can be traced back almost 2000 years, to the Imperial Examinations of ancient China, for the selection of suitable capable candidates for Imperial positions within the Han dynasty. Its roots can also be traced back to the early ivy league universities such as Oxford and Cambridge in the 1600's. As pointed out by Boud and Falchikov (2007) assessment practices for the purposes of certification still dominate modern assessment systems, locked in a paradigm of performativity and underpinned by the strong link between assessment and the quality of the qualification / graduates' abilities. It is also embedded in the personal epistemologies of lecturers and students and the inevitable impact on future job prospects is arguably what has kept assessment locked in a certification paradigm, with assessment practices slow to change as a result:

"Assessment affects people's lives. The future directions and careers of students depend on it. There are risks involved in changing assessment without considering the consequences. This has meant that there has been very slow movement in the development of new assessment ideas and changes in practice." (Boud and Falchikov 2007, 3).

In the last decade however, there has been a predominance of literature on the transformative role of assessment in higher education, especially in the context of the move away from assessment of learning or as measurement (Boud and Dochy 2010; Sambell et al. 2013). The traditional way of seeing assessment as an end point in the curriculum to determine how well students have learnt or achieved a particular outcome (with implications for institutional reputation and status, indicated in blue on the right hand side of Figure 1), has not kept up with the conception of students as an integral part of curriculum, pedagogical and assessment design processes in preparation for life-long learning and a sustainable future (indicated in red, to the left of Figure 1).

However, conceptions of assessment should not be seen as a binary, serving either the purpose of certification on one hand, or the purpose of learning and self-regulation on the other. Instead, assessment should be viewed as serving a "double duty", as described by Boud and Falchikov 2007, with certification and learning mutually reinforcing.

This notion is reinforced by Race (2020) in a critique of the dominance of assessment for certification and quality assurance purposes, pointing out the gulf that exists between the intended learning outcomes as published in course outlines, and what is actually measured by the traditional exams usually favoured by lecturers. Race points out that learning by doing processes that are fundamental to learning are severely curtailed even if students study actively before an exam. This is because students may be required to learn aspects of the curriculum that are not fundamental or threshold concepts (Meyer and Land 2005), and that the exam writing process itself usually does not involve learning by doing.

We extend this argument by suggesting that the processes involved in preparing for assessments, performing the assessments (usually in invigilated exam formats), or learning by doing generally do not extend or deepen conceptual understandings significantly, and that the evidence shows that for many summative assessment tasks, students have been encouraged to rote learn material and memorise concepts without significant application or higher order thinking and creativity, even when assessment tasks require students to engage critically with ideas and integrate information from multiple sources into their argument.

Despite these arguments (that, in our observation, often find strong resonance with academics), invigilated assessments have remained the dominant method of assessment (with the pendulum stuck on the far right in most instances). This is due largely to the view that assessment is mainly for certification purposes, and that invigilated exams are the most valid and reliable way to achieve this. A recent study (Harrison et al. 2017, 2) revealed that many academics share the belief in the "primacy of the summative assessment paradigm" which preclude possibilities for "radical redesign solutions". However, COVID-19 caused a drastic swing in the pendulum to the far left (red), when invigilated exams as the only option for certification of individual competence was no longer possible. In our institution, as in many others, this shift precipitated a flurry of discussions and debates on assessment purposes and strategies, thereby opening up the space for rethinking assessment practices.

In this article, we explore some of these discussions and debates and reflect on the ways in which emergency shifts in assessment manifested in various ways in our organisation over the past year. We consider the conditions and circumstances that triggered these shifts, and how these emergent practices have illuminated previous assessment blind spots for some academics, while highlighting possibilities for a revitalised and more contextually relevant conceptualisation of assessment in higher education beyond COVID-19, for others. We have structured our reflection using Archer's social realist framework to illustrate the constancy of assessment as a social practice prior to COVID-19, and the dialectical relationships between institutional structures and cultures and individual agency over the past 2 years, that has led to the potential for structural and cultural elaboration of the assessment system.

 

SOCIAL REALISM

Archer explains that at any given moment in time, structural elements within a social system will either constrain and enable agents, whose actions deliver intended and unintended consequences. This interplay, in turn, leads to structural elaboration and the reproduction or transformation of the existing structures, or maintenance of the status quo. In this interaction, the role of human agency is a critical element. Extending this to assessments, when academics (and students) act with intentionality in the face of assessment constraints and enablements, their responses to structural features like policies, resources and leadership, and cultural features such as departmental principles and practices, may result in the stasis or transformation of assessment practices.

A social realist approach therefore provides an appropriate analytical framework to enable a nuanced analysis of the responses and emergent practices in the assessment landscape in two faculties at a research intensive suburban university in South Africa. What follows is our description of the state of assessment structures during 2021, a year after COVID-19 first forced us into ERT. We then move reflect on the reasons for the differences that manifested, considering the role of various actors in eliciting the changes that were observed, and the kinds of social interactions and structural adaptations that led to these changes. At the same time, we critically evaluate the affordances and drawbacks of the changes, and the consequences for the various actors within the assessment system, with special attention paid to lecturers and students.

Our deliberation eventually brings us back to the structural conditions that existed prior to the pandemic, that resulted in the assessment challenges that were experienced in the past year. Here we draw attention to some of the perceived difficulties stemming from the beliefs and values associated with the traditional assessment paradigm. We then illustrate, by way of reflection on current practices coming to the fore in 2022 (as COVID-19 becomes more endemic), how long term sustainability of emergent thinking and practices around assessment emergent gains risk being lost, and how long term changes to assessments will only be achieved when there are significant shifts in the personal beliefs of lecturers and students regarding the purpose and value of assessments. Our concluding argument is that ultimately, it is not a paradigm shift that is necessary, but rather more explicit and deliberate balancing and integration of "assessment of learning" and "assessment for learning" practices.

 

REFLECTING ON EXISTING ASSESSMENT PRACTICES DURING ERT

The COVID-19 pandemic disrupted academic programmes in ways that could not have been predicted by universities. The changing circumstances triggered an unprecedented shift in modality of teaching and learning and consequently on assessment practices. Lecturers had to move their traditional timed and invigilated examinations to online exams and take-home assessments which are flexibly timed and where students have access to their books and course notes. At our institution, assessment policies and rules (standing orders) at the structural level were adjusted to accommodate the more flexible arrangements and requirements of take-home assessments and online exams and specific disciplinary needs. Promisingly, there appeared to be an increasing emphasis on integrating sound pedagogical principles into assessment practices for formative and summative purposes. However, there was also the emergence of a strong discourse around cheating and plagiarism, and various structural measures to curb these dishonest behaviors and practices amongst students. Underpinning the pre-occupation with academic dishonesty (or rather academic integrity), was (is) the increasing sense of unease amongst lecturers that the shift to online learning and online assessment was accompanied by serious compromises in quality, specifically in relation to validity and reliability of assessments. We discuss these behavioral trends and practices next, focusing on how these emerged in the past two years.

Assessment challenges and consequences for student engagement during ERT

Before analysing the key shifts in assessment of the past year, it must be acknowledged that these fundamental changes in assessment modalities presented many challenges for students and lecturers in a context characterised by high student numbers, severe technological challenges, students' inadequate readiness for university, and an entrenched culture of summative assessment that favours assessment for the purpose of accountability.

Apart from the issues of data and connectivity, for many students, the loss of embodied spaces became evident in their written work, which revealed the absence of the influence of rich conversations with peers and tutors. In our observations and from conversation with many lecturers, students struggled to connect and participate in synchronous and asynchronous discussion forums and appeared unprepared for independent or autonomous learning, a capacity which tends to develop in contact courses over an entire degree programme, and occurs as a result of iterative cycles of engagement with knowledge (in its various forms and modes), and critical self-reflection. This process requires time for individual reflection and thinking about learning through writing and discussion in order for students learn from their mistakes to improve in subsequent tasks. However, it became strikingly apparent that time was not always consciously considered by the lecturers in course and assessment design, nor was it factored in by students, many of whom found themselves falling behind on learning and assessments.

Lecturers also expressed their frustration about no longer receiving everyday clues in the lecture hall as to how their students are progressing, usually gauged through facial expressions in class, or hands raised to ask questions at various interactive webinars and discussion forums on online teaching and assessment. They bemoaned the fact that they were no longer physically present with students to answer their questions about assessment tasks and to discuss the related criteria. They indicated their concerns that students appeared not to be sufficiently prepared to take more responsibility for their learning and in deciding when and where they should prepare for their assessments. As a result, some lecturers have since begun to resist online teaching because they perceive it to be non-interactive, and contradict possibilities for meaningful learning, especially for students already struggling to meet the demands of academic and digital literacy challenges. The result has been a quick reversion to traditional face-to-face teaching and assessment methods for some in 2022.

Short-term "damage-control" strategies

At the start of emergency remote teaching, it was easier for lecturers to make rapid short-term changes to online assessment such as changing unseen timed exams to multiple choice questions, tweaking exam questions for longer open book exams and simply recycling essay and assignment questions. While staff were trained to use the learning management system, they were not experienced in online teaching methods or pedagogies or in how to prepare students for engaging with online teaching. Lecturers thus faced the challenge of having to adjust their pedagogical knowledge and skills to an unfamiliar mode of teaching and for engaging students in unfamiliar online and digital spaces, within a context of extreme uncertainty, unpredictability and anxiety.

Responsiveness to "redesign solutions"

It is striking that despite these inevitable frustrations and challenges, several lecturers in the two faculties used these learning opportunities to reflect on and re-conceptualise their course design, teaching and assessment practices during ERT. Once they had come to terms with the realities of ERT, they began to think more deeply about what they were trying to achieve, how their courses were structured to achieve their intentions, and what assumptions and values underpinned their teaching and assessment. These lecturers realised that they could not simply transfer traditional assessments into online form and became more open to the transformative possibilities of online assessments. These realisations and possibilities for changing practices was supported by additional funding given to different the faculties to enhance their staff development capacity to train lecturers and tutors to develop sustainable online and/or blended assessment strategies. The new professional learning opportunities offered as a result, improved some lecturers' online proficiency and confidence and gave rise to an ongoing process of reflection on the learning affordances and constraints of ERT, and an examination of the extent to which online teaching and assessment could enable engagement and learning.

 

REFLECTING ON KEY PEDAGOGICAL SHIFTS: LINKING ASSESSMENT AND ENGAGEMENT

Innovative assessment practices

Dealing with the above challenges, lecturers became open to embracing new forms of decolonial assessment practices that were arguably more authentic and responsive to the realities of South African higher education, that allowed for "disruptive conversations about knowledge" (Sayed, De Kock and Motala 2019), and that would allow for greater epistemic access particularly for students from under-resourced backgrounds. This was in keeping with students' views of the need to decolonise assessments as a core component of curriculum decolonisation during the #FeesMustFall protests of 2015 and 2016 (Padayachee, Matimolane, and Ganas 2018). In the more recent context of COVID-19, Brown (2020a website) declared at the start of ERT, that the "hunt was on for immediate alternative ways to assess students in this academic year". She hoped that this would herald a permanent shift away from the "ineffectiveness, fragility and inauthenticity of traditional exams". Brown (2020b) further posited that ERT would be an opportunity to enact transformative and pedagogically driven assessment principles built on decades of higher education research, while Swaffield (2011) and Villarroel et al. (2018) suggested it would be an opportunity to move towards more authentic, meaningful () tasks that foster student engagement, motivation and self-regulation. It would place as much emphasis on the product and process of learning and had the potential to elicit higher order thinking and metacognition. Czerniewicz et al. (2020) presented a similar argument that the pandemic appeared to have accelerated teaching and assessment innovations in South African universities. For instance, the affordances of take home assessments and portfolio or continuous assessments, that were used more extensively during ERT, promotes cognitive challenge and evaluative judgement in ways that traditional exams cannot (see below). However, it also became apparent to some lecturers in the two faculties, that they needed to explicitly signal to students that these new assessments would require a higher quality of learning and thinking than the "cram pass forget" strategies that many students adopt for handling traditional exams (Sambell and McDowell 1998). There was a recognition from their participation in more authentic assessment tasks, that students would be able to see the relevance of what they were doing to their future lives and selves in a rapidly changing labour market. They would therefore be more likely to demonstrate the knowledge and capabilities contained within the specified learning outcomes of their courses.

Despite these new insights and shifts in assessment praxis, it also became clear that sustainability of innovation and change in the longer term requires ongoing professional teaching and learning support within the schools and faculties, with greater emphasis in these on the design of assessment tasks, assessment criteria, marking rubrics, allocation of marks and feedback strategies in online spaces. Facilitating these processes of reflection and change requires deliberate and integrated support from key support structures, including senior management, deans, academic development staff and learning experience designers, as well as changes in assessment plans and policies. In our context, this realisation precipitated a revision of the Senate Standing Order on Assessment, paving the way for long term shifts towards more balanced assessment practices that enable all students to participate more equitably and learn more effectively.

Lecturers' critical reflective practice

Since the start of the pandemic, cross faculty webinars, staff workshops and extensive guidelines and materials on how to promote assessment for learning in teaching programmes have given staff opportunities to share their insights and demonstrate critical self-awareness of their innovative approaches. Staff have been given opportunities to consider how to adopt new technologies to redesign assessments for the online learning environment, and to shift towards approaches of assessment for learning, even in high stakes summative assessments. Fundamental assessment questions were posed to stimulate discussion:

• What do I want learners to know?

• What should be evaluated or assessed?

• Why is it important that this specific aspect should be evaluated or assessed?

• What assessment approaches/ strategies will provide the best opportunity for demonstrating learning and/or attainment of competence?

• How will assessment strategies will best reflect course outcomes?

These webinars also allowed staff to become familiar with the principles of learning-oriented assessment (Carless 2015) for supporting students under conditions of emergency remote teaching, a key principle being the integration between formative and summative assessment. In particular, lecturers drew on their resources to find creative and innovative ways to change the culture of participation in the online learning environment and to address the divides between those who have access and those who do not. They also had to interrogate the purposes of each assessment task as well as their overall assessment strategies, critically examining the assumptions and values underpin their thinking and design.

It has since become clear that most lecturers will never be able to return to the same ways of teaching and assessing before COVID-19, and will not return to their short-term assessment methods or content overloaded lectures of the past. Lecturers have also begun to realise that rather than trying to replicate face to face teaching in online spaces, it is more effective to use these online spaces to facilitate important conversations about enhancing high quality learning. This realisation shows a shift for some, from the surface level Substitution and Augmentation phases of the SAMR model of technology integration in teaching and learning (Puentedura 2010; 2013), to embracing the possibilities of substantive shifts through Modification and Redefinition of assessments, mediated by technology.

Rethinking formative and summative assessment

Summative assessments such as end of term invigilated exams are generally thought of as the sets of tasks and activities that confirm the achievement of learning outcomes, or tasks that enable measurement of competency and / or quality, i.e., assessment of learning (reviewed by Boud and Falchikov 2007). As mentioned earlier, it is this dominant view of assessment, rather than assessment as an integrated part of the teaching and learning process, that underpins the challenges being faced by lecturers during the switch to ERT and the requirement to design take home summative assessments. There is also the perception of online assessments as lower quality because of the limited types of questions that can be set (generally believed to be questions that can only assess lower order cognitive domains), and concerns about cheating and the possibility that online submissions may not be true representations of students' competencies and understanding.

Many of the assessment for learning webinars we conducted, and the guidelines produced (Padayachee and Dison 2020), have emphasized that formative assessments that are used as part of the learning process can be integrated systematically and build up towards summative assessments, whether they are ungraded or have a low point value. They provide opportunities for students to self-monitor with instant feedback and provide multiple opportunities for students to try again (retake the quiz) or improve subsequent tasks until their understanding is clarified.

Bloxham and Boyd (2007) have outlined a number of assessments for learning principles which involve a mixture of formative and summative assessment activities. Key amongst these is the recognition of the value of tasks that are cognitively challenging and that require students to integrate knowledge they have learned on the course with their experiences and knowledge from other contexts. Other core principles are that assessment should not come at the end of learning but should be an integral part of the learning process; students are involved in self-assessment and reflection on their learning; assessment expectations should be made visible to students as far as possible; tasks should be authentic, worthwhile, relevant and offer students some level of control over their work. In other words, effective online learning, like learning under any circumstances, requires the seamless integration of teaching activities with formative and summative tasks.

The process (not only the product) of learning

An important assessment principle assessment noted by Bloxham and Boyd (2007) is that assessment should encourage metacognition and promote thinking about the learning process and not just the learning outcomes. Race (2020) confirms this notion by arguing that an effective learner is one who makes informed and strategic decisions about "where to invest time and energy", and that this needs to be acknowledged in assessment tasks and criteria. Lecturers who realised and embedded this principle were able to use their online assessments to enable students to self-monitor or assess their own performance. In the ERT learning environment that was often defined by limited teacher presence, such mechanisms for identifying learning target and self-evaluating progress became critical for many students.

As mentioned earlier, authentic assessments also came to the fore during ERT as lecturers embraced and embedded several forms of realism (not only related to "employability" and the development of professional skills). Emergent forms of assessments, especially portfolios and learning journals, provided opportunities to represent the way information is synthesized and applied in real life situations and encouraged an appropriate level of cognitive challenge and the development of evaluative judgment. Adoption of more authentic assessments also provided more opportunities for more explicit integration, assessment and development of the "ways of thinking and practice" (Barradell, Barrie, and Peseta 2018) for specific disciplines, and more explicit induction of students into their professions by introduces a real-world aspect as well as assessing and teaching the concepts and skills that the graduate will need to function in this field outside the academy. These lifelong skills are appropriate for the labour market and highlight the importance of discussions on diversity, inclusion and equity. Authentic assessment tasks proved particularly useful in practical/ applied disciplines where evidence could be provided in diverse forms including text, image, video, audio, practice notebooks scientific reports, conference posters, narrated PowerPoint presentations, reflective commentaries/accounts, critical incident accounts, mind maps, flow charts, case studies, annotated bibliographies, mind maps and so on. During ERT, more lecturers in Humanities and Sciences also introduced rationales into portfolios of evidence and research reports, as a space where students think through and explain why they made the choices they did in the task. However, there was also the realisation, as pointed out by Carless (2015), that for this aspect to be effective, the criteria to which students have to perform need to be very clearly laid out and understood.

Dialogic feedback practices

A key approach to developing criticality and the capacity to self-regulation is dialogic feedback (Winstone and Carless 2019). However, giving students meaningful feedback on their learning is an area that has presented multiple challenges in the ERT spaces. Lecturers realised the need to increase the complexity of their online questions in objective multiple-choice formats and performance tasks in disciplines. They wanted to move students beyond description levels of analysis to achieve high level cognitive, performance and personal outcomes and recognised that they require skills to design learning-oriented objective assessments. The advantage of automated assessment tools like tests and quizzes is that students receive their feedback instantly and can learn from them to correct their misconceptions. A caveat here however, is that these types of questions and the general mode of feedback provided by automated assessment tools can sometimes rob students of the personalised and dialogical feedback that builds criticality, interest and self-regulation. A potential consequence is that students may again lean towards a focus on the mark and the right answer, without reflecting on why their answers were wrong and learning from the feedback.

If online feedback is to play a strongly formative role in achieving student success (Winstone and Boud 2019) even for low stakes rewards, structures need to be in place to support the various actors in eliciting these changes online. Through cognitive, social-affective, and structural feedback dialogues, students can be positioned as equal partners in a reciprocal process, thus "moving away from teacher-dominated forms of communication and enabling students to solicit and engage in feedback interactions" (Winstone and Carless 2019, 97). Stommel (2020) discusses his "authentic assessment" approach centring around self-evaluation and reflection in which he asks students to "engage directly as experts in their own learning". They are asked to reflect regularly on their own progress by drawing on examples of their own work to illustrate different facets of their learning.

A more process-oriented approach to feedback underpinned by a sociocultural view has come to the fore as it focuses on making sense of feedback information and using it for enhancement purposes (Boud and Molloy 2013; Carless and Boud 2018). This view of feedback emphasises the student's role in generating, processing, and responding to feedback information. For this to be actualised, students require exposure to feedback literacy: the understandings, capacities and dispositions needed to make the most of feedback opportunities (Carless and Boud 2018). In practical terms, the shift to students owning process-oriented feedback has necessitated designing forms of student dialogue into feedback. The limitations of teacher-telling forms of feedback, and the impracticality of one-to-one dialogues at scale, has also required attention to how these dialogues take place (e.g., across time and space, utilising technology, as noted in Telio, Ajjawi, and Regehr 2015), and who they take place between (e.g., involving peers as in Nicol, Thomson, and Breslin 2014).

Various strategies have been suggested for involving learners in making informed judgements on how their learning is progressing, and how well they are becoming able to provide evidence of their achievement in each of the forms which will make up their overall assessment. Feedforward, for example, is a strategy that aims to increase the value of feedback to the students by focusing comments on what the student might aim to do, or do differently, in the next assignment or assessment if they are to continue to do well or to do better (Hounsell et al. 2008).

To conclude this point, for students to be able to apply feedback, they need to understand the meaning of the feedback statements and focus on aspects of their work that need attention. Students need support to use feedback as a basis for strengthening their writing and their engagement in peer assessment.

Developing students' capacity to reflect

A key assessment principle that has emerged during ERT is to assist students to develop the capacity to reflect in courses when they set goals at the beginning of the semester, and the extent to which these have been achieved at the end of the semester. Online teaching specialists have drawn on blended and online principles from Laurillard et al. (2013) for allowing students to meaningfully participate in online discussions. They stress the value of including participation as part of the grading scheme, especially in threaded discussions and motivate lecturers to identify the qualities they look for in meaningful discussions in their disciplines as a basis for formulating assessment criteria.

There also appears to be a far greater reliance on reflection on practice to authenticate written work with the recognition that reflection is not a skill that happens automatically but has to be built into the programme. Reflective writing can be a powerful means of enabling students to demonstrate complex learning outcomes including critical thinking. The personalized nature of the portfolio is viewed as a mechanism for "designing out" plagiarism, by promoting a sense of student voice/ownership.

Students are expected to show originality and creativity alongside their mastery of subject knowledge and to make use of digital formats that are amenable to tracking dialogic feedback processes over time development. Bain (2010, 14) has suggested a model of assessment within a critical pedagogy framework that: supports partnerships in assessment that leads to "empowered autonomous learners" and provides opportunities for student voice that supports the students' growing ability to think critically about - and take the responsibility for - "their own assessment".

 

PRAGMATIC CONCERNS

The prevailing mantra during ERT has been that "lecturers need to measure less, but measure it better" using a wider range of evidence of achievement and flexibility to accommodate circumstantial uncertainty. However, the emergent strategies discussed in section 2 need to be considered in relation to the overall burden for lecturers and students. A range of solution-oriented tools have been proposed such as using audio feedback to record comments, using group feedback mechanisms and giving feedback to a sample of students or in staggered form throughout the term. Such "minimal marking" approaches (Hyland 1990) have become effective for empowering rather than overwhelming students by enabling students to identify and remedy a few key issues in their own writing. Lecturers and tutors are taught to prioritize structural and argument-related aspects of writing before they consider surface-level aspects as well. These effective strategies for diversifying assessment require extended training for both staff and students and need to be bolstered by educationally sound policies and teaching plans. Another issue that emerged with the shift in assessments strategies during ERT is plagiarism and academic dishonesty. Race (2020) points out that "the difficulties associated with plagiarism are so severe that there is considerable pressure to retreat into the relative safety of traditional, unseen written exams once again, and we are coming round full circle to resorting to assessment processes and instruments which can guarantee safety regarding 'whodunit'?" Plagiarism and academic dishonesty, based in student agency as well as the deeply entrenched marks driven culture, could thus trigger a shift back to pre-COVID assessment strategies dominated by quality assurance concerns and institutional reputation.

 

DISCUSSION: CAN EMERGENT ASSESSMENT PRACTICES BE SUSTAINED BEYOND ERT?

Our reflections on emergent assessment practices are suggestive of significant structural changes, such as changes in assessment policies, and of course, changes in the physical nature of assessments from face-to-face, invigilated examinations to online, open book and take home assessments. Universities have had to relook at the way they interact and engage with students in changing learning spaces. The South African Education system, through its Higher Education Qualification Sub-Framework (Department of Higher Education and Training, 2013), highlights the need to meet the demands of the diverse student population as increasing numbers of students enter higher education. One of the greatest learnings during ERT is that lecturers need to understand students' learning circumstances much better and carefully consider issues of physical and epistemological access when designing assessments. These factors have implications for flexible course and task design as students navigate these challenging times.

Also highlighted in the past year is the crucial need to draw on sound assessment concepts and processes to enable student learning and better results. Progressive policies and practices will go some way towards addressing the tension between using assessment to stimulate student learning and the ways in which assessment is used for accountability purposes by institutions. Also highlighted during ERT is the need to think deeply about what counts in our methods of assessment in our teaching programmes. Lecturers need to re-consider the purpose informing their task design and assessment formats, e.g., whether the purpose of a multiple choice test is for students to demonstrate their basic understanding or more advanced application of threshold concepts. These are the kinds of deeper questions and reflections that will enable emergent practices to become embedded in the long term.

The emergent thinking and practices, therefore, show some indications of shifts towards greater balance between assessment for, and assessment of learning. In other words, there appears to be a window of possibility for a more integrated approach to assessment, rather than the dominance of the "assessment of learning" paradigm. However, ongoing concerns about quality (especially in terms of assessment validity, reliability, and fairness), issues of academic integrity, student cheating and plagiarism, as well as concerns about student access and social justice, appear, to reflect a lingering value for assessment of learning. In other words, although structural changes occurred in terms of mode and methods, for many lecturers, their individual agency remains rooted in deep seated beliefs about the purpose of assessment being evaluative. Similarly, these concerns related to quality are also indicative of an assessment culture that is still conditioned by pre-COVID-19 ideologies and views of assessment being required for quality control, gatekeeping and credentialing. From this perspective, a long-term shift towards greater balance between assessment for learning and assessment of learning is perhaps less likely.

The potential backtrack to the traditional "assessment of learning" paradigm may be explained in terms of the rapid, emergency nature of the shift to online assessments, with lecturers not having been afforded the time to reflect on and to deeply sense the need for a shift in practice. Instead, lecturers were forced rapidly into prototyping new methods, with no time to reflect on the need for changes (beyond the context and urgency of COVID-19). As a result, in most instances, the assessment changes were reactionary and performative, rather than being transformative. Scharmer (2009) suggests, with reference to the theory of personal change and transformation called Theory U that changes underpinned by old habit of mind are unlikely to stick, since the unchanged beliefs and values will inevitably drive a reversion to old practices.

The experiences of COVID-19 have exposed students and staff to many new possibilities and provided the grounds for "prototyping" novel assessment methods, which can now be refined and adapted in a more intentional and strategic way, with student learning foregrounded to a much greater extent than before. For lecturers who embraced the possibilities and gained new insights through prototyping during COVID-19, the new assessments methods may persist in the long term. However, for those lecturers whose assessment practices remain underpinned by a strong quality assurance purpose, continued enactment of assessment strategies that emphasise more balance between assessment for and of learning, is less likely.

 

CONCLUSION

As educators in higher education spaces, the events occurring during ERT during the covid-19 pandemic have prompted us to think deeply about the end goals of assessment and what it means to transform assessment practices in the context of vastly different student realities. In this article we have highlighted several steps that were taken by lecturer to shift assessment practices taking recognising the prevailing context with its structural, cultural and agential constraints and enablements. We highlighted the catalytic role that COVID-19 played in requiring lecturers to question their assumptions about the role of formative and summative assessment and feedback and to interrogate the ways in which assessment can facilitate or block epistemological and ontological access. In our view, what is needed next is the evaluation of emergent thinking and practices in relation to issues of massification, commodification, decolonisation, access and quality more broadly, and the structural conditions that are required to sustain new practices in the long term. This will enable lecturers to conceptualise and enact assessments in a vastly different way, leading to the adoption of assessments strategies that synergistically integrate the different purposes of assessment.

 

REFERENCES

Archer, Margaret Scotford. 2013. Social morphogenesis, Vol. 1, ed. Margaret Scotford Archer. Dordrecht: Springer.         [ Links ]

Bain, Jennifer. 2010. "Integrating student voice: Assessment for empowerment." Practitioner Research in Higher Education 4(1): 14-29.         [ Links ]

Barradell, Sarah, Simon Barrie, and Tai Peseta. 2018. "Ways of thinking and practising: Highlighting the complexities of higher education curriculum." Innovations in Education and Teaching International 55(3): 266-275.         [ Links ]

Bloxham, Sue and Pete Boyd. 2007. Developing Effective Assessment in Higher Education: A Practical Guide. McGraw-Hill Education (UK).         [ Links ]

Blum, Susan. 2020. Ungrading: Why rating students undermines learning (and what to do instead). West Virginia University Press.         [ Links ]

Boud, David and Filip Dochy. 2010. Assessment 2020. Seven propositions for assessment reform in higher education. Sydney: Australian Learning and Teaching Council.         [ Links ]

Boud, David and Nancy Falchikov. 2007. "Introduction: Assessment for the longer term." In Rethinking assessment in higher education, 3-13. Routledge.         [ Links ]

Boud, Davids and Elizabeth Molloy. 2013. "Rethinking models of feedback for learning: The challenge of design." Assessment and Evaluation in higher education 38(6): 698-712.         [ Links ]

Brown, Sally. 2020. "Assessment, Teaching and Learning in Higher Education." https://sally-brown.net.         [ Links ]

Butler, Shay Suellen. 2004. "The assessment of complex performance: A socially situated interpretive act." Harvard Educational Review 74(3): 307-329.         [ Links ]

Carless, David and David Boud. 2018. "The development of student feedback literacy: Enabling uptake of feedback." Assessment and Evaluation in Higher Education 43(8): 1315-1325.         [ Links ]

Carless, David. 2015. "Exploring learning-oriented assessment processes." Higher Education 69(6): 963-976.         [ Links ]

Czerniewicz, Laura, Najma Agherdien, Johan Badenhorst, Dina Belluigi, Tracey Chambers, Muntuwenkosi Chili, Magriet De Villiers, et al. 2020. "A wake-up call: Equity, inequality and COVID-19 emergency remote teaching and learning." Postdigital Science and Education 2(3): 946-967.         [ Links ]

Department of Higher Education and Training. 2013. General and Further Education and Training Qualifications Sub-Framework and Higher Education Qualification Sub-Framework of the National Qualification Framework. Pretoria: Government Printer.         [ Links ]

Harari, Yuval. 2018. "Yuval Noah Harari on what the year 2050 has in store for humankind." https://www.wired.co.uk/article/yuval-noah-harari-extract-21-lessons-for-the-21st-century.         [ Links ]

Harrison, Christopher J., Karen D. Könings, Lambert W. T. Schuwirth, Valerie Wass, and Cees P. M. Van der Vleuten. 2017. "Changing the culture of assessment: The dominance of the summative assessment paradigm." BMC Medical Education 17(1): 1-14.         [ Links ]

Hounsell, Dai, Velda McCune, Jenny Hounsell, and Judith Litjens. 2008. "The quality of guidance and feedback to students." Higher Education Research and Development 27(1): 55-67.         [ Links ]

Ken, Hyland. 1990. "Providing productive feedback." ELT Journal 4(4):279-285.         [ Links ]

Laurillard, Diana, Patricia Charlton, Brock Craft, Dionisios Dimakopoulos, Dejan Ljubojevic, George Magoulas, Elizabeth Masterman, et al. 2013. "A constructionist learning environment for teachers to model learning designs." Journal of Computer Assisted Learning 29(1): 15-30.         [ Links ]

Meyer, Jan H. F. and Ray Land. 2005. "Threshold concepts and troublesome knowledge (2): Epistemological considerations and a conceptual framework for teaching and learning." Higher Education 49(3): 373-388.         [ Links ]

Nicol, David, Avril Thomson, and Caroline Breslin. 2014. "Rethinking feedback practices in higher education: A peer review perspective." Assessment and Evaluation in Higher Education 39(1): 102-122.         [ Links ]

Padayachee, Kershree and Laura Dison. 2020. "University of the Witwatersrand Important considerations for take home assessments." University of the Witwatersrand, Johannesburg. https://ulwazi.wits.ac.za/courses/189/files/1181/download?wrap=1.         [ Links ]

Padayachee, Kershree, Mapula Matimolane, and Rita Ganas. 2018. "Addressing curriculum decolonisation and education for sustainable development through epistemically diverse curricula." South African Journal of Higher Education 32(6): 288-304.         [ Links ]

Puentedura, Ruben. 2006. "Transformation, technology, and education in the state of Maine." http://www.hippasus.com/rrpweblog/archives/2006_11.html.         [ Links ]

Puentedura, Ruben. 2013. "SAMR: Moving from enhancement to transformation." http://www.hippasus.com/rrpweblog/archives/000095.html.         [ Links ]

Race, Phil. 2020. "Assessment for learning." https://phil-race.co.uk        [ Links ]

Sambell, Kay and Liz McDowell. 1998. "The construction of the hidden curriculum: Messages and meanings in the assessment of student learning." Assessment and Evaluation in Higher Education 23(4): 391-402.         [ Links ]

Sambell, Kay, Liz McDowell, and Chris Montgomery. 2013. Assessment for Learning in Higher Education. Routledge.         [ Links ]

Sayed, Yusuf, Tarryn de Kock, and Shireen Motala. 2019. "Between Higher and Basic Education in South Africa: What Does Decolonisation Mean for Teacher Education?" In Decolonisation in Universities: The Politics of Knowledge, ed. Jonathan D. Jansen, 156-180. Wits University Press. https://doi.org/10.18772/22019083351.         [ Links ]

Scharmer, C. Otto. 2009. Theory U: Learning from the future as it emerges. Berrett-Koehler Publishers.         [ Links ]

Stommel, Jessie. 2020. "Ungrading: An FAQ." Personal blog.         [ Links ]

Swaffield, Sue. 2011. "Getting to the heart of authentic assessment for learning." Assessment in Education: Principles, Policy and Practice 18(4): 433-449.         [ Links ]

Telio, Summer, Rola Ajjawi, and Glenn Regehr. 2015. "The 'educational alliance' as a framework for reconceptualizing feedback in medical education." Academic Medicine 90(5): 609-614.         [ Links ]

Villarroel, Veronica, Susan Bloxham, Daniela Bruna, Carola Bruna, and Constanza Herrera-Seda. 2018. "Authentic assessment: Creating a blueprint for course design." Assessment and Evaluation in Higher Education 43(5): 840-854.         [ Links ]

Winstone, Naomi and David Boud. 2019. "Exploring cultures of feedback practice: The adoption of learning-focused feedback practices in the UK and Australia." Higher Education Research and Development 38(2): 411-425.         [ Links ]

Winstone, Naomi and David Carless. 2019. Designing effective feedback processes in higher education: A learning-focused approach. Routledge.         [ Links ]