Proctoring as a human substitution for online summative assessments in a comprehensive open distance e-learning institution: Opportunities and obstacles

 

 

Lorette Jacobs^I; Siphamandla Mncube^II

^IUniversity of South Africa, South Africa
^IIUniversity of South Africa, South Africa

 

 

---

ABSTRACT

At a comprehensive open distance e-learning (CODeL) university in South Africa, assessment practices were converted and offered online during and post COVID-19. Innovative proctoring methods were applied to secure the authenticity of online summative assessments. However, the use of such proctoring tools to ensure ethical behaviour of students during summative assessments remains questionable, as these tools are not infallible. A qualitative approach was applied to explore ways in which online proctoring tools can contribute toward ensuring authentic summative assessments. Through the application of collaborative autoethnography, the researchers obtained insight into opportunities and obstacles that may influence the effective use of online proctoring tools to support summative assessment activities. Findings indicate that students would go to great lengths to consult unauthorised material and share answers during summative assessments. Therefore, to enhance perceptions of ethical conduct in higher education, it is recommended that ethical principles are expanded through extended training and guidance with revolutionised proctoring technologies.

Keywords: proctoring tools, COVID-19, artificial intelligence, comprehensive open distance e-learning, ethical behaviour

---

 

 

INTRODUCTION

The advent of the COVID-19 pandemic has resulted in drastic changes in higher educational institutions. Globally, COVID-19 hastened the adoption of online learning, teaching and assessment practices (Hussein, Yusuf, Deb, Fong & Naidu, 2020). These changes have been experienced in all spheres of higher education, where social distancing and fewer human interactions influenced teaching, learning, assessment and research practices. In the circumstances, to increase student' engagement in higher education contexts, academics were encouraged to revise and move to online platforms to apply teaching and learning practices. Though a blended mode was followed by many institutions of higher learning pre COVID-19, all had to adapt to a fully online environments, inclusive of online assessments, during the pandemic. Clay (2020) explains that the role of assessment in the online environment has become even more important to shape students' behaviour and open opportunities for students to become active, engaged and independent learners. Towards supporting advanced learning, online assessments must illustrate authenticity to prepare students for completing tasks related to module outcomes; must be accessible to all students, irrespective of available technology tools; and be secure, to support the authentication of levels of competency (Pefialov, Corell, Abella-Garcia & Grande-de-Prado, 2020). Added to the above requirements, Holden, Norris and Kuhlmeier (2021) emphasise the importance of ensuring academic integrity as part of online assessment practices. The authors explain that values such as honesty, trust, fairness, respect and responsibility should become foundational in the execution of assessment practices.

However, Lee and Fanguy (2022) argue that where students are under pressure to perform, values such as honesty, integrity and responsibility are often difficult to maintain. Free access to online sources, varied opportunities to communicate with others or to cheat, become increasingly prominent. Towards ensuring the trustworthiness and integrity of online assessments, and specifically online summative assessments, higher education institutions had to consider alternative practices (Weleschuk, Dyjur & Kelly, 2019). Proctoring tools became a key component of online summative assessment practices. According to Coghlan, Miller and Paterson (2021), the term proctoring refers to any online invigilator applications that can authenticate the identity of a student and the originality of the assessment submitted by each individual student. Online proctoring solutions claim to efficiently monitor students' assessment practices in an online environment by combining artificial intelligence (AI) algorithms and human examiners (Coghlan et al., 2021). Where human invigilators were used prior to COVID-19 (especially during summative assessment practices) to monitor the authenticity of assessment activity executions, such invigilators were then replaced by software in the online environment (Thombare, Sapate, Rane & Hutke, 2022).

In as much as proctoring tools offer opportunities to monitor the quality of assessment practices, these tools are not foolproof and do not prevent the execution of unethical behaviour during summative assessments (Lee & Fanguy, 2022). Utilising technology to monitor student behaviour and ensure the integrity of the assessment process in higher education has become the norm and has expanded during and post the COVID-1 9 period, according to Stephens (2021). The view that advocates the expansion of technology to ensure the integrity of online assessments supports the argument by Woldeab and Brothen (2019), that teaching, learning and assessment in the 21^st century is impossible without the use of varied technology tools. Whilst striving towards ensuring the integrity and trustworthiness of assessment practices, proctoring tools offer solutions to detect anomalies by monitoring students whilst engaged in summative assessment activities. Linked to the use of proctoring tools, the aim of this research is to explore opportunities and obstacles in the use of such tools at a comprehensive open distance e-learning (CODeL) university in South Africa.

 

CONTEXTUALISING THE RESEARCH

Hussein et al. (2020) state that interest in online assessment practices and the use of proctoring tools have increased since the COVID-19 pandemic. Despite this interest, Kharbat & Daabes (2021) state that more research is required to explore the impact of proctoring tools on online examination experiences. Aligned to this need for additional research, the emphasis of this study is to identify the challenges and opportunities related to the use of online proctoring to support the integrity of summative assessment practices at a CODeL institution. A CODeL higher education institution refers to a multi-dimensional institution aimed at promoting the objectives of online learning, to overcome distance, temporal and spatial barriers (Manyike, 2017). Openness is important to provide students from diverse educational and socio-economic backgrounds with the opportunity to engage in further and lifelong learning. E-learning brings a new dimension to the concept of open learning to overcome traditional barriers to education. The possibilities of unlimited access to information and global communication offered via e-learning, provide students with opportunities to control and direct their own learning. To promote e-learning, the CODeL framework is founded on the premise that student learning can be optimally supported by modern electronic technologies, where multiple teaching, learning and assessment strategies and a range of technologies are used, combined with the deployment of physical and virtual resources, to encourage active engagement with students (Heeralal, 2015). The selection of technologies is determined by cognitive, affective and systemic structures (Fynn & Janse van Vuuren, 2017). In terms of cognitive support, technology is utilised to optimise access to resources, such as literature and formal learner support services related to tutorial classes, engagement with academics and peers and assessment practices. Affective support refers to the improvement of the study environment to build commitment and self-esteem, whilst systemic support is concerned with the technology infrastructure to enable students to experience comprehensive learning, engagement and support.

Pertaining to cognitive support towards enhanced assessment practices via technology, the transition from venue-based to non-venue based summative assessments was exacerbated by the outbreak of the COVID-19 pandemic (Hussein et al., 2020). Higher education institutions implemented alternate assessment methods such as timed tests, take-home exams, randomised multiple-choice exams, and online portfolio examinations, by using various technology tools (Majola & Mudau, 2022). However, ensuring the integrity of such online assessments, according to Noorbehbahani, Mohammadi and Aminazadeh (2022), require that practices of cheating and unethical behaviours of students be minimised. Therefore, many higher education institutions adopted proctoring technologies (Lee & Fanguy, 2022). The use of proctoring tools became imperative, according to Swart (2015) as well as van Breda & van Wyk (2018) to minimise cheating, assessment anomalies, and unauthorised help to students during summative assessments. Hussein et al. (2020) postulate that for online proctoring to function effectively, academics must explore ways in which proctoring tools can be used efficiently This include ascertaining experiences related to the positive use of proctoring tools, as well as obstacles that may result from the use of proctoring tools. As explained by Holden et al. (2021), such understanding will improve the use of proctoring tools as part of ensuring the integrity of summative assessment practices.

 

BRIEF LITERATURE REVIEW

The emerging online summative assessment strategies are achievable through the availability of technological developments. These include computers, laptops, digital cameras, headphones, and smartphones, as well as AI tools such as Scikit Learn, Keras and TensorFlow. AI tools offer machine-learning opportunities and allow for the creation of neural networks and graphical visualisation that can run on computers, Android and iSO devices (UNESCO, 2022). Through a combination of technology tools, classroom instructions and assessments can be expanded to foster engagement, creativity, and knowledge exchange (Phuthela & Dwivedi, 2020). Online assessments, and in particular summative assessments can create opportunities to support problem-based learning, and to encourage students to engage actively and creatively in solving real-world problems that they may encounter in their different professions (Gilbert, 2022). Online summative assessment practices also save money in that students in an ODeL context do not need to travel to examination centres to engage in summative assessments to sit for their examinations. Money is also saved on printing and logistical costs. Academics can build databases of questions that may be posed to students on a rotational basis; thereby minimising the opportunity for students to collaborate during summative assessments (Kharbat & Daabes, 2021).

In as much as online assessments can contribute to enhanced learning, it may also impede learning. Challenges of Internet connectivity and data access, digital incompetency, readiness, and support may negatively impact on the ability of students to engage positively in summative assessment opportunities (Majola & Mudau, 2022). There is a necessity to ensure that the-staff and students know how to and are comfortable to engage with online summative assessments (Hussein, et al., 2020). This requires extensive knowledge of the learning management system (LMS) used by a higher education institution to create an infrastructure for teaching, learning and assessment. Examples include Blackboard, Moodle and Sakai (Badaru & Adu, 2022). Though all these systems are capable of hosting online examinations (Al-Zoubi, Dmour & Aldmour, 2022), issues may be experienced with catering for different types of assessments or different grading methods (Gilbert, 2022). Though instant grading via assessments, linked to multiple-choice exams may be supported through an LMS, questions that involve interpretation and longer answers cannot be auto assessed. An academic or assessor is still required to read through, and grade answers, though AI and machine learning will improve auto-grading in the future (Lee & Fanguy, 2022).

For the present, one of the key problems experienced during assessments, and particularly summative assessments, relates to unethical student behaviour. Since the emergence of online examinations, institutions have explored possible methods to guarantee academic integrity and institutional prestige after discovering academic fraud in unproctored online assessments (Khalil, Prinsloo & Slade, 2022). Gilbert (2022) explains that no matter the online environment, there will always be opportunities for unethical behaviour such as cheating, sharing examination answers with others or obtaining support from a third party. The key to curbing such inappropriate behaviour is using online proctoring. Kharbat & Daabes (2021) expand that it is only using methods such as proctoring that the integrity of online summative assessments can be assured. The importance of utilising proctoring tools during online assessments has also been emphasised by Duncan and Joyner (2022), who articulate that e-proctoring technologies are key to monitor online summative assessments. It is only through such proctoring tools that students' identities can be validated, suspicious activities flagged, plagiarism identified and cheating discouraged.

An online or e-proctoring system is an intelligent practical approach to meeting the need for online examination, according to Jia and He (2021). Lee and Fanguy (2022) indicate that online proctoring tools emerged in 2008 and has gained favour during and past the COVID-19 pandemic. Educational institutions all over the world employ numerous online examination proctoring systems that provide diverse solutions to lowering the likelihood of cheating (Masud, Hayawi, Mathew, Michael & El Barachi, 2022). These proctoring tools may range from simplistic tools that limit the number of opportunities that may be given to a student to complete an assessment (non-AI driven), to proctoring tools that examine recordings of online summative assessments to identify suspicious behaviour (AI driven) (Lee & Fanguy, 2022). AI is useful for mobile ID biometric verification (fingerprint, face, and voice), and identifying potential infractions and probable fraud activity through screenshots, audio recordings, and video (Slusky, 2022). In terms of the use of the latter type of proctoring tools, Kharbat & Daabes (2021) explain that through webcams, screens and microphones, students' movements and their environments can be monitored during online summative assessments, to ensure the integrity of the examination process.

Castets-Renard and Robichaud-Durand (2022) declare that the degree to which an online proctoring tool can be used with success will depend on the extent to which technology tools are used to monitor and track student behaviour. E-proctoring tools with access to microphones, webcams and a 360° view of a student's workspace will be more effective than a tool that merely randomises questions to prevent students from sharing answers. Through the continuous development of AI-based proctoring technologies, higher education institutions will be more successful in minimising cheating or inappropriate behaviour during summative assessments (Takyar, 2022). Sridhar and Rajshekhar (2022) support this view by explaining that the increasing need for effective and efficient proctoring tools will encourage designers and developers to be more creative in providing solutions to enhance the integrity of online summative assessments. Developments in AI and machine-learning proctoring may therefore continuously enable higher education institutions to secure the integrity of their summative assessment practices through advanced proctoring technology (Coghlan, Miller & Paterson, 2021).

 

THEORETICAL FRAMEWORK

Theories are important 'priori' to be used as an explanatory lens to interpret patterns of events (Shanks & Bekmamedova, 2013). Furthermore, a theoretical exploration provides a context in which to explain and understand the findings of research (Ngulube, 2018). It provides direction, gives meaning to the research enterprise and assists in the identification of key findings. To execute this research, the Giddens Structuration theory was applied, to provide a framework for key components to consider in exploring opportunities and obstacles relevant to the use of proctoring tools at a CODeL university. This multidisciplinary theory is based on the premise of social action, which argues that society should be understood in terms of action and structure (Englund, Gerdin & Burns, 2011). Three key components inform the structuration theory, namely the structure, system, and structuration. The structure can be described as rules and resources established as part of a social system. These are rules that humans draw from and reproduce as they take some form of action, based on what is socially accepted (Jones & Karsten, 2003). The second component relates to system, which describes repeated relationships between rules and society that are organised as common social norms. The third component refers to structuration or the production and reproduction of social systems brought about by the interaction of rules and resources (Orlikowski & Robey, 1991).

The inter-relationship between these three components enables structures of social systems such as rules to inform social practices (systems). But social practices are also informed by resources that influence structure and actions. Neither structure, nor action can exist independently and together form the structuration, where structures create social action and social action supports structures (Corgi, 2022). Figure 1 provides an overview of the inter-relationship of the components structure and action (system) to create structuration:

 

 

Contextualised for this research, the description of the Giddens Structuration theory by Thompson (2017) provides the context for the examination of a proctoring tool for an information literacy module at an online distance e-learning institution. Within the context of summative assessment practices, academic integrity is key to ensure trustworthiness and validity of assessment outcomes. To ensure the upkeep of the integrity of social structures such as higher education institutions, agents in the form of academics and students are responsible for perceptions about academic integrity. If students do not follow rules to adhere to ethical practices, the integrity of the higher education system will fail. To encourage students to apply these rules, academics at higher education institutions have become reliant on proctoring technology, to ensure that regular social practices associated with the completion of summative assessments are adhered to. Assuring the maintenance of the structure of higher education thus require rules and resources, such as proctoring tools, to be implemented to ensure the action of maintaining the integrity of online summative assessment practices. This interpretation is supported by the views of Tanner and Piper (2010) who explain that rules related to academic honesty and respect and the preservation of these through measures such as proctoring tools, are important to maintain structuration.

 

RESEARCH METHODOLOGY

The research methodology provides the 'blueprint' for how research is to be executed (Grant & Osanloo, 2014:12). This study opted to follow a collaborative autoethnographic design to explore the views of academics on the use of proctoring tools to enhance the integrity of summative assessments in higher education. Collaborative autoethnography expands on a well-known idea of qualitative research, while suggesting a whole new method for gathering social data (Stigter, 2016). The application of collaborative autoethnography was conducted within the context of an interpretivist paradigm, where the theoretical framework provided the ontological context (depth and detail) within which the authors used self-reflection and dialogue to explore experiences pertaining to proctoring, related to a wider cultural and social understanding of the use of technology. Roy and Uekusa (2020:384) refer to collaborative autoethnography as a 'commentary', which involves exploring experiences of scholars. Miyahara and Fukao (2022:2) expand on this description by indicating that collaborative autoethnography refers to the collaboration between two or more researchers to gain meaning and understanding related to a phenomenon. Collaborative autoethnography provides opportunities for collaboration where researchers are giving the opportunity to become self-focused in certain context and dialogue through critical reflection, interaction, and exploration. While teaching information literacy skills, we started wondering: how has the use of proctoring tools influenced engagement in summative online assessments? From an epistemological point of view, the construction of knowledge was a democratic process (McMillan & Schumacher, 2010:370), which involved searching for our personal experiences in the use of proctoring tools and comparing these to literature that presented larger cultural and social meanings, to enhance understanding. The emerging qualitative research technique of collaborative autoethnography enabled the authors to explore the topic of proctoring in a highly personalised manner (Wall, 2006). During collaborative autoethnography, the authors retrospectively and selectively shared their experiences. The aim is to look at experiences analytically and present them in such a way that they may relate to others.

Collaborative autoethnography primarily has its roots in two long-standing approaches to inquiry and meaning making: autobiography and ethnography (Ellis, Adams & Bochner, 2011). Edwards (2021) posits a deep emergence in self-experience and reflection. Roy and Uekusa (2020:386) argue that collaborative autoethnography as a qualitative method can be used to gain insight and a rigorous understanding of experiences. It is not the retelling of a personal narrative but a careful organisation of systematically collected and analysed data (Miyahara & Fukao, 2022:2). This rigorous understanding of issues requires the application of ethical conduct to ensure an objective reflection of the events under consideration. Such objectivity requires that personal views be compared with existing sources and with the views of others involved in the research. As both researchers share experiences and expertise in the discipline of Information Science, it was a natural choice to share first-hand experiences of using proctoring tools as they relate to summative assessments pertaining to this discipline. In short, Information Science is the science and practice that deals with the collection, storage, retrieval and use of information. It includes areas of recordable information and knowledge, and the use of technologies to facilitate the management and use of information (Association of Information Science and Technology, 2023). The researchers opted for the collaborative autoethnography approach because they have been employed at a CODeL university for more than ten years and have first-hand experience of the use of proctoring tools in online summative assessments. This study, which involves two collaborators who trust each other, began and aimed to make the shift from the individual collective agency to a more comprehensive understanding of the use of proctoring in continuous assessment. Lapadat (2017) postulates that when personal experiences are acknowledged, respected, and seen as embedded within workplaces and social structures, people are more likely to work together to change the workplace and societal practices, therefore working towards structuration to improve structures and systems.

 

COLLABORATIVE AUTOETHNOGRAPHIC ENGAGEMENT: FINDINGS

Linked to the structuration theory, the researchers for this study engaged in discussions to explore three components relevant to structure and systems namely, rules, resources (structure) and action (system). Related to these components, the autoethnographic discussion revolved around questions about what rules or guidelines were in place to guide academics and students through the process of utilising proctoring tools, where and how proctoring tools would be used, and how academics could ensure the optimum use of proctoring tools, to ensure that the integrity of summative assessments could be attained. In addition, the discussion focused on training interventions that were required to support approved practices of no cheating during summative assessments, and the identification of obstacles and further opportunities for the use of proctoring technology.

Rules and guidelines pertaining to proctoring technology use

Hussein et al. (2020) explain that the use of proctoring tools requires two key components, namely a web-camera linked to the student's device to record the physical learning space and a lockdown space, to ensure that students do not obtain input or guidance from other parties. Similarly, Fiano, Medina and Wahlen (2022) argue that parties involved in the proctoring process should be clearly informed about the requirements needed to use proctoring technology and their roles and responsibilities. The importance of understanding the way proctoring should be applied and the rules to follow, was a key point of discussion among the researchers. Since the implementation of proctoring was a new addition to online summative assessment during COVID-19, academics had to engage in extensive training to familiarise themselves with how proctoring tools operate, the requirements to ensure that students were not disadvantaged unnecessarily, and how information could be transferred to students to ensure that all parties involved in the summative assessment process were well-informed. In collaboration with academics within the Department of Information Science and support departments related to teaching, learning, quality assurance and module development within the CODeL university, a guideline was compiled to provide understanding of the scope and context of proctoring technology. The document provided detail on requirements to be put in place to ensure the effective use of proctoring technology, and how students should be supported to prepare for the use of proctoring during summative assessments. Detail about proctoring tools were communicated with students via the institutional website, but also through announcements sent out via the learning management system. It was explained that invigilator tools require a laptop, desktop or smartphone with a functional camera, as well as reliable Internet connectivity, and a head and shoulder profile picture of the student to be uploaded onto the learning management system prior to the summative assessment (UNISA, 2022). Although the information that was presented to staff and students was deemed sufficient at the time, it became evident during the online summative assessments that more information was needed to prepare staff to use proctoring technology and for students to fully understand how to engage with the technology, without it negatively impacting on the assessment process (Gous, 2019).

Support departments within the CODeL environment embarked on an extensive testing and training project to support academics in understanding proctoring tools and their uses. Fiano et al. (2022) explain that testing is imperative to provide detailed information to academics on the functions of proctoring technology and the impact (positive and negative) of the use of these technologies. Internal training on proctoring technologies was provided on a weekly basis, coupled with Moodle (LMS) Café sessions, where staff could discuss the use of the tools with ICT experts. More updated 'how to' manuals were also made available with screenshots, to assist lecturers in understanding the back-end use of the proctoring technology, and how to link the technology to summative assessment sites. Within the Department of Information Science, Moodle-champions were appointed and extensively trained, so that they could provide additional support and assistance to other academics, who had to learn how to utilise proctoring technology most effectively. These champions were able to provide hands-on guidelines and assistance to support other academics in uploading and activating the proctoring technology to their online summative assessment sites, to ensure the integrity of the examination processes.

In addition, an extensive campaign was launched by the CODeL university to inform students of the rules and guidelines pertaining to the use of proctoring technology. This campaign involved the distribution of e-mails with clear guidelines as to the use of proctoring tools and the requirements to engage effectively in the use of proctoring technology, reasons for the use of proctoring technology, and mitigating rules if proctoring negatively affected a student during the summative assessment process. Tutorial letters with guidelines on the use of proctoring technology were used to portray 'a more complete view of the phenomenon' (Muncey, 2005). However, the use of Teams sessions to prepare students for the use of online proctoring and the offering of mock examinations to practice the use of proctoring technology, lay the groundwork to prepare students for the use of such technology. During Teams sessions, online demonstrations were given on how to activate web cameras and to ensure that the environment around the student was sufficiently quiet to yield a positive proctoring report. Hussein et al. (2021) explain that such demonstrations provide opportunities for students in a non-threatening environment to practice the use of proctoring technology. Although students tend not to participate in Teams sessions, the authors found that students were particularly interested in sessions on proctoring, especially when they had not had the opportunity to use it before. Coupled with a live engagement during mock examinations, students could learn quickly how to follow guidelines provided via e-mail and announcements on activating the web camera, uploading photos where required, unblocking the web camera if blocked by mistake, and refreshing the camera if Internet connectivity had been lost (UNISA, 2022).

Resources relevant to proctoring technology use

Fiano et al. (2021), as well as Hussein et al. (2022) explain that a technology infrastructure must be in place for proctoring technology to be effectively used. Depending on the scope of proctoring required and the technology infrastructure available within a higher education institution, various types of online proctoring may be considered. Proctortrack (2021) refers to these as Proctorlock, where proctoring data captured include video, audio and desktop screenshots, ProctorAuto, which provides an automatic identity verification, data recording and data analysis, and ProctorTrack QA, where the emphasis is on manual quality assurance reviewed by trained proctors. In addition, ProctorLive AI as a type of proctoring technology, provides real-time hybrid solutions via remote invigilation and human proctors with AI proctoring capabilities, to identify suspicious behaviour and activities. Due to the large number of students involved in summative assessments at the CODeL university, the introduction of a new LMS, namely Moodle, and the inexperience of staff in the use of proctoring technology, the Proctorlock type was mostly used during summative assessments to ensure the integrity of the examination process. This means that screenshots are taken at various intervals via web cameras, videos, and audio, as well as a desktop, to ensure that the correct student is writing the assessment, without assistance from other parties, and who can complete the assessment within the set timeframe given for the examination.

Though the required resource and infrastructure to execute the Proctorlock type seem simple enough, the researchers found that it was not the case. Due to the digital divide in South Africa and other areas in Africa where students embark on summative assessments, many students do not have access to the necessary hardware or network infrastructure to execute the guidelines for using the proctoring technology. The lack of infrastructure, data, and resources to engage in online proctoring technology is emphasised by Woldegiorgis (2022), who explains that students from disadvantaged backgrounds are influenced the most by the move to online summative assessments and the use of proctoring technology. In South Africa there is still inadequate digital infrastructure, lack of funds for digital investment, and a lack of digital literacy (Gqoboka, Anakpo & Mishi, 2022). In the appropriation of proctoring tools, the type of device the student owns and how it works with proctoring software became important considerations, because some students don't have a laptop to study with, and instead engage in online assessments via their cell phones. (Ndovela & Marimuthu, 2022). Even for those who own devices, the issue of access is costly. Rahman (2022) discloses that the biggest obstacles to successfully using online proctoring systems include increased prices of data, a shortage of appropriate devices, limited bandwidth in remote locations, and students' technological limitations. This poses a challenge to academics to customise online examination, so that it can be conducted on a mobile application. Encouraging the use of mobile applications may be necessary because, as affirmed by Bejrajh and Themane (2022), the majority of South African students own smartphones that can be used more effectively to engage in academic activities. Though the CODeL university does supply students with data to engage in educational activities, the researchers acknowledge that the amount of data is not sufficient to properly train and encourage students to utilise proctoring technology during online assessments. In fact, a trend developed where students would log on to a summative assessment site, activate the proctoring tool and leave before completing the assessment, possibly due to data shortage. This phenomenon, however, requires more investigation to confirm the assumption.

Actions towards proctoring use

A third component of the structuration theory, focused on the systems component relates to action. The agency of action is critical to reproduce and encourage transformation within a specific context. Individuals involved in creating the action are required to bring about change, whilst also reproducing rules and resources contextualised during the structure part of the structuration theory (Thompson, 2017). To achieve action, Pham (2019) proposes that interaction and routinisation, based on knowledge of the structure, be considered. Agents should rationalise rules and resources, and in doing so, coordinate ongoing projects, whilst contextualising and performing actions. While considering this component of the structuration theory, the researchers discussed the future use of proctoring technology and its expansion to enhance the integrity of summative assessment processes. The researchers agreed that for proctoring to be efficient, attention should be given to the technology infrastructure of the examination system, both within the institution and for students. Our experience was that during examinations, the system would often fail due to large volumes of activity; implying that neither academics, nor students could engage with summative assessment activities.

The issue pertaining to the stability of the examination system emanated in the resetting of examination question papers and offering of additional examination opportunities, so that students could complete summative assessments within a trustworthy and reliable environment. This not only increased the workload of academics, but also brought clashes between summative assessments scheduled for students studying towards a degree in information science. Similar experiences were shared during joint operations meetings with academics from other departments. For students, the instability of the examination system brought about anxiety, uncertainty and loss of valuable data that was needed to complete the summative assessment. It also created opportunities for students to share questions that they had already answered with other students, leading to large scale engagement with assessment questions and answers in the hope of receiving similar question papers. None of this was conducive towards supporting the integrity of the online summative assessment process. What did however arise, was insight into the urgent need for academics to train students on issues of avoiding plagiarism, institutional policy requirements towards the integrity of the examination process and digital literacy. Towards maintaining and improving structuration, such training is deemed imperative to bring about transformation towards the improved and sustained use of proctoring during summative assessment activities.

 

DISCUSSION OF FINDINGS: OPPORTUNITIES AND OBSTACLES

The proctoring systems and their tools have brought many opportunities for academics and students at the CODeL university. Prior to COVID-19, the institution had been theorising about and strategising for online assessments. However, the outbreak of the pandemic forced the CODeL university to offer fully online summative assessments, linked to proctoring tools, to ensure the integrity of the process. The use of proctoring technology reduced the costs of human invigilators and enhanced the opportunity for students to engage in summative assessment activities in an environment most comfortable to them. The hiring and maintenance of examination venues have also been minimised because students are no longer conducting examinations in these venues. Students have also reduced their transport costs since they are no longer expected to travel to physical venues to take examinations. The online summative assessment process enhanced the security of examinations, as academics do not have to worry about the unauthorised sharing of examination scripts, or the late arrival of summative assessment scripts at the examination venues. There were, however, costs involved in the purchasing and use of proctoring tools. These costs were excessive during the initial purchase and implementation, but over time has become minimal, as the purchase was once-off and the tools are available on the examination system for future use. What is important is that the costs are determined by the type of proctoring tools used. Whilst some proctoring tools can be purchased as a software programme for continuous use, others require the purchase of licensing. Licensed proctoring can become excessively expensive, as licenses must be purchased for each student during each examination period. This is, however, not the type of proctoring tool used by students engaged in the information literacy module. The proctoring tool used was purchased by the university with little additional cost to execute during each examination.

A variety of online summative assessment methods were also introduced, to provide variety and assess the understanding, application, analysis and evaluation skills of students. Students' engagement with various assessment methods could all be monitored through online proctoring technology, to ensure the integrity of the process. Examples of assessment methods include multiple choice true/false assessments, completion of short answer assessments and fill in the missing word assessments. To offer these online summative assessment methods, sustainable ICT infrastructure is imperative, to ensure the effective execution of the summative assessment process and the use of proctoring technology. In our experience, the instability of the system was the main reason why students and staff struggled with proctoring technology and its application. Another issue that impacts negatively on the use of proctoring technology relates to the scattered geographic areas from where students engage in assessment activities. Many students enrolled at the CODeL university are from rural areas, where access to Wi-Fi, Internet, and ownership of devices are still challenges. Internet connection, including issues of data bundles for many students, has not been fully addressed. Some had to share a computer and connection with friends, neighbours and the rest of the family, all of which impact negatively on the proctoring report produced for those students. Limited technology knowledge was also perceived as an obstacle. As academics, we would teach our students how to access the proctoring tools and webcam devices. Some of them often asked, 'what is the webcam'. This is an indication that some students lack the necessary skills to engage online, whether for studies or assessment in a CoDEL institution.

Lee & Fanguy (2022) state that the adoption of online proctoring technologies has produced negative impacts on students' and lecturers' subjectivities, pedagogical relationships and educational outcomes. This was evident in the increased aggression and frustration levels expressed by students during Teams sessions or e-mail correspondence, as experienced by the authors. In communication to the authors, students blamed academics for poor performances or the inability to activate proctoring tools (Hybertext, 2023). The use of different assessment methods frustrated students, as they often felt ill-prepared to engage in answering questions related to analysis and evaluation. In such instances, students reverted to copy and paste activities from existing sources to produce possible answers; thereby extending, rather than minimising instances of cheating. Students also found it difficult to transition from venue-based to non-venue based summative assessments, and to take responsibility for their own actions to ensure the integrity of the examination process (Dyomfana, 2021).

 

CONCLUSION

The research has shown that rules, resources and actions are needed for the effective and continuous use of proctoring technology to support the integrity of summative assessment processes. Related to the structuration theory, more time and engagement is needed for students and academics to find a space where social systems pertaining to the use of proctoring technology can become a familiar norm and be expanded to include more advanced technology, to ensure the integrity of the summative assessment process. Reciprocity between staff, students and other members of the CODeL community is important, as it will impact the nature of interaction in the use of proctoring technology. This reciprocity will also influence the rules and resource requirements necessary, to ensure a continuous flow of activities related to proctoring use that are reflexively monitored.

Lee & Fanguy (2022:12) declare that 'it is difficult to know whether the benefits of these online proctoring technologies outweigh their risks. The most reasonable conclusion is that the ethical justification of these technologies requires us to ensure that a balance is struck between the concerns and the possible benefits.' Related to this view, enhanced focus on reshaping the thoughts of students on cheating and academic integrity is needed to discourage unethical behaviour. Though proctoring technology can support the integrity of the assessment process, it is more important that students act ethically, based on their own convictions to support validity, trustworthiness and accountability as part of the summative assessment processes. Within the context of information science, this requires extensive training on the ethics of information, knowledge sharing and digital literacy. As a way forward, it is proposed that the Department focuses on the development of open educational resources on these topics that can be shared, not just with students from the Department, but with all students studying at the CODeL institution.

 

REFERENCES

Al-Zoubi, A., Dmour, M. & Aldmour, R. (2022) Blockchain as a Learning Management System for Laboratories 4.0. International Journal of Online & Biomedical Engineering 18(12) pp.16-34.         [ Links ]

Association of Information Science and Technology. (2023) What is information science? https://www.asist.org/about/what-is-information-science/ (Accessed 30 January 2023).

Badaru, K.A. & Adu, E.O. (2022) Platformisation of Education: An Analysis of South African Universities' Learning Management Systems. Research in Social Sciences and Technology 7(2) pp.66-86.         [ Links ]

Bejrajh, V. & Themane, M. (2022) Using Smartphones in Teaching English to Secondary School Students in South Africa. Interdisciplinary Journal of Education Research 4 pp.76-90.         [ Links ]

Castets-Renard, C. & Robichaud-Durand, S. (2022) Online Test Proctoring Software and Social Control: Is the Legal Framework for Personal Information and AI Protective Enough in Canada? Blockchain and Artificial Intelligence Solution to Enhance the Privacy in Digital Identity and Internet of Things. Forthcoming: Taylor & Francis/CRC Press.

Clay, J. (2020) Digital assessment in higher education, https://ijtl.iie.ac.za/newpage5c162231 (Accessed 30 January 2023).

Coghlan, S., Miller, T. & Paterson, J. (2021) Good proctor or "big brother"? Ethics of online exam supervision technologies. Philosophy & Technology 34 pp.1581-1606.         [ Links ]

Corgi. (2022) Anthony Giddens and the theory of structuration. https://studycorgi.com/anthony-giddens-and-the-theory-of-structuration/ (Accessed 30 January 2023).

Duncan, A. & Joyner, D. (2022) On the necessity (or lack thereof) of digital proctoring: Drawbacks, perceptions, and alternatives. Journal of Computer Assisted Learning 38(5) pp.1482-1496.         [ Links ]

Dyomfana, B. (2021) Unisa's rule for online exams. https://www.careersportal.co.za/education/universities/unisas-rules-for-online-exams (Accessed 30 January 2023).

Edwards, J. (2021). Ethical autoethnography: Is it Possible? International Journal of Qualitative Methods 20(3), doi:10.1177/1609406921995306        [ Links ]

Ellis C., Adams T. E. & Bochner A. P. (2011) Autoethnography: An overview. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, 12(1) p.10, http://nbn-resolving.de/urn:nbn:de:0114-fqs1101108 (Accessed 30 January 2023).         [ Links ]

Englund, H., Gerdin, J. & Burns, J. (2011) 25 years of Giddens in accounting research: achievements, limitations and the future. Accounting, Organisations and Society 36 pp.494-513.         [ Links ]

Fiano, K.S., Medina, M.S. & Whalen, K. (2021) The need for new guidelines and training for remote / online testing and proctoring. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8500292/ (Accessed 30 January 2023).

Fynn, A. & Janse van Vuuren, H. (2017) Investigating the role of non-academic support systems of students completing a Master's Degree in Open, Distance and e-Learning. Perspectives in Education, 35(1) pp.186-199.         [ Links ]

Giddens, A. (1984) The construction of society. Outline of the theory of structuration. Berkeley: University of California Press        [ Links ]

Gilbert, C. (2022) Advantages and disadvantages of online examination system. https://cirrusassessment.com/advantages-and-disadvantages-of-online-examination-system/ (Accessed 30 January 2023).

Gous, N. (2019) Students who leaded exams papers faced up to five years' suspension: Unisa. https://www.timeslive.co.za/news/south-africa/2019-05-27-students-who-leaked-exam-papers-face-up-to-five-years-suspension-unisa/ (Accessed 30 January 2023).

Gqoboka, H., Anakpo, G. & Mishi, S. (2022) Challenges Facing ICT Use during COVID-19 Pandemic: The Case of Small, Medium and Micro Enterprises in South Africa. American Journal of Industrial and Business Management 12(9) pp.1395-1401.         [ Links ]

Grant, C. & Osanloo, A. (2014) Understanding, selecting and integrating a theoretical framework in dissertation research: Creating the blueprint for your house. Administrative Issues Journal: Connecting Education, Practice and Research. Volume 4(2) pp.12-26 https://files.eric.ed.gov/fulltext/EJ1058505.pdf (Accessed 30 January 2023).         [ Links ]

Heeralal, P.J.H. (2015) Improving postgraduate supervision in an open and distance learning environment: A case study at the College of Education, University of South Africa. South African Journal of Higher Education 29(3) pp.87-100.         [ Links ]

Holden, O.L., Norris, M.E. & Kuhlmeier, V.A. (2021) Academic integrity in online assessment: A research review. https://www.frontiersin.org/articles/10.3389/feduc.2021.639814/full (Accessed 30 January 2023).

Hussein, M.J., Yusuf, J., Deb, A.S., Fong, L. & Naidu, S. (2020) An evaluation of online proctoring tools. Open Praxis 12(4) pp.509-525.         [ Links ]

Hypertext. (2023) Unisa's online exams to become a permanent fixture. https://htxt.co.za/2020/09/23/unisas-online-exams-to-become-a-permanent-fixture/ (Accessed 30 January 2023).

Jia, J. & He, Y. (2021) The design, implementation and pilot application of an intelligent online proctoring system for online exams. Interactive Technology and Smart Education 19(1) pp.112-120.         [ Links ]

Jones, M.R. & Karsten, H. (2003) Giddens's structuration theory and information systems research. MIS Quarterly, 32 pp.127-157.         [ Links ]

Khalil, M., Prinsloo, P. & Slade, S. (2022) In the nexus of integrity and surveillance: Proctoring (re) considered. Journal of Computer Assisted Learning 38(6) pp.1-14. https://doi.org/10.1111/jcal.12713 (Accessed 30 January 2023).         [ Links ]

Kharbat, F.F. & Daabes, A.S. (2021) E-proctored exams during the COVID-19 pandemic: A close understanding. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7884061/ (Accessed 30 January 2023).

Lapadat, J.C. (2017) Ethics in autoethnography and collaborative autoethnography. Qualitative inquiry 23(8) pp.589-603.         [ Links ]

Lee, K. & Fanguy, M. (2022) Online exam proctoring technologies: educational innovation or deterioration? https://bera-journals.onlinelibrary.wiley.com/doi/full/10.1111/bjet.13182 (Accessed 30 January 2023).

Majola, M.X. & Mudau, P.K. (2022) Lecturers' Experiences of Administering Online Examinations at a South African Open Distance E-Learning University during the COVID-19 Pandemic. International Journal of Educational Methodology 8(2) pp.275-283.         [ Links ]

Manyike, T.V. (2017) Postgraduate supervision at an open distance e-learning institution in South Africa. South African Journal of Education 37(2) pp.1-11.         [ Links ]

Masud, M.M., Hayawi, K., Mathew, S.S., Michael, T.& El Barachi, M. (2022) Smart Online Exam Proctoring Assist for Cheating Detection. International Conference on Advanced Data Mining and Applications pp.118-132.

McMillan, J.H. & Schumacher, S. (2010) Research in Education. Evidence-based inquiry. (7^th ed.) Boston: Pearson.         [ Links ]

Miyahara, M. & Fukao, A. 2022. Exploring the use of collaborative autoethnography as a tool for facilitating the development of researcher reflexivity. System 105(102751) pp. 1-9.         [ Links ]

Muncey T. (2005) Doing autoethnography. International Journal of Qualitative Methods 4(3) p.5, http://www.ualberta.ca/~iiqm/backissues/41/pdf/muncey.pdf (Accessed 30 January 2023).         [ Links ]

Ndovela, S. & Marimuthu, M. (2022) aMangosuthu University of Technology, Umlazi, South Africa, University of KwaZulu-Natal, Durban, South Africa. Academic Voices p.443.

Ngulube, P. (2018) Handbook of research on heritage management and preservation. Pennsylvania: IGI Global.         [ Links ]

Noorbehbahani, F., Mohammadi, A. & Aminazadeh, M. (2022) A systematic review of research on cheating in online exams from 2010 to 2021. https://link.springer.com/article/10.1007/s10639-022-10927-7 (Accessed 30 January 2023).

Orlikowski, W.J. & Robey, D. (1991) Information technology and the structuring of organisations. Information Systems Research 2(2) pp.143-169.         [ Links ]

Pefialvo, F.J., Gorell, A., Garcia, V.A. & Prado, M.G. (2020) Recommendations for mandatory online assessment in higher education during the COVID-19 pandemic. https://link.springer.com/chapter/10.1007/978-981-15-7869-4_6 (Accessed 30 January 2023).

Pham, H.T. (2019) The application of structuration theory in studying collaboration between librarians and academic staff in universities in Australia and Vietnam. http://informationr.net/ir/24-3/paper829.html (Accessed 30 January 2023).

Phuthela, N. & Dwivedi, S. (2020) A qualitative study of students' perspective on e- learning adoption in India. Journal of Applied Research in Higher Education 12(4) pp.545-559. https://doi.org/10.1108/JARHE-02-2019-0041 (Accessed 30 January 2023).         [ Links ]

Proctortrack. (2021) Online proctoring technology: Key to secure online exams, https://www.proctortack.com/blog/article/online-proctoring-technology-key-to-secure-online-exams (Accessed 30 January 2023).

Rahman, A. (2022) Mapping the Efficacy of Artificial Intelligence-based Online Proctored Examination (OPE) in Higher Education during COVID-19: Evidence from Assam, India. International Journal of Learning, Teaching and Educational Research 21(9) pp.76-94.         [ Links ]

Roy, R. & Uekusa, S. (2020) Collaborative autoethnography: 'self-reflection" as a timely alternative research approach during the global pandemic. Qualitative Research Journal 20(4) pp.382-392, doi:10.1108/QRJ-06-2020-0054        [ Links ]

Shanks, G. & Bekmamedova, N. (2018) Case study research in information systems. https://www.researchgate.net/publication/323515278_Case_study_research_in_information_systems (Accessed 30 January 2023).

Slusky, L. (2020) Cybersecurity of online proctoring systems. Journal of International Technology and Information Management 29(1) pp.56-83.         [ Links ]

Sridhar, A. & Rajshekhar, J.S. (2022) AI-Integrated Proctoring System for Online Exams. Journal of Artificial Intelligence 4(2) pp.139-148.         [ Links ]

Stephens, N.M. (2021) How to catch student exam cheating with remote proctoring. https://assess.com/remote-proctoringsecurity/ (Accessed 30 January 2023).

Stigter, S. (2016) Autoethnography as a new approach in conservation. Studies in Conservation 61(2) pp.227-232.         [ Links ]

Swart, O. (2015) Alternative Assessments: The journey from venue-based examinations to take-home and online timed assessments. Proceedings of the 14th World Conference on Mobile and Contextual Learning, mLearn, Venice, Italy, October 17-24. Springer International Publishing, pp. 378-390.

Takyar, A. (2022) Remote proctoring using AI - Enabling seamless management of online examinations. https://www.leewayhertz.com/remote-proctoring-using-ai/ (Accessed 30 January 2023).

Tanner, L. & Piper, C. (2010). Succumbing to temptation - to cheat or not to cheat? Faculty and student perceptions of academic integrity in distance education courses. https://www.learntechlib.org/p/35760/ (Accessed 30 January 2023).

Thompson, K. (2017) Giddens' structuration theory - a summary. https://revisesociology.com/2017/03/02/giddens-structuration-theory-a-summary/ (Accessed 30 January 2023).

UNESCO. (2022) Artificial intelligence in education. https://www.unesco.org/en/digital-education/artificial-intelligence (Accessed 30 January 2023).

University of South Africa (2023) Know your proctoring tools for the examination. https://www.unisa.ac.za/sites/myunisa/default/Announcements/Know-your-proctoring-tools-for-the-examination (Accessed 30 January 2023).

University of South Africa. (2022). Activating your camera for Moodle proctoring tool for MCQ examinations. https://www.studocu.com/en-za/document/university-of-south-africa/introduction-to-african-philosophy/activating-your-camera-for-moodle-proctoring-tool-for-mcq-examinations-edited-23may2022/29161775 (Accessed 30 January 2023).

van Breda, M. & van Wyk, M. (2018) Electronic-Portfolio Approach to Enhance Self-Directed Learning. Transnational Perspectives on Innovation in Teaching and Learning Technologies pp.45-66. Brill.

Wall, S. (2006) An Autoethnography on Learning About Autoethnography. International Journal of Qualitative Methods 5(2) pp.146-160. https://doi.org/10.1177/160940690600500205 (Accessed 30 January 2023).         [ Links ]

Weleschik, A., Dyjur, P. & Kelly, P. (2019) Online assessment in higher education. https://taylorinstitute.ucalgary.ca/sites/default/files/TI%20Guides/Online%20Assessment%20Guide-2019-10-24.pdf (Accessed 30 January 2023).

Woldeab, D. & Brothen, T. (2019) Twenty-first century assessment: online proctoring, text anxiety and student performance. International Journal of E-Learning and Distance Education 34(1) pp.2-10. https://files.eric.ed.gov/fulltext/EJ1227595.pdf (Accessed 30 January 2023).         [ Links ]

Woldegiorgis, E.T. (2022) Mitigating the digital divide in, the South African higher education system in the face of the COVID-10 pandemic. Perspectives in Education 40(3) pp.197-211. doi: http://dx.doi.org/10.18820/2519593X/pie.v40.i3.13 (Accessed 30 January 2023).         [ Links ]

 

 

Date of Submission: 5 February 2023
Date of Review Outcome: 12 June 2023
Date of Acceptance: 31 August 2023