TEACHERS VOICE / HANDS-ON ARTICLES

Teaching and learning of History at a high school level-the reality of AI/ChatGPT and the process of assessing understanding

Gordon Brookbanks

Westerford High School, Cape Town, South Africa

A recent article appeared in The Conversation, authored by five academics from South Africa, Australia, the United Kingdom, and the United States, all working in education-related fields. Their article, entitled "ChatGPT is the push higher education needs to rethink assessments", argues that ChatGPT can show learners "the wonder and responsibilities of acquiring and building powerful knowledge". They suggest four potential applications of ChatGPT in this regard. The perspective of these academics is contrary to the argument that universities can no longer confidently assert that tests assessed by academics have been produced by their learners.

While the article addresses the reality and use of ChatGPT in tertiary institutions, and the four potential applications of ChatGPT offered are in a university context, a similar conversation is needed within a high school context like that of Westerford. While high school teachers need to inform themselves as to the impact of ChatGPT on the process of teaching and learning generally, each subject department needs to specifically consider whether assessments which are not undertaken in standardised and controlled circumstances have in fact been produced by their learners.

At the outset, it is important to understand that our conscious and unconscious use of and reliance on artificial intelligence is not new. Reliance on our phones for using Google Maps (GPS) to get from point A to B is a case in point. As a 'Baby Boomer', I acknowledge that I use Google Maps regularly when travelling. I no longer own a 'map book' in the car- the traditional 'A to Z', but I still know how to use a 'map book. While Generation Z (our high school learners), who are colloquially known as 'zoomers', which number about 32% of the world's population, neither know what a 'map book' is nor have heard of the A to Z'-they immediately will access Google Maps if the need arises. Reliance on and familiarity with the use of Artificial Intelligence (AI) is a reality in our lives, but the challenge of ChatGPT in terms of assessing understanding in a high school context relates specifically to 'take home' tasks. It raises similar concerns which we had pre-COVID-19 in determining the extent of plagiarism, or what we categorised as 'intellectual theft', in a student's submission. While it is not possible to accurately quantify the extent of the problem we were confronted with, we would periodically address the tendency in a whole school assembly. In the interest of uniformity of approach, we decided the whole school would only use the Harvard system of referencing, and for two years we followed the example of the University of Cape Town and used Turnitin for research submissions. The cost of using Turnitin for a high school was prohibitive, so it was not renewed after two years.

How did we engage with ChatGPT in history at Westerford High? Based on a SWOT analysis, a starting point was to identify to what extent AI/ChatGPT impacts forms of prescribed assessments within a subject department as part of the required School Based Assessments (SBA). It is worth bearing in mind that the objective of an assessment is to determine a student's understanding of the learning and teaching process. In history, we determine understanding of a topic which has been taught, including the application of specific history skills, in either a source-based or an essay question.

Each history teacher in our department is a Grade Coordinator of a specific Grade. As such, each Grade Coordinator was to undertake a SWOT analysis of the impact of AI/ChatGPT on assessments in their Grade. The tasks, with explicit timeframes, were as follows:

1) All History teachers are requested to read the article below. While authored by academics in the education sphere of tertiary institutions, it offers a perspective that could be applied to assessing the 'learning and teaching' of history at a high school level.

2) With each history teacher being a Grade Coordinator for a specific grade, they are required to do a SWOT analysis (Strengths, Weaknesses, Opportunities, Threats) for the learning and teaching of history in your grade, as such relates to the reality of AI/ ChatGPT and the assessment of understanding in your grade.

2.1) In our assessing of understanding in history, and with the exception offtake home tasks' such as research projects (Gr 10, 11, 12) and oral history tasks (Gr 9), all assessments are undertaken as controlled assessments in the classroom where learners have to use pen and paper with no access to electronic devices. Similarly, examinations are undertaken in controlled circumstances. It is therefore the former, research tasks and the oral history tasks, which may be a subject of assessment addressed in your SWOT analysis in this respect.

3) Based on your grade-specific SWOT analysis and informed by your understanding of the CAPS curriculum AND related assessment requirements, each Grade Coordinator is then required to develop suggested forms of explicit assessment, using the reality of AI/ChatGPT for implementation in either the 2nd and/or 3rd term (due date: 15 April 2023).

4) It is suggested we pilot the suggestions in the 2nd and/or 3rd term 2023 per grade to determine whether it adds value to the 'learning and teaching' of our subject.

5) It would be useful if a tertiary institution monitored the initiative independently to determine as to whether it adds value to the 'learning and teaching' of history at a high school.

Based on the above, we were tiptoeing into the intentional use of AI/ChatGPT in the teaching and learning the grade 12 history curriculum, essay-based assessment. While all history teachers were compiling their grade-specific SWOT analysis, it was decided to ask ChatGPT the question which was to be provided to grade 12 history learners on the completion of their P2Q4 topic (SA in the 1970s)-the form of assessment for P2Q4 is an essay. In addition to providing the question, ChatGPT was instructed to produce a history essay at the level of high school in a maximum of 1,500 words. The essays were marked according to the DBE Essay Marking Matrix.

Using the marking guide and the matrix, the essay produced by ChatGPT was then marked and moderated by history teachers. The history learners then wrote the essay assessment under standardised and controlled circumstances in the classroom, where they had to use paper and pen with no access to electronic devices or notes, and their teacher invigilated the assessment. When the marked scripts were returned to them, we intended to explain to the learners why we arrived at the levels we gave to the essay produced by ChatGPT (L4/L5 - 33). It is important to emphasise that the ChatGPT essay responded to the same question the learners had just responded to in the assessment and was marked using the same marking guide and essay marking matrix.

The objective of providing this explanation to the learners was to remind them of the marking matrix descriptors and caution them on the limitations of AI/ChatGPT (if they elect to use it) in preparing for an essay assessment. At the start of the 2023 academic year, all history learners were given a hardcopy of the essay marking matrix, and the same matrix had been uploaded onto their shared Google Classroom. But it is our experience that the essay marking matrix is seldom referred to by learners when they receive their marked scripts back. They tend to only focus on the actual mark allocated to the essay.

We intended to explain to learners that in terms of content ChatGPT omitted evidence that we have taught in class, included some incorrect facts and terms, and included irrelevant information, some of which related to events outside of the period in the question. Therefore, it was allocated a L4 for content.

In terms of presentation (argument), it will be explained to the learners that ChatGPT did respond to the explicit question posed in its introductory statement, it maintained a line of argument in the body of the essay, but there were problems in its concluding statement. The concluding statement correctly sustained the introductory argument, but it then repeated information from the body of the essay and included irrelevant information. Furthermore, ChatGPT included sub-headings in the essay which are not acceptable for a history essay. Therefore, it was allocated a L5 for presentation/argument.

Learners are encouraged and able to access previous grade 12 question papers (limited largely to 2021 and 2022, as some topics changed from previous years), to take note of the essay question posed in previous papers. They can then enter the question into AI/ ChatGPT to see how the essay introduction is constructed in response to the explicit question in that paper and how the line of argument is maintained through the body of the essay. However, learners must be warned that they are required in an essay-based assessment to respond to the explicit and unique question in front of them. So, in our view, they cannot 'learn' an introduction, line of argument, and conclusion through AI/ChatGPT.