SciELO - Scientific Electronic Library Online

 
vol.112 issue4Guest Editorial NoteDesign and Implementation of a Luganda Text Normalization Module for a Speech Synthesis Software Program author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Article

Indicators

Related links

  • On index processCited by Google
  • On index processSimilars in Google

Share


SAIEE Africa Research Journal

On-line version ISSN 1991-1696
Print version ISSN 0038-2221

SAIEE ARJ vol.112 n.4 Observatory, Johannesburg Dec. 2021

 

How assessment shapes learning: A perspective from engineering students

 

 

T.S. HattinghI; L. DisonII

INorth-West University, Potchefstroom, South Africa (e-mail: teresa.hattingh@nwu.ac.za)
IISchool of Education, University of the Witwatersrand, Johannesburg, South Africa (e-mail: laura.dison@wits.ac.za)

 

 


ABSTRACT

This study explores the influence of assessment on student approaches to learning through the voice of the student. The aim is to understand how assessment practices in an engineering school shape how students respond through their learning approaches and behaviors. The study is framed using a learning-oriented assessment paradigm. Focus groups were conducted with second-, third- and fourth-year engineering students. Four groups were formed using academic performance as a sampling criterion. The findings are presented using selected emergent themes and rich, narrative descriptions. A range of expected and unexpected student responses to assessment practices is highlighted. These are discussed in the context of the assessment literature to provide insight into the relationship between assessment and learning. The study confirms the importance of constructive alignment in assessment design, and includes specific findings related to tutorials, assignments, feedback, and group work. The study reveals that assessment is a powerful driver of learning, and highlights the need for a more collaborative teaching and learning environment to shift learning in a positive direction.

 Index Terms: constructive alignment, focus groups, learning-oriented assessment, student learning, student voice.


 

 

I. INTRODUCTION

A. Background and motivation

The purpose of assessment has shifted away from solely rewarding individual achievement (assessment of learning) to understanding the role of assessment in promoting student engagement (assessment for learning). From this latter perspective, assessment practices have the potential to influence the way that students learn [1,2]. Evidence has shown that learning-oriented and student-centered assessment approaches can yield positive learning outcomes and improve the quality of assessment practices [3,4]. In engineering education, learning-oriented assessment is a key enabler as universities strive to develop students as lifelong learners, moving beyond knowledge, skills, and application [5].

However, it is not well understood how the principles that underpin these theories are interpreted and used in an engineering context. The context of assessment practices is important, as the constraints in higher education, such as increasing student numbers and decreasing teaching capacity, can compromise the alignment between teaching and assessment. This mismatch can negatively influence student learning, and drive assessment practices away from a learning-oriented paradigm [4]. The shifts in teaching and assessment that occur under these conditions can impact students' ability to succeed, and are particularly likely to disadvantage weaker or marginalized students.

Global evolutions also influence the delivery mode of engineering education. There is a steadily increasing interest in distance and online education [6] that has been accelerated by the introduction of emergency remote teaching during the COVID-19 pandemic. It is believed that, although teaching and learning may return to previous delivery mechanisms after COVID-19, these new modes are likely to play an important role within curricula in the future [7]. In addition, triggered by accreditation boards and an enhanced awareness of the importance of a broad range of educational stakeholders, more focus is being placed on the measurement of engineering student competencies such as communication, lifelong learning, teamwork, innovation, and creativity [8], all of which create new complexities in the assessment space. These challenges spotlight the role of assessment practices, and prompt a responsive change in the assessment culture [5] in engineering education.

While assessment practices are adapting to meet these dynamic circumstances, it is essential that an understanding of the delicate relationship between assessment practices and student learning approaches and behaviors is strengthened. Changes to assessment practices can frequently have unintended and sometimes adverse consequences for student learning. As a result, appreciating how assessment plays out in students' daily lives is of great significance in optimizing the learning benefits of assessment. Several studies in engineering education have highlighted the importance and usefulness of exploring student perceptions and experiences of learning and assessment in practice [9,10]. These studies have focused on first-year engineering student experiences in project and authentic learning contexts.

This investigation tackles assessment more holistically, and considers students in higher years and their response to assessment in general across all modules and assessment types. This unique perspective considers the interplay that exists as students navigate their studies, and provides insights into student experiences and the consequences of assessment decisions and practices on their development as learners.

B. Purpose

The purpose of this study is to gain insights, through the students' voice, into an engineering school's assessment practices and how these shape student learning. The study frames the role of assessment as facilitating learning, informed by the theoretical lens of learning-oriented assessment [3] and the need to develop engineers as lifelong learners [5]. As a result, the research question for this study is: What are engineering students' current learning practices, and how are these shaped by assessment practices in an engineering school?

 

II. LITERATURE

Traditional thinking claims that formative and summative assessment are distinct, implying that formative assessment supports student learning (assessment for learning) and summative assessment measures the student learning that has taken place (assessment of learning). However, it has been argued that, in some cases, formative and summative assessment can be indistinguishable [11]. Informal tasks (traditionally seen as formative) can be summated to provide input into final (summative) results, and results from a summative assessment task can be formative by providing feedback on learning [12]. This perspective shakes up the traditional view by implying that both summative and formative assessments can be used of and for learning. Assessment thinking has evolved, therefore, considering perspectives that view assessment more holistically, and focusing on the influence that assessment can have on student learning. As a result, the assessment literature is becoming more learning-oriented [3], focusing on assessment tasks that encourage high levels of cognitive engagement over an extended period, and stimulating deeper approaches to learning [3]. There is also an emphasis on the need for strong constructive alignment between learning outcomes, teaching and learning activities, and assessment tasks [13]. Learning-oriented models can result in an improved quality of student approaches to learning [3] and the development of lifelong learning skills [5]. The transition from assessment of learning to assessment for learning requires a fundamental shift in the paradigms that underpin teaching, assessment, and the curriculum as a whole [11]. There needs to be a move away from traditional teacher-centered and syllabus-centered methods toward a focus on the assessment design - that is, from a product-centered approach to a focus on the learning that takes place through assessments - a process-centered approach [14]. Learning-oriented assessment, therefore, shifts agency toward the student, moving from a lecture-driven learning environment to one in which the students themselves play an active role and take responsibility for their own learning [15]. Building independent learning and self-judgment skills are therefore seen as key features of a supportive learning-oriented context [16]. Feedback needs to play a strongly formative role to facilitate student success in relation to student agency, authentic learning experiences, and the development of lifelong learning skills [17]. The phrase 'feedback literacy' has been introduced [18], which spans the cognitive, social, and affective capabilities that students need to develop to engage productively with feedback opportunities. It requires students to appreciate the relationship between assessment and learning, understand the nature and meaning of assessment criteria, and become familiar with assessment techniques.

However, it is ultimately students' perceptions of assessment that influence their approaches to learning [10]. In reality, students are often more focused on the time they need to allocate to assessment tasks [19] - and so are preoccupied with marks and passing an assessment - rather than on learning [20]. It has been found in engineering contexts that assessments are central to the student response, regardless of how valuable or worthwhile students experience them to be as learning opportunities [10]. It is these types of challenges that learning-oriented assessment needs to address so that assessment practices encourage student engagement and learning but, more importantly, do not drive approaches that do not lead to appropriate learning. This also emphasizes the need to consider how students perceive and react to their assessment context.

 

III. RESEARCH METHODOLOGY

A.Study context

This study takes place at an engineering school ('the School') in a South African university. Most courses in the degree use a traditional lecture-centered format, and in-class activities are limited owing to the large class sizes, which can exceed 250 students. Some courses use formal, scheduled tutorial sessions during which students work through problems, individually or in smaller groups, with assistance from tutors. Attendance at lectures and tutorial sessions is not compulsory, although students are sometimes 'encouraged' to attend tutorials through spot tests. Practical laboratories take place in dedicated courses. Assignments (or projects), tests, and exams are the primary means of assessing student performance. Exams are invigilated assessments at the end of a course, and typically contribute 5070% toward a course mark. Tests are invigilated assessments that take place during the course, and usually contribute 2030%. Assignments (take-home assessments) can be individual or group-based, and require students to tackle a task over a longer period. These can contribute up to 20%, except in design and lab courses, where these can contribute significantly to the course mark.

B.Research approach

The study presented in this paper forms part of a larger triangulated case study that has also used student surveys [21] and lecturer interviews [22]. This study focuses and reports on student focus groups that were conducted to gain a richer picture of students' experiences and their response to the assessment environment in the School.

C.Method

A question guide, consisting of 13 questions (included in the appendix), centered the discussion on the issues that emerged from the literature, the student surveys, and the lecturer interviews conducted previously [21,22]. Questions encouraged students to reflect on both their own feelings and, where necessary, to comment on what other students might experience or do [23]. The questions probed several issues, including: students' overall approach to their studies in respect of different assessment tasks; how they knew what was expected from them in assessments; an example of a situation in which they were disappointed by an assessment, and how they reflected on this experience; forms of received feedback, and their response to feedback. The focus group ended with questions that aimed to elicit a degree of self-reflection in the students, asking them whether they believed that assessments supported their learning. It also required them to make recommendations of how they, and the School, could change their practices to facilitate and support improved learning. Students were asked to reflect individually by writing down their thoughts on selected questions before engaging in the group discussion. This eased students into a reflective mode of using 'writing as thinking' [24], and allowed them to formulate their thoughts before being influenced by others. The focus groups were facilitated by the researcher and observed by a research assistant. Both took notes, and the sessions were recorded and later transcribed. After each focus group, the researcher wrote a reflective essay to capture observations, mood, social interaction, and initial thoughts [23].

D. Data collection and sampling

The focus groups sampled all the students in the School from the second, third, and final years of study, where failure rates are still high even though students have already passed their first year of study. This suggests that the transition from school to university is not the core reason for the lack of student success, and that interventions and support in the first year do not necessarily create sustainable changes in students that extend into their higher years. The study is also not focused on any particular course or module, as it is believed that student experiences and associated responses to assessment manifest at a holistic level.

Four separate groups of students were chosen for the focus groups, using maximum variation sampling [25]: a high-performing group, a low-performing group, a mid-performing group, and a group of students categorized as turnaround students. The turnaround students performed exceptionally poorly in one year, followed by a year when they performed particularly well. The reason for choosing the groups in this way was twofold. First, because of the wide performance range in the original data set, it was anticipated that some students might not be comfortable speaking out in a group when their performance was significantly different from others in the group. Second, it was hoped that further insight could be gathered on how students in these four groups experience assessment differently. This could prove useful in understanding the relationship between different approaches, experiences, and assessment performance. To categorize the students, academic records were obtained, and all the students were ranked based on the total number of courses they had failed in their second year of study. (Results were extracted as of July 2018.) The population of students, therefore, included second-, third-, and fourth-year students, although the number of courses failed by these students corresponded with every student's second year of study. Emails were then sent to students from all of the groups, inviting them to be part of the focus group on a specified day. The number of emails sent out was increased until five to ten students [23,26] consented to be part of each focus group. A total of 22 students participated in the focus groups. Details of the focus group participants are included in Table I. The performance characteristics for the group (average percentage for all courses, and number of courses failed) correspond with their second year of study. A student fails a course when they obtain less than 50%. The average percentage is the average of percentages obtained for all courses in the year. At the time of the study, students completed 12 courses in their second year. The current year of study is also shown for each group. The high-performing group did not include any second-year students, since no current (2018) second-year students had completed their second year when the data was collected. For this group, high-performing students currently in their second year of study were identified (based on the mid-year results) and invited, but no students agreed to participate. The turnaround students were chosen by analyzing their marks for two consecutive years of study. The characteristics of the emailed students, the number of students emailed, and the responses received are included in Table II. 'First attempt' refers to the number of courses failed and the average percentage for the first time they attempted the year, while 'Second attempt' refers to the year in which they repeated the year/courses. All of the turnaround students were in their third year of study at the time of the focus group.

 

 

 

 

E. Data Analysis

The focus groups ran for between 47 and 95 minutes, depending on the size of the group and the level of interaction. The analysis incorporated both the individual voices and the collective voice [27]. The voice of the individual in this research enabled the researcher to explore the differences between students and to determine the extent to which individuals can be summarized as a collective voice [28]. The analysis, therefore, considered the individual voice, the collective/group voice, and group interactions [29]. 'Group interactions' refers to the sense-making in a focus group as a shared understanding emerges from the conversation [26].

Each focus group was initially analyzed by the researcher, who read through the transcript in conjunction with the notes and reflective essay, and identified emerging concepts, tracked using a coding system. These codes were linked to key supporting quotes (or evidence), making personal notes where appropriate. This provided a rich overview of each focus group, including the process, individual ideas, and group interactions. This process was repeated for each focus group, comparing and adding codes when required. The identified codes and evidence were then captured into a case study database [30]. To organize the data for analysis purposes, it was clustered into topical categories informed by the literature [23]. Once data from all four focus groups had been captured, a comparative analysis was carried out across the four focus groups for each category, capturing similarities and differences. The identified codes under each category were then summarized, compared and, where possible, reduced until a set of emergent themes was obtained. These themes were arranged in a mind map to sequence the data and to create relationships in order to discuss the data logically, but not to identify cause-and-effect relationships. The mind map enabled the researchers to create links between the emerging themes and to provide a framework for comparison and interpretation. To address the credibility of the findings, rich descriptions and evidence were used to support the claims, and efforts were made to avoid being selective or distorting data when presenting the findings. Surprising or 'outlying' evidence was investigated, and rival explanations were considered when interpreting data by constantly referring to the literature [25]. Peer reviews by two colleagues (one from a different engineering school, and one from a school of education) were used at several key points in the research process to challenge interpretations and to raise further alternative views [23]. This interrogation reduced the likelihood that the data was attached to preconceived opinions or conceptions [25].

F.Limitations of the study

Although it is acknowledged that assessment cannot exist in isolation from the curriculum and teaching practices, this study has been narrowed in scope to look specifically at how assessment influences student learning. A limitation of the research is that assumptions are made about the relationship between assessment and student learning. Although this assumption is well-supported by the literature, the study inherently eliminates other external and structural factors from the discussion. The study does, however, consider how teaching practices influence and inform assessment processes, and relevant recommendations are made where appropriate. This is an inherent limitation of interpretive approaches [25]. Although a case study approach can limit the extent to which findings can be generalized to a broader population, they can describe and interpret instances that can be used to understand phenomena or to reinforce or challenge existing generalizations [23].

G.Introduction to the findings

Process observations noted that all of the participants contributed to the discussions. While participants often agreed or reinforced the ideas of others, there were situations in which participants openly disagreed with one another, suggesting that they felt free to participate, and did not feel pressured to have a particular opinion by either the facilitators or fellow participants. It was easy to tell when there was consensus around any participant's response, with laughter being the most common way for students to indicate that they could relate to and agree with what someone was saying. Other forms of telling how the group responded to any participant were the nodding or shaking of heads. Many students were particularly open and candid during the focus groups, sharing personal and emotional experiences.

 

IV. FINDINGS AND DISCUSSION

Selected emergent themes are presented in this section. Quotations are used to provide evidence, referenced back to the corresponding focus group as follows: FG1 - mid-performing students; FG2 - high-performing students; FG3 - low-performing students; and FG4 - turnaround students.

A. Planning, prioritization, and studying for passing

Participants described how they approached their studies on a week-by-week basis during the term. The dominant discussion for all of the groups centered on planning and prioritizing work.

In the high-performing group, all the students spoke about planning and schedules. These students tend to have long-term plans that consider an entire semester of work at a time. Participants also spoke about the fact that, although planned, their work schedules were flexible, based on current work demands. The plans of the high-performing students also appeared to be proactive, anticipating the potential stress caused by deadlines, and accommodating this in their schedules. In the mid-performing group, some students admitted to planning, while others indicated that they did not plan, preferring to approach each day as it came. The planning in this group appeared to be less formalized and proactive. The low-performing students also indicated that they planned, but that the time horizon was much shorter, considering only a week or a month in advance. The low-performing students also seemed to be more reactive, often planning on the basis of the stress that they were already experiencing. The degree to which planning took place, and the time horizon they adopted, were significant differences between the groups, reinforced by the turnaround students, who indicated that one of the critical shifts in their weekly studying routines was toward a more planned approach: "So I have a list of things that I need to do depending on like assignments or tests that are coming up and ja... that's what I do. But before I never used to do that, I used to just...the weekend before a test I'd start studying. " (FG4) Participants elaborated that work was typically prioritized on the basis of the marks or course weighting for the assessment task. Another factor that played a role in the prioritization of work was the perceived 'value' that tasks had for students. It emerged from the interviews that students sometimes struggled to connect the purpose of assignments with the purpose of the tests and exams; and because of prioritization, they were hesitant to spend time on work for an assignment if they did not feel that they would 'need' it in the exam. The following quotation reveals several interesting ideas: "Some courses are structured such that the assignments that they give you are stuff that they can't test in a written test, and usually in those particular situations the stuff that you do for the assignments you won't need for the exam. ... those types of assignments usually get sent to the back of my list because they're not part of the core material of the course and they won't help me with the test and the exam. Whereas with some assignments, ... the assignment is linked to what you 're given in the test and linked to what you'll get in the exams, so doing the assignment is actually helping you prepare for the test and the exam as well. So I tend to like to focus on those. " (FG3) Students, therefore, appear to dedicate more time and effort to assignments that are seen to support the learning process of preparing for exams - which emphasizes the need for constructive alignment [13].

Despite valuing assessments that are geared toward exams, participants also described how these assessments tend to compromise understanding, resulting in a series of behaviors that revolve around trying to replicate answers to problems without deep understanding. Students said that they dedicated many hours to practicing problems, using rote learning approaches to solve standard problem types that are likely to come up in tests or exams, rather than trying to gain a deeper understanding of core material. These 'practicing' behaviors resonate with procedural approaches to learning [31] in which students have a clear intention merely to reproduce knowledge and procedures [32].

"...and if you get hold of the answers, you just look at the common thread throughout the answers, and then you try and formulate your answers based on that. " (FG1) "If I don't understand something. I normally divert to just doing problems then at least even if I don't understand it but if I'm given an equation then I might be able to solve it. " (FG3) These procedural approaches, across focus groups, spill over to other facets of the teaching and learning process, with participants adopting more strategic approaches to learning, lectures, and the purpose of worked examples. Attention becomes more about the method that needs to be replicated than the concept that is illustrated in a particular application. This participant describes the value of worked examples used in class:

".because that's where they actually show us like what method we 're supposed to use. " (FG2)

It also seems as if the pace of course delivery is too fast for low-performing participants, thus affecting their ability to be proactive. Although they try to understand the material and to keep up with the pace, they indicated that they frequently cannot do everything that is required to understand fully the concepts that are being taught in class. This participant described how understanding sometimes needs to be compromised to keep up:

"I had an initial plan, because I had a timetable that I wanted to follow, so my plan was to first prepare for every lecture and study after every lecture. But then it became hard to stick to that plan because some courses maybe it will take me like a lot of time to understand certain concepts and I end up maybe spending more time on one course instead of giving others attention. So from there I now stick to the pressure thing. I will just look at the test timetable, which tests are closer, then I'll study based on that. " (FG3)

A theme that emerged was the notion that students were studying for passing rather than studying for learning. This was prevalent in all of the groups, with this approach of studying for passing selected as a conscious tactic for the high- and mid-performing students, and being used more as a survival tactic for the low-performing students.

Thus the participants start to shift their entire perspective on studying and learning. The focus becomes what needs to be known to pass a particular assessment, rather than a holistic view that incorporates and synthesizes the skills and knowledge needed to address problem-solving tasks in the profession. The participants struggle with this shift, as it compromises their desire for meaningful engagement in order to 'get through' all the material that they need to.

"In second year, I wanted to know, but I ended up knowing more than I had to know. But I ended up being like confused as to what I had to know. " (FG1)

"I remember talking to people who had been here, I remember this guy. So he once told me that, whatever you need to know, just know. But some of the things that you know you don't have to understand. Just do it because you have to.... Okay, you don't know how this thing, why you solved it this way, don't go into knowing why, just do it that way and get the answers. " (FG1) Despite this shift in perspective toward a studying for passing mindset, many participants acknowledged their discomfort with this approach, suggesting that they would prefer to grasp the material and build their understanding, but feel constrained by the environment in which they find themselves. "I wouldn't spend time on a subject that doesn't have an exam or an assignment over one where I need to pass. Ja, it's not who I am but because of time and pressure, I sort of gravitate towards and end up doing that just because I have to pass." (FG1)

Furthermore, the general performance of students in the School suggests that these approaches to studying do not enable students to master concepts and so to do well in tests and exams. A participant described the process of preparing for a test as follows:

"...we did all the question papers they gave us, we did all the question papers and then we were so confident only to go to the test and it was so hectic. So it was really a dramatic end." (FG1)

These findings resonate with those of Fernandes et al. [10] who found a similar tension among first-year engineering students who focused their attention on assessment regardless of the experienced learning value.

B. Assignments, the real world, and group work

The participants generally indicated a preference for extended assignments, which are perceived as less stressful because they have more time to work on them. They explained that, as a result, they gained a better understanding of concepts. "I think assignments are valuable because there's usually like a lot of time, or enough time allocated so... you have... well, I've found that I have. it's easier for me to learn concepts and put them into practice in an assignment as opposed to.in another setting.because, I don't know, I feel like I learn better when I'm not under pressure. So with assignments I feel like they just work better for me because I have had enough time to read up on stuffand, you know, and discuss things with my classmates. " (FG3)

The participants also really value the practical work-based nature of assignments that give them a better sense of relevance to the real world and the engineering profession.

"I think they expose you and then like encourage you to also be able to try to find out some things even those you are not studying what they are... so I think it's exposure. " (FG3)

"I think assignments are better because they help you apply and you get to see, okay, how does what we 're learning apply to the everyday world. " (FG2)

"You 're getting more like practical approaches to whatever you 're being taught. " (FG4)

An aspect of assignments that students struggle with is group work. Many assignments are group projects, and this has an impact on how students view and are able to learn from these assignments. Group assignments involve several individuals; and this affects the control that students have over time management and scheduling. This was raised by participants as one of the features of group assignments that they found most difficult to manage.

"We had a lot of group assignments this year and time management has been an issue always. " (FG3)

"I think assignments are a bit tough because if it's a group project then you can try and schedule it, but sometimes you don't stick to it because there's other people's schedules." (FG4)

More importantly, however, many participants expressed the view that they struggle to work productively in groups. This was more prevalent in the lower-performing groups, who articulated that, although they see the development of these skills as essential, they battle with the learning process. There appears to be insufficient scaffolding to provide students with the necessary skills and to facilitate the process of working in teams.

"I think a valuable skill is learning how to work with people, learning how to get information from people and I think that in the university setting, you know, the best way to learn that is through group work. And I think that is, in that sense, assignments are really good in forcing... in giving the students that skill... for a lot of students it's overwhelming... you don't know how to deal with all these different kinds of personalities, and everything is just happening all at once. So I think that it's something that should be like introduced gradually from first year. " (FG3)

Some participants described negative experiences with group work, which also appears to affect their overall self-confidence and motivation.

"I do value assignments a lot, but I have a problem with a group. whoever is managing the group now, tends to be the boss. He wants to send you there, he wants to send you there, he wants to drag everyone around, he wants to rush everyone, .he wants everything to be according to their own things. Whatever input that you may put in the group, when they type the report. he'll find ways to change your work and put it in his own understanding, or change everything completely. You won't even find your work in there. " (FG3) The participants emphasized how they see real value in assignments, and prefer them to other assessment methods owing to their ability to facilitate learning [15] and their relevance to the real world. This is an important finding, as it relates to the inclusion of project-based and authentic assessment in many engineering curricula [9,10]. However, the issues around group work hinder students' ability to develop effective teamwork skills, affecting the potential for assignments to reinforce learning. This has implications for the design of assignments, especially if the development of group skills is a valuable competence that is facilitated through these methods of assessment. These findings also suggest a lack of focus on the development of self- and peer-evaluation skills that are seen as critical in supporting learning-oriented assessment and the development of lifelong learners [5,9,10].

C. Tutorials, marks, and getting help

Tutorials are one of the few formative assessment practices used in the School. Tutorials can create collaborative spaces that facilitate interaction between students and lecturers [12]. However, the introduction of marks to encourage everyone to attend the sessions appears to confuse the purpose of these sessions, and tends to drive the inappropriate learning practices in students described earlier. Most importantly, it removes one of the few learning spaces that assist many of the students in the School, particularly those that struggle with the pace and workload.

The participants' experiences of tutorials are complex, with some groups indicating that tutorials are one of their most valued assessment methods, while other groups revealed that these are their least favorite. Although there were students who said that they would not attend tutorials if they did not count for marks, introducing marks adds stress to the learning environment of a tutorial, and hinders the learning process of those who need to and want to attend:

"If the tutorial is not for marks then I don't give it attention. I'm probably going to the tutorial class just to tick that off but I think if the option was given, I wouldn't go. And for those that are for marks, then I am forced to work days beforehand such as I am able to ace the actual tutorial. If tutorials are for marks, it adds stress. " (FG1)

Most of the participants suggested that tutorial sessions are the most useful means of assessment, as they facilitate significant conversations between students and lecturers or tutors.

"Some tutorials are more useful, when they are not for marks and actually reinforce the material.and you can get help, it's like a consultation because there is an opportunity if there is something that you don't understand." (FG1)

For participants who find it challenging to approach lecturers, the tutorial environment seems to break down some of these barriers, making it easier for students to approach lecturers with questions.

"I haven't always been one to consult one-on-one with the lecturer but I've found that in a tutorial environment it's very relaxed. It's easier for me to ask questions from the tutors.even from the lecturers, I don't know why but it's just...I just feel like it's a lot less intense. (FG3) There are affordances in assigning marks for tutorials, as it breaks down the process for students who are not able to ask for assistance or work through problems in smaller groups. "Tutorials are (useful) because if you don't understand, you are able to ask the lecturer during the tutorial and they will explain to you. But if it's for marks, there's no way that they can give the solution or help you to such an extent that you get the solution. So the ones that are for marks, we have to understand the work on your own and then come back hoping to get the (marks). But this one, even if you don't understand you know that (attending) the tutorial and (you) will come up with the knowledge. (FG1)

There were some participants, mainly from focus group three, the low-performing group, who seemed to lack the self-regulation to keep up with the work, and the tutorial sessions became a form of 'guided homework' that structured their studying and 'forced' them to keep up with the material. "If tutorials are weighted then like you are forced to work, like to always prepare for it. So staying up to date with the course content it's easier that way. " (FG3)

Although it is important to ensure that students keep up to date, it appears as if some participants become reliant on the 'forced' tutorial environments, and might struggle in higher years when students are expected to use more independent learning skills. There appeared to be a lack of agency and the development of lifelong learning skills in some participants. As a result, the tutorials 'for marks' in the lower years reinforce this tendency, especially since there is no scaffolding that gradually diminishes the support and structure that enables self-directed learning and independence.

Overall, there seems to be real confusion among the participants about the purpose of tutorials, owing to the mixed messages they receive from tutorial structures and design. They struggle to understand what the tutorials are trying to achieve and, as a result, what is expected from them when they interact in this learning environment. The marks that are sometimes allocated to tutorials seem to exacerbate this confusion. Most participants prefer a learning process that starts with reading through the textbook or notes to gain a deep understanding of the material before they practice problems. The forced tutorial sessions may not come at the right time in an individual student's learning trajectory, which forces them to jump over trying to understand straight into practicing the problems without conceptual understanding [31]. The tutorials for marks can, therefore, drive learning behaviors that are misaligned to individual learning because of the mismatch in timing or pace. As a result, tutorials can become 'product' oriented [14] - all about the answers and whether they are right, and not about learning or understanding. The following participant captures the essence of the tutorial environment very effectively:

"I do attend because it's compulsory, but I don't like the whole setup of the tutorial......I don't understand what that approach is supposed to do because I think tuts (tutorials) are supposed to be there for us to help ourselves. So I need to go to a tut if I feel like it will help me. But it's not going to help me if I'm told, okay, you need to complete these questions because chances are you're probably not going to put in all the effort to actually understand it, you're going to do it to complete it, as opposed to actually understanding it. So I think it's... tuts are good for like structure, to help you like keep up with your work, yes, but also in that regard you need to be able to work at your pace and understanding. " (FG4)

D. Expectations, criteria, and feedback

When the participants were asked to describe how they knew what was expected of them in assessments, most shook their heads. This response was consistent across the groups, suggesting that, in general, students do not find it easy to know what is expected from them in assessments. It appears that, in general, expectations for assignments are vague. For tests and exams, participants pointed out that they refer to past papers to give them an indication of what to expect. For assignments, some participants mentioned that consulting with lecturers is the best way to gain a better understanding of what is expected. "... I think when people consult they usually have more insight into what's going to be there." (FG4)

It is striking that this behavior is much more apparent in the higher-performing groups than in the lower-performing groups, who find it challenging to approach lecturers. In these cases, it appears that most participants make use of other students who have already been through the particular course to provide insight into what is expected.

"Each time that I have had an idea of what to expect it's always been from people that have done the course before. And then sometimes the lecturer will like give you a vague overview of what it's like or, you know, what... but it's very vague. It's a lot more. I felt it's been a lot more insightful coming from someone who's actually done the course. " (FG3) Interestingly, participants suggested that they need a high-level picture of what is expected from them in a course, and a view of how the course will unfold over the semester. Past students are seen as the most useful source of this information.

"If there was a way to prepare people for the course then I don't know how but let them know how it's going to happen or how it's likely to happen. " (FG4)

Since most participants find it difficult to understand what the expectations are for any particular assessment, examples were described in which participants felt that they had performed well in an assessment, only to find out that they had done poorly. Participants find this experience very disappointing, and struggle to know what they need to do to improve. For a particular assignment:

'We put in the extra effort... when the marks came back they were so, so low. They were so embarrassing, I can't even say the marks, they were so low. So I was really disappointed because I thought... I considered the effort, the amount of work we put in, the time, I thought that we'd actually get a good mark. So that was really disappointing. " (FG1) This suggests a gap between expectations and criteria, and the weakness in alignment makes it challenging for students to meet the intended learning outcomes [33]. Rubrics were singled out as a particularly useful resource for understanding the expectations around assignments. However, it appears that the use of these rubrics is aligned more to a studying for passing rather than a studying for learning paradigm. Participants are using the rubrics to 'tick-off' sections where they will be allocated marks, rather than using it to guide their overall learning process associated with the assignment.

"Most of the time, a rubric, I find that helpful... .a break down of what they expect. So they'll tell you that the quality of the report it's going to (be) out of five and the introduction is going to be out of ten so you sort of have an idea of what is expected of you according to the mark allocation." (FG1) "Like also in the rubric, it's not always helpful, because some of the lecturers only give it to you when (you've already done the work) and that doesn't help you because then you 're kind of just doing whatever and when you get the rubric you might have to add something in or take out or stuff. So if they give it to you beforehand you then know like how to use your time." (FG1)

"I think getting a rubric also helps. Especially if there'll be a follow or similar type thing, then you can see exactly where the marks are, where you need to improve. " (FG2) Participants find it difficult to know what to expect from assignments with lecturers and past students, seen as the primary means of clarifying what is expected. This suggests a major re-conceptualization of the implementation of assessment criteria and rubrics to accommodate student diversity if learning-oriented assessment [34] is valued as underpinning the assessment process. Rubrics are frequently used in project-based learning; and these findings highlight the importance of clear criteria [10] - but they also emphasize the importance of constructive alignment with outcomes and criteria [13] and, more importantly, with the learning that needs to take place - a learning-oriented view [3]. Depending on the focus group, the responses about feedback were particularly interesting. Participants were either confused by the question, not understanding what was being asked, or immediately responded by discussing marks. These responses highlight the perceived lack of useful feedback that students receive and use. It also raises questions about the extent to which students engage with feedback processes [35].

"feedback from... ? You get feedback on your marks so that's where you see if you 're doing good or bad, but if there's an assignment there is usually a rubric and a comment, can you please repeat the question? " (FG1)

"I'd say, personally this School doesn't really give feedback, like from their side, all they do is just give marks. So. and then depending on the type of person that you are, then you'll actually follow up why I got a specific mark. .the only time when we actually get feedback (is) with assignments because it's very detailed. There 's a rubric and everything, sometimes the lecturers do sort of make side comments on the reports. " (FG1)

Most participants commented that, if they wanted feedback, they would need to consult with a lecturer.

"I think most people see feedback as important when they don't do well. When they don't do well they will try to understand it but when you do well, I don't really see people going to see people speak to a lecturer if I don't do well or I fail, I go and see the lecturer. " (FG2)

Emerging from this quotation are two themes associated with feedback - the focus on marks, and the role of consulting with a lecturer as a feedback mechanism. Ultimately, feedback and the associated consulting process are generally focused on marks, aligning with the theme of studying for passing. Participants are not trying to improve their understanding or learning processes, but are trying to improve their mark - either for the current assessment task or for future tasks. Students in all four focus groups made this point:

"... the interactions that you end up kind of having with the lecturers are okay, why are my marks so low. it's not okay, help me understand this concept, it's my mark needs to advance...so at the end of the day, it's all about marks." (FG2) "I think feedback is very one-dimensional in this School. There's... from the very first year you get discouragedfrom sort of checking your marks if they're correct. so by the time you get to the later years you don't even care about checking your marks; if it's like that, it's like that, you move on." (FG3)

Despite the learning affordances of rubrics, participants also use rubrics to increase their marks and not as an opportunity to reflect on their strengths and weaknesses. Thus they do not see value in any feedback for an assignment if they are not going to get something similar again, since the rubric is only used to 'tick boxes' against mark allocations, rather than to change learning behaviors and approaches to solving problems that would be universally useful in future assignments.

"feedback on assignments is only useful if you get the same one again, but if it is a course on a random assignment then it is not exactly helpful in any way. " (FG1)

Even improvement is described by this participant as merely finding out where students can get more marks:

"I think getting a rubric also helps. Especially if there'll be a follow or similar type thing, then you can see exactly where the marks are, where you need to improve." (FG2)

The mindset around feedback and 'mark-seeking' seems to extend beyond the students to the lecturers as well. The following participants described how they try to use consultation opportunities for feedback on how to gain a deeper understanding of concepts, but expressed their frustrations at the limited success of this as an option.

"Because sometimes you go there and they are thinking that you are here to fight for more marks. And then like maybe you 're trying to understand more. When you get there sometimes some lecturers when you ask them, okay, I didn't get this correctly so how was that. like they want you to answer it again and like you will put the idea on the paper and it didn't work, so sometimes you think that, no, going there is just a waste of time." (FG3)

"I also think just the devil is in the detail with some things, you know. I think lecturers sometimes take for granted the value of answering a small question, or a question that they deem is not worthy, type thing. And for some people that question was the cornerstone to opening the entire course for you, or the entire section that you were doing. So that answering, I think if lecturers could focus on being cognizant of the fact that we 're not all in the same headspace and our understanding procedures are all different, I think that would make things easier because then they'd answer all questions no matter how dumb, or how dumb you sound. And I think you do get lecturers that if you ask a dumb question they ridicule you, and then you can't approach them any more. " (FG3)

These experiences surface the issue of lecturer approachability. There is an additional idea that emerges around the notion of threshold or bottleneck concepts [36], as it seems that lecturers are not aware of the key concepts that students need to understand to unlock further knowledge in the discipline. This inhibits lecturers' ability to explain concepts, drawing on their expert knowledge in a way that enables students to gain a deeper understanding of the course material. Students' ability to self-evaluate and improve is limited by the lack of clear expectations for and feedback on assessments. Without these key elements, students are unable to see where they are going wrong or to develop ways of reflecting, changing, and improving. The following participant sums up this frustration:

"You work really hard on your assignment and you get it back and you realize you've gone wrong, and it gets you really down because you spent so much time doing it and it doesn't show you what's wrong. " (FG1)

Students also appear not to be developing self-evaluation skills that allow them to develop a level of independence from feedback to align with a learning-oriented assessment paradigm [16].

 

V. CONCLUSIONS

The findings of this study on student perceptions of assessment practices confirm that assessment plays a critical role in determining the planning and prioritization of study processes and student learning strategies in this engineering context. The intention of students when choosing learning strategies is strongly influenced by their perceptions of the assessment environment, and can lead to the adoption of surface approaches to learning. This, in turn, can undermine the development of lifelong and sustainable learning skills [12]. Excessive workloads and conflicting assessment tasks [5] can further amplify unproductive approaches to learning. The authenticity and relevance of assessment tasks also influence student engagement and motivation.

Participants in the study emphasized how they saw real value in written assignments, and preferred them to other assessment methods because of their flexibility and potential to facilitate learning. Participants also felt that these assignments were more authentic and gave them better access to developing the skills they needed for the real world [9]. There appears to be insufficient scaffolding for the development of the group skills that these assignments require, reducing the extent to which they are oriented toward learning [3,16]. It emerged from the interviews that this could lead to experiences that affect the self-confidence and motivation of many students, particularly those who are already struggling. The issues about group work hinder students' ability to develop effective teamwork and communication skills. This has implications for the design of assignments, especially if the development of group skills is to be facilitated through learning-oriented methods of assessment [13].

Assessment practices in the School have limited ways of communicating expectations and feedback to students, making it difficult for them to develop self-evaluative skills [10,16]. One of the primary strategies participants use to fill these gaps is through consultations with lecturers and the development of student networks, particularly for participants who find it difficult to approach lecturers.

Tutorials are one of the few formative assessment practices that the School has established; and they have the potential to create collaborative spaces that facilitate interaction between students and lecturers [12]. Tutorial sessions are being used by students to approach lecturers and obtain feedback as a result of reduced barriers in these more informal settings. Forcing attendance at tutorials by creating a testing-oriented environment destroys this possibility, and closes down alternative learning possibilities. Most importantly, it removes one of the few learning spaces that assist many students in the School, particularly those who struggle with the pace and workload demands. Although tutorials are often seen as a very useful formative assessment process, the current purpose and structure of tutorials is not constructively aligned [13]. This means that tutorials are unable to meet diverse student needs, and are not oriented toward learning [3].

Although participants find themselves adopting studying-for-passing behaviors, there is evidence that they have a strong desire to move toward practices that develop deeper levels of understanding.

This study shows that the selection and design of assessment strategies and tasks has a significant impact on student learning, and that there needs to be a constructive alignment [13] between assessment, curriculum, and teaching design to create a learning-oriented assessment environment [34]. Furthermore, the findings based on focus group interviews provide valuable insights for transforming the structure and culture of assessment practices and for addressing the rigid divide between formative and summative assessments. This critical examination of assessment practices is particularly significant because it acknowledges the strength and value of exploring the student voice across performance levels. The recognition of student experiences of assessment through such empirical studies contributes to the ongoing development of pedagogically driven assessment practices to enable student access, engagement, and success in engineering and other higher education contexts.

 

References

[1] M. Prosser, & K. Trigwell, K. Understanding learning and teaching: the experience in higher education. Buckingham: Open University Press. 1999.

[2] F. Marton, & R. Saljö. "Approaches to learning." In. Marton, F., Hounsell, D. and Entwistle, N. (Eds.) The experience of learning. Edinburgh: Scottish Academic Press. 1984.

[3] D. Carless. "Exploring learning-oriented assessment processes." Higher Education, 69(6), pp.963-976, 2015.         [ Links ]

[4] G. Gibbs. "How assessment frames student learning." In. Bryan, C., & Clegg, K. (Eds.). Innovative assessment in higher education. Oxon: Routledge, 2006.

[5] O. Rompelman, "Assessment of student learning: evolution of objectives in engineering education and the consequences for assessment." European journal of engineering education 25 (4) 339-350. 2000.         [ Links ]

[6] S. Kocdar, A. Bozkurt, & T. Goru Dogan. "Engineering through distance education in the time of the fourth industrial revolution: Reflections from three decades of peer-reviewed studies." Computer Applications in Engineering Education. 2020.

[7] A.F.A. Rahim. "Guidelines for online assessment in emergency remote teaching during the COVID-19 pandemic." Education in Medicine Journal, 12(3). 2020.         [ Links ]

[8] M.L. Cruz, G.N. Saunders-Smits, & P. Groen. "Evaluation of competency methods in engineering education: a systematic review." European Journal of Engineering Education, 45(5), 729-757. 2020.         [ Links ]

[9] K. Roach, E. Tilley, & J. Mitchell. "How authentic does authentic learning have to be?" Higher Education Pedagogies, 3(1), 495-509, 2018.         [ Links ]

[10] S.Fernandes, Flores, M.A. & Lima, R.M. "Students' views of assessment in project-led engineering education: findings from a case study in Portugal." Assessment & Evaluation in Higher Education 37(2), 163-178. 2012.         [ Links ]

[11] K. Sambell, L. McDowell, L & C. Montgomery. Assessment for learning in higher education. London: Routledge, 2013.

[12] H. Fry, S. Ketteridge & S. Marshall. A handbook for teaching and learning in higher education: Enhancing academic practice. Third Edition. New York: Routledge, 2009.

[13] J.B. Biggs. "Constructive alignment in university teaching." HERDSA Review of Higher Education, 1(5), pp. 5-22, 2014.         [ Links ]

[14] P.T. Knight. "Summative assessment in higher education: practices in disarray." Studies in Higher Education, 27(3), 275-286. 2002.         [ Links ]

[15] G. Gibbs & C. Simpson. "Conditions under which assessment supports students' learning." Learning and Teaching in Higher Education, 1, pp. 3-31, 2004.         [ Links ]

[16] K. Willey & Gardner, A.P. "Investigating the potential of self and peer assessment to develop learning oriented assessment tasks." In Annual Conference of Australasian Association for Engineering Education., The University of Adelaide, 2009.

[17] P. Dawson, D. Carless, & P.P.W. Lee. "Authentic feedback: supporting learners to engage in disciplinary feedback practices." Assessment & Evaluation in Higher Education, 46(2), 286-296. 2021.         [ Links ]

[18] D. Carless, & D. Boud. "The Development of Student Feedback Literacy: Enabling Uptake of Feedback." Assessment & Evaluation in Higher Education. 43 (8): 1315-1325. 2018.         [ Links ]

[19] L. Scott & C. Fortune. "Towards the improvement of the student experience of assessment and feedback in construction management education." European Journal of Engineering Education, 38(6), 661 -670. 2013.         [ Links ]

[20] S. Bloxham & P. Boyd. Developing Effective Assessment In Higher Education: A Practical Guide. London: McGraw-Hill Education, 2007.

[21] T.S. Hattingh, L.Dison, & L.Woollacott. "Student learning behaviours around assessments". Australasian Journal of Engineering Education, 24(1), 14-24. 2019.         [ Links ]

[22] T.S. Hattingh & L. Dison. "Why lecturers in an engineering School assess the way they do". Research in Engineering Education Symposium, July 2019, Cape Town, South Africa, 2019. Available at: https://www.reen.co/papers-proceedings. 2019.

[23] S.B. Merriam. Qualitative Research: A guide to design and implementation. San Francisco: Jossey-Bass, 2009.

[24] G. Gibbs. Analyzing qualitative data. London: Sage, 2007.

[25] L.Cohen, L.Manion & K. Morrison. Research Methods in Education, 7th Edition Oxon: Routledge, 2011.

[26] J.W.Creswell. Educational Research: Planning, conducting and evaluating quantitative and qualitative research, 4th Edition, Boston: Pearson Education, 2012.

[27]

[28] S. Wilkinson. "Focus groups: A feminist method." In. S.N. Hesse-Biber& M.L. Yaiser (Eds.). Feminist perspectives on social research. Oxford: Oxford University Press, 2004.

[29] R. Barbour. Doing focus groups. Thousand Oaks: Sage, 2008.

[30] K. Willis, J. Green, J. Daly, L. Williamson & M. Bandyopadhyay. Perils and possibilities: achieving best evidence from focus groups in public health research. Australian and New Zealand journal of public health, 33(2), pp. 131-136, 2009.         [ Links ]

[31] R.K. Yin. Case study research: Design and methods. 5th edition.

[32] J. Case & D. Marshall. "Between deep and surface: procedural approaches to learning in engineering education contexts". Studies in Higher Education, 29(5), pp. 605-615, 2004.         [ Links ]

[33] K. Crawford, S. Gordon, J. Nicholas & M. Prosser (1994). "Conceptions of mathematics and how it is learned: The perspectives of students entering university". Learning and instruction, 4(4), pp. 331-345, 1994.         [ Links ]

[34] J. Biggs, & C. Tang. Teaching for quality learning at university. London: Open University Press. 2011.

[35] D. Carless. Excellence in University Assessment. Oxon: Routledge. 2015.

[36] D. Boud, D., & E. Molloy. Feedback in Higher and Professional Education: Understanding it and doing it well. Oxon: Routledge. 2013.

[37] J. Middendorf, & D. Pace. "Decoding the disciplines: A model for helping students learn disciplinary ways of thinking." New directions for teaching and learning, (98), 1-12. 2004.         [ Links ]

 

 

Teresa S. Hattingh received the B.Sc. and M.Sc. degrees in mechanical engineering from the University of Witwatersrand, Johannesburg and a Ph.D. degree in education, also from the University of the Witwatersrand, in 2019. From 2008 to 2018 she was a lecturer in industrial engineering at the University of the Witwatersrand and has been a lecturer in industrial engineering at the North-West University in Potchefstroom, South Africa since 2019. Her research interests include assessment and curriculum transformation in engineering education, and the application of industrial engineering in emerging contexts. She is also vice-chair of the Research in Engineering Education Network, and is an associate editor for the Journal of Engineering Education Transformations and a section editor for the South African Journal of Industrial Engineering.
Dr Hattingh holds a number of teaching excellence awards, including the Distinguished Teaching Excellence Award from the North-West University and the Vice Chancellor's Transformation Award from the University of the Witwatersrand.

 

 

Laura Dison was awarded her PhD in 2009 entitled Higher order thinking in transition: a case study of first year university students based on her involvement in the extended curriculum programme in the Faculty of Humanities. She has worked at the University of the Witwatersrand as a teaching and learning specialist for 28 years. In 2010, she established the Wits School of Education Writing Centre and worked with lecturers to design embedded writing interventions in Education disciplines. Since 2015, she has worked as a senior lecturer in the Curriculum Division and is the Coordinator of the Post Graduate Diploma in Education in the field of higher education, a professional qualification for lecturers. She became the Academic Head of teaching and learning at the School of Education in 2017 and was appointed Assistant Dean for Teaching and Learning in the Faculty of Humanities in 2020. She has written on the role of reflection in professional qualifications and co-edited a book with Sherran Clarence, Writing Centres in Higher Education. Working in and across the disciplines (African Sun Media, 2017). She supervises several postgraduate students in curriculum and assessment

 

This work was supported in part by a South African Society of Engineering Education (SASEE) grant.
Based on "Exploring student perspectives on assessment practices in an engineering context" by Teresa Hattingh and Laura Dison which appeared in the Proceedings of2020IFEES World Engineering Education Forum - Global Engineering Deans Council (WEEF-GEDC), virtual conference, 16 - 19 November 2020.

 

 

APPENDIX - QUESTION GUIDE

Questions in bold: for individual self-reflection. Questions in italics: for further probing where required.

1. How do you approach your studies on a week by week basis throughout the term?

2.How do you go about preparing specifically for a test or an exam?

3. How do you know what is expected of you in your assessments?

4. Can you share an example of a time when you have been disappointed by your result for a assessment?

5.What do you think that the reasons for this were and what did you do about it?

6. How do you experience feedback?

7. In the lecturer interviews that were conducted, lecturers indicated that they were hoping that students would be more curious about and engaged in what is going on in the engineering world? Do you have any thoughts on this issue?

8.Can you think of ways in which you could explore engineering beyond what happens in the classroom?

9. In the lecturer interviews that were conducted, lecturers indicated that attendance at lectures is often poor Can you comment on why you (or other students that you know) don't attend lectures?

10. Do you think that the way that assessment is set up in the School enables you to gain a good understanding of concepts?

11. Has this process of chatting about learning and studying provided you with any insights or ideas that may affect the way in which you approach your studies in the future?

12. Can you think of at least two ways in which you may change the way that you approach your studies in future?

13. Can you think of at least two recommendations that you would give a lecturer on how they could support your learning better?

Creative Commons License All the contents of this journal, except where otherwise noted, is licensed under a Creative Commons Attribution License