A SOUTHERN AFRICAN SILVER ANNIVERSARY MEETING, 2014 SOMP

 

An evaluation of the effectiveness of teamwork, with an emphasis on peer assessment and peer review, in an introductory engineering course

 

 

C. Daly

School of Mining Engineering, University of New South Wales, Sydney Australia

 

 

---

SYNOPSIS

The Faculty of Engineering at the University of New South Wales (UNSW) offers a core first-year engineering design and innovation course, ENGG1000, undertaken during the first and second semesters. This course is highly regarded in the sense that it provides an introduction to many concepts and activities that students will experience over the four-year minimum for which they are undergraduates at UNSW. Approximately 1400 students enrol in the semester 1 course across the Faculty, typically 80 of which undertake the Mining Engineering stream.
Students in teams of between six and eight design and construct a physical model to represent an aspect of their chosen discipline. For example, in 2013 the mining engineers designed and built a model dragline.
This paper concentrates a major aspect of the course - the involvement of team members in group activities and the development of the associated skills of peer assessment and peer review as the course progresses over a period of 12 weeks.
The term 'peer assessment' in this paper refers to the requirement for students to assess the design components of their peers. This course has a structured requirement in terms of how a successful design is a result of a sound design process rather than a 'try and see' approach. Each student must describe in detail the process they undertook to achieve their final design - hence the approach is independent of the discipline and/or project selected.
Peer review is a process whereby students review the contribution of their team members to the overall design. This activity encourages team involvement and interaction. The final assessment mark can be moderated by the outcome of this peer review, although it is run twice during the semester. The first 'run' is for feedback only during week six and hence no moderation is undertaken.
It was found through consultation with students and from questionnaires that both processes are well accepted and highly regarded by students, as they give them a degree of ownership of the assessment process. In addition, the processes provide rapid and relevant feedback on the progress of individual students.
Peer review and peer assessment are also considered to be very valuable tools for use in courses in succeeding years. For instance, many of the courses in mining engineering rely heavily on group assessment tasks.

Keywords: teamwork, peer assessment, peer review, Moodle, engineering design, first year.

---

 

 

Background

The Faculty of Engineering at the University of New South Wales (UNSW) in Sydney, the largest engineering faculty in Australia, comprises nine independent schools. These are the Graduate School of Biomedical Engineering, the School of Chemical Engineering, the School of Civil and Environmental Engineering, the School of Computer Science and Engineering, the School of Electrical Engineering and Telecommunications, the School of Mechanical and Manufacturing Engineering, the School of Mining Engineering, the School of Petroleum Engineering, and the School of Photovoltaic and Renewable Energy Engineering. The School of Materials Science and Engineering is based in the Faculty of Science but offers a similar first-year programme. Approximately 1800 undergraduate students join the Faculty each year to undertake an essentially common year comprising eight courses, including the core courses of Design, Mathematics, Physics, Computing, and Mechanics plus three electives. Most students join a discipline at the commencement of year 1; however, approximately 15% enrol in what is termed a 'Flexible First Year' where in addition the common core units, students can choose electives from any of the 10 school-based disciplines. At the completion of their first year, students must choose a discipline to commence in year 2. This approach caters for students who on entry have not decided their discipline. Dual degree programmes can also be selected, ranging in duration from 5 to 6 years but still based essentially on the common first year.

ENGG1000, Engineering Design and Innovation, is an introductory engineering design course offered twice a year. In semester 1 approximately 1400 students enrol, with around 400 in semester 2. I am currently course convenor of the faculty-wide semester 1 course. Each school provides a course coordinator forthe design project they offer. Projects can range in size from 40 to 270 students. There are 14 projects offered by the 10 schools. The course runs centrally for the first 2 weeks and then for 10 weeks it moves to the school level.

 

What does ENGG1000 set out to achieve?

Many tertiary institutions offer a similar course in year 1. The value of such a course in introducing students to engineering design is well recognized. For example, the University of Queensland offers ENGG1100, which operates in a very similar manner but with slightly less numbers. However, the major difference is that the course is centrally managed for its duration rather than schools managing their own projects. This approach is advantageous from an administrative point of view, and also promotes more cross-disciplinary interaction. Students choose from a small number of projects managed centrally. In addition, students attend common lectures each week - a distinct advantage compared to UNSW where lecture theatre capacity limits this. Other examples of successes in this approach include the Multidisciplinary Design Project at the University of Twente (UT) Netherlands (Vos et al., 2000) and in the USA at the University of Tennessee (Parsons and Klukken, 1995), where a similar course was developed to impart the core engineering values of being complete problem solvers, innovators, and the ability to collaborate with peers in solving a team-based problem.

The other major reason for ENGG1000 relates to personal development. The ability to function as a member of a team is today considered an almost essential requirement at university and in the workplace, especially in an engineering discipline. However students often enter year 1 of university with little or no experience of team membership (Dutson et al., 1997) and can find the whole process quite daunting. However Dutson also comments that this inexperience can lead to poor leadership, poor communication and procrastination, internal conflicts etc. As I will show later, this does not appear to be the current case with ENGG1000. Maybe it is because the Australian high school system has changed over the past 20 years. From personal communications it appears that team and group work is quite common in Australian high schools. It could also relate to the structure of the ENGG1000 groups: a team leader is appointed who is committed to being a team leader. The appointment of senior undergraduate students as mentors, rather than academics, often means that a closer relationship develops between mentor and team members not only due to the similarity in age, but mentors are seen as colleagues rather than lecturers. However this does not mean that issues and conflicts do not arise. They do, but from my experience in limited numbers and often relating to group members having commitments that limit the time they can devote to their project. All team meetings are scheduled at mutually convenient times so that team members should be able to attend.

In addition, a very successful outcome of this approach to experiential learning is the wealth of feedback that is available from team interaction, from the peer assessment, and the peer review tasks (McAlpine and Reidsema, 2007). It has always been recognized that feedback to students is an essential aspect of the learning process, but is something we do not do well.

Learning outcomes and accomplishments

The learning outcomes of this course are quite extensive and include the requirement that students:

> Be familiar with the process of engineering design and the use of design methods for defining an open-ended design problem, generating alternative conceptual solutions, evaluating these solutions, and implementing them

> Understand the basic elements of project management and be able to plan and schedule work activities in accordance with standard practice

> Understand the dynamics of collaborative teams and how to work effectively within a team to accomplish tasks within given deadlines

> Be able to organize, conduct, and record engineering meetings

> Be able to effectively convey thoughts and ideas in an engineering design report

> Be able to understand the issues of quality, safety, diversity, and equal opportunity as they apply to university and professional life

> Understand the roles and responsibilities of a professional engineer.

Learning outcomes and the assessment framework

ENGG1000 has been designed to ensure there is equivalence and alignment between the implementation of the course by the various schools. Each school operates within an agreed framework of learning outcomes as indicated in Table I. The course convenor provides administrative support to each school, which is able to modify its offering up to a maximum of 20% in line with the school's own preferences or expertise.

After completing one week of introductory classes, students select one of 14 projects offered across the Faculty. Students are encouraged to choose a project outside their selected discipline. For example, a Civil Engineering student could choose the mining project and a Mining Engineering student the civil project. A cross-discipline approach is quite appropriate to this type of course and also provides a basic experience of another discipline, which is particularly important for students enrolled in the flexible first year and who are required to make a discipline selection at the end of year 1.

From the above it is clear that this course emphasizes teamwork, communication skills, and an introduction to what engineering is all about. Irrespective of the project chosen, these skills are developed and each student is able to easily change disciplines at the completion of year 1.

Structure of the course

It is a challenge to provide an introductory or welcoming lecture to 1400 students on their first day at university. Our largest lecture theatre holds 1000 students. We have experimented with a few ways of lecturing to all students at one time. One attempt involved the video distribution of the main live lecture to other locations on campus. This entailed many challenges, including technical issues, and was used only once. Such an approach distances the student from the presenter.

As a potential solution in 2013, two theatres that hold 1000 and 500 students at the same time were used. This approach meant the duplicating of the material presented. For example, I would take the larger lecture and a colleague would take the other using exactly same Powerpoint slides. Students were required to enrol in one of two classes with associated locations to ensuring not all arrived at the same venue. Overall, this was quite a satisfactory approach and will be used again.

The first challenge - impromptu design

The overall philosophy of Engineering Design and Innovation is to generate scenarios in which students must work together in small teams with the ultimate aim of not only producing quality group submissions, but also to gain experience in working together and becoming comfortable with assessing the quality of their peers' work and contributions to a task.

The initiation into the basics of teamwork to achieve a common goal commences in week 1 with an impromptu design exercise. This task requires all 1400 students to be involved. A three-hour period is set aside and 30 classrooms booked. There are 700 places available for one hour to create a design and then to demonstrate the design to a judging panel. Students work in groups of eight. The groups form spontaneously as they enter the classroom. Each classroom has two or three staff to supervise the process and present the design brief for the first time.

 

The design brief for semester 1, 2013 - water tower challenge

You are a team of design engineers and have been appointed by your engineering firm to prepare a bidfor tender of a new multi-million dollar development being put up by 'Sydney Power'. The development is to design a fully sustainable water tower/reservoir capable of holding a large volume of water at a high elevation, which is pumped up during day using a solar powered pump. It can be used as a back-up power source during peak energy usage times or power outages by releasing water into a lower reservoir and past a turbine. The mechanical energy from the flowing water is transferred into electrical energy and diverted back into the main power grid. This is important as a coal-fired power station requires around a week's notice to adjust its power output to meet demand.

To design and build a scale model (1: 100) of the water tower structure capable of supporting/holding 100 marbles (water) for 10 seconds. Your aim is to build the tallest structure in order to produce the most potential power from your design. Emphasis will be placed on an innovative design, the aesthetics of the design and the overall performance of the structure under loading. Be aware that your tower must have some way to hold the marbles. The tallest tower to support the most marbles for the specified time wins!!'

Figures 1 and 2 represent a typical group working on building the tower from a range of 'materials' provided. Figure 1 shows the materials provided including paper cups, drinking straws, 'paddle pop' sticks etc. Each team is provided with the same materials and project specification.

 

 

 

 

A brief online survey was held after the completion of the task, in which students were asked to rate their experience. The responses (Figures 3 and 4) are quite positive. The day is hectic and there are always challenges. One of the main issues is that despite considerable reinforcement, students still do not know where to go on the day. This means that some students are often late, and timing is very important as they only have 50 minutes to complete the task. Another issue has been congestion at the testing 'station' due to the number of teams arriving simultaneously to be assessed. Next time we will arrange for more assessors.

 

 

 

 

Team Builder

Once all students have selected their project, most schools then require access to a Moodle application - Team Builder. This application is basically a very brief survey - it asks questions relating to an individual's experience with hand tools, teamwork experience, interest in being team leader, writing ability etc. The resulting data is used to create groups that have an appropriate mix of skills. This approach is regarded an improvement over the more common random assignment of students to groups. Hence by the second class of week 2 all students are assigned to groups and are asked to meet. When students arrive at this class they are moved into their groups and a mentor is assigned. A mentor is typically a senior undergraduate student who is paid an hourly rate and meets with their group for an hour or two each week.

Mentors are appointed to act as guides during the design process. They are not meant to give suggestions or assist in the construction of the model, but provide support from their previous experience, and are able to comment on the path that the team may wish to follow and provide guidance if the group is not working well together.

A structured overview of the design process

It is widely accepted that engineering design is a systematic process of analysing the problem, creatively considering a range of potential solutions, then evaluating the solutions in relation to the requirements of the task until a final solution is reached (McAlpine and Reidsema, 2007). In ENGG1000 the design process is completed in a series of phases throughout the semester (Figure 5):

Phase 1 Formulating the problem to identify the range of aspects of the task that may be investigated further. This leads to a statement of the design problem

Phase 2 Conceptual design - generating a range of design concepts for solving the problem

Phase 3 Evaluation - critique and evaluate the proposed concepts to select the best solution

Phase 4 Detailed design - refine the solution and consider implementation issues

Phase 5 Implementation - building and testing the design prototype.

The first three phases involve peer assessment where a student submits their own contribution to each phase. The student then self-assesses their assignment. In addition, three randomly selected students assess the submission. Each phase has an assessment value of 7%, 80% of which comes from the averaged assessment of the three peer assessors and 20% from the 'effort' a student puts into the peer assessment - including feedback etc. The assessment stage of each phase takes approximately one week. This means that quality feedback is received quite rapidly.

 

Why peer assessment?

As this course progresses the active involvement of the students in the assessment increases. It becomes a very student-centred approach to design, which in many cases is quite appropriate as the course is all about exploring potential designs as well as how to achieve the selection of a final design. It is experiential learning. There is no pre-determined answer - students will reach a conclusion at the end of week 12. They will have presented a final design and gained from the experience of developing this final design. The process can be chaotic at times - individuals have their own ideas on what is a good design but it is the team's responsibility to put forward a methodically evaluated design that will have the best chance of success. The process is also staged in a way that each student must contribute to the team and to the team's project each week. There is no easy way to avoid responsibilities. The success of the project depends on the contribution of all team members. In addition, each week a student must complete a reflective diary entry via a Wiki to document their contribution for all team members to see.

This course was originally developed to provide a unique learning opportunity for students entering into the first semester of year 1 engineering. Besides this course, students enrol in more traditional courses including mathematics, physics, and an introduction to discipline course. The latter courses essentially operate by requiring individual assessment tasks to be completed. However, it was felt that more realistically, students will be required to operate in teams or groups in the later years of their studies and in their eventual workplace. ENGG1000 was planned not only to provide an introduction to the design process, but also to introduce students to teamwork and give them an understanding of the assessment process. Along with this, of course, is the added advantage of a unique opportunity for peer feedback. Lack of meaningful or timely feedback is considered one of the major concerns of students. Time-challenged academics often do not return adequate feedback to students. Often a student will gain, say, 8/10 for an assignment without any comments on why (Race, 2001).

There is considerable literature available on peer involvement in the design process - most of which is quite positive in terms of the students' and the teachers' experience. Phil Race is regarded as one of the experts in this field. Towards the end of this paper I present data gained from a recent survey of students. It is important to state that assessment is according to a rubric. A rubric is made available to all students very early in the process, allowing them to prepare their assignment along the lines of the rubric with clear knowledge that it will be self- and peer-assessed. This removes any concern or confusion over how an assignment will be assessed and at least provides the basic feedback that students will receive.

What are the advantages of peer involvement?

There are many advantages of this approach discussed in the literature, but I consider the most important being the fact that it involves the students in the whole process of completing, submitting, and reviewing an assignment. It gives them more ownership of their learning. It promotes the concept of deep learning as opposed to surface learning -reflection is basically forced on the student. It appears that students are more concerned that another student will see their work compared to their lecturer seeing it, and they undertake more effort so as not to be potentially embarrassed (Kennedy, 2005)

I feel that a major component of the peer assessment process is the associated self-assessment task that I include in all similar assessment tasks. This really requires a student to focus on their own contribution before assessing what their peers have submitted. This activity further involves students in the assessment process and hence the learning process (Gibbs and Simpson, 2004).

In addition, a student can undertake the peer assessment process in their own convenient time, in a suitable place where they can essentially spend as much time (within reason) as they wish completing the assessment task. Hence I really see the advantage of the online assessment process as opposed to a more public class-based environment. From my experience, students in an online environment give thoughtful and detailed feedback because it is what they would like to receive themselves. This is of course, not always the case as there are those that resist this form of participation.

The following comments highlight the strength of this approach.

From a current student:

Please give me good marks I need this omg I'mfailing please help me please

And this response from a peer:

'Nice title. Unlikely to workfor actual examiners, but as this is peer-assessed it's easy to see how you thought it was worth a shot. Rather than just giving you good marks though, I thought I'd be honest but fair and help explain why you might be failing', which is probably more useful for you in the long run anyway. Results section was good. Individual work bit covered how you discerned between design ideas, however you didn't mention your design goals and how you decided which goals were the most important. Without that it's hard to tell what criteria you used to judge which designs were better than others.'

However, the detractors will say that assessment is a lecturer's responsibility, they know best and are able to assess all students independently. Students will also say that they are not able to assess. However, as I have often discovered, if the process is explained carefully and the support to undertake the assessment is provided via a rubric, students are more accepting, especially when they realize that the feedback they will receive is rapid and beneficial. It is important that the contribution towards the final assessment for the course is reasonable. I recommended and use 21% for the three phases in this course, and this is probably a maximum for this type of course.

Phase 1 as an example

The following is the information provided to the students regarding phase 1.

For this module, individually and in teams, you will develop a working problem statement that will guide your decision as you progress through the design process. There are many structured approaches toformulating the problem - a number of these approaches are described in Chapter 3 of Dym's Engineering Design textbook. In preparationfor the first Learning Portfolio exercise, you will use one or more of the techniques outlined in Chapter 3. These include questioning the client (ie the authors of the project brief or their representatives) and brainstorming.

Individuals will develop problem statements along with objectives (goals) and constraints. Then teams will use techniques described in Chapter 3 to refine the tentative problem statements, resulting in one working problem statement for the team. In the Learning Portfolio entry, you will reflect on this activity.

Individual task

Read the relevant sections of the text (Section 3.1 as well as Chapter 1 and 2 if you haven't read them already). Develop your own tentative problem statement, including objectives and constraints.

Group work

1. Present your refined problem statement to your team. Note theirfeedback or suggestions.

2. Break into a few subgroups of 2-3 and select a problem statement that was not written by a member of the subgroup. Use brainstorming or another method to refine the problem statement.

3. As a team, write a working problem statement. Note the date and time and put a big red box around your problem statement. Later, you can reflect on your first attempt at a problem statement.

Be sure to note the date and time, as well as the names of the participants, for your entries.

Follow up assignment

When your team hasfinished the above activity you are ready to do the Phase 1 Portfolio submission detailed in the Project Plan.

> Learning Portfolio Phase 1 submission-Length: 500 to a maximum of 800 words.

> Results-Write the team's problem statement.

> Reflection-Consider how your thoughts and ideas about the project have developed during the problem statement phase of the design project. Include the following headings in your reflection of the process.

> My original problem statement-Include under this heading your original statement, what you understood about the problem statement process and how effective was the way you developed your statement.

> Team problem statement refinement process-Include how your team went about this process and how effective the process was in helping the team members develop their ideas.

> What I learned about design and teamwork?-Include in your reflection:

- how you think the team will approach the design problem and

- what experience and ability to learn do you have that will help you to make a positive contribution to design and teamwork

 

How is the student assessed?

> Criterion 1-Does the student clearly show a good understanding of the techniques available and explain how the technique was used to generate a problem statement?

O No

O Yes

> Criterion 2-Does the student include a description of the group's application of a recognized approach to the development of a problem statement?

O None

O One

O More than one

> Criterion 3-How clearly does this student demonstrate an understanding of the problem statement process and the techniques as applied by the student and the group?

O Limited understanding and application

O Reasonable understanding and application

O Clear understanding and application

> Criterion 4-Does the student appear to have learned how to contribute to the team process and how to help the team to be effective?

O Limited understanding and application

O Reasonable understanding and application

O Clear understanding and application

> Criterion 5-Rate this text from 1 to 10. (With hindsight this could have more detail).

Typical feedback

'Overall this was an excellent portfolio as all sections were answered clearly and were direct to the point. You demonstrated a deep understanding of the problem statement, however it was too specific rather than having an 'outline' like you mentioned'.

A great advantage of this approach is that students receive very rapid feedback (usually within a few days of submission). A student's final mark is determined by the average of three assessments. The total mark for this task is 80% of the final average assessment plus a 20% component for assessing three other students. This 20% is determined by how well the assessor assessors a peer's submission compared to how well they assess three exemplar submissions previously assessed by the lecturer.

Student opinion on peer assessment

Even though 1400 students were enrolled, not all schools agreed to make use of peer assessment in this course. Of all the schools involved, only one school did not. This meant that approximately 1000 students undertook the peer review task. This was the first year it was run as a Moodle module. There were a number of technical issues as our installation of Moodle had problems handling 1000 student submissions. Technical issues always produce a bit of a negative response to an otherwise good idea. Overall I am happy how it went - I did, however, have to intervene and assess a few submissions myself. A very few students unfortunately did not take the task seriously. In addition, each school was responsible for introducing the task and presenting essentially the same introduction to the task. This did not necessarily happen as I had planned.

I received a number of responses to a brief questionnaire. Some of the more favourable responses are as follows:

(Q7) Please provide somefeedback on what you thought was good about the peer assessment exercise.*

It gave me a very good assessment of my own marking standards, allowing me to be more objective about the tasks requirements in the future.

(Q8) Please provide somefeedback on what you think we can do better next time.*

Clearer instructions; I thought that the problem statements i was reading for my exemplars were for a completely different task, and this led to confusion.

(Q10) I am really enjoying the group work component of this course.*

Agree

There were other similar comments regarding the lack of clear explanations.

Q7) Please provide somefeedback on what you thought was good about the peer assessment exercise.*

It allowed me to gain an idea of how others approached an objective in comparison to one another and myself.

(Q8) Please provide somefeedback on what you think we can do better next time.*

Explain the purpose of peer assessment clearly.

(Q10) I am really enjoying the group work component of this course.*

Agree

However, the group work is almost universally supported by students.

A further two responses are summarized in Figures 6 and 7.

 

 

 

 

Some more of the comments regarding the value of peer assessment need reviewing, but I feel that the response relates to a lack of information provided on this. The 'Agree' response is quite encouraging.

 

The final stage - peer review of student contribution

One of the major challenges with group work is in assigning individual grades from a group project. In general, not everyone contributes to the group project at the same level, or even in the same way. Trying to decide what marks to assign to individuals can be difficult - giving all the students the same mark is also not always fair. I believe that the group members are the best judges of this.

To counter the argument that 'group projects are not fair', as the course progresses to approximately halfway through the semester an exercise is undertaken to provide feedback on how group members are contributing to the task. The term 'peer review' is generally used to describe this task. The task is a formative task - it generally provides a very positive outcome for all involved, andcan be quite a morale booster. In this course I use a commercial site that promotes a specific peer review approach. SPARKPlus is a web-based self- and peer-review approach that enables students to confidentially rate their own and their peers' contributions to a team task or individual submissions (Willey and Freeman, 2006)

Students are required to rate their peers on a scale from NC (no contribution) to AA (above average) as shown in Table II.

 

 

Based on the following criteria, students use a slider scale from 0-100 to input their assessment based on the following criteria.

Rating criteria

> Efficient functioning of group - how does the team member rate in:

- Helping the group to function well as a team?

- Level of enthusiasm and participation?

> Contribution to design groups

- Did the team member attend and participate in team meetings and complete assigned tasks on schedule?

- Was the team member dependable and reliable in doing their share of the work?

- Was the team member effective and valuable in accomplishing tasks and assignments?

- Did the team member take initiative to seek out tasks and responsibilities?

- Did the team member facilitate the team process, provide valuable direction, and motivate others?

- Did the team member help to create a positive team experience and contribute to team morale?

Once the assessment has been completed by all team members, individuals receive a score called an SPA.

SPA = SQRT (Total ratings for individual member/Average of total ratings for all team members)

An SPA of 1.0 would indicate that the team member's contribution was rated as being equal to the average contribution of the team. A major divergence from 1.0 would indicate a need for further investigation to determine if there were issues within the group.

The main use of the SPA is as an assessment moderator for a group submission. For instance, with some pre-set conditions, an individual's mark = the team mark x the individual's SPA.

In addition, the student is asked to self-assess their contribution to the project. This second score or factor that is generated is termed an SAPA. It is calculated as:

SAPA = SQRT Self-assessment value/Average rating of all team members)

This is a powerful feedback option as it compares what the student 'thinks' their contribution is with their team members' views. Again, the ideal score is 1.0.

Once the task is completed by all students the results are released. Students are aware of the implications of the SPA and SAPA scores. Table III and Table IV indicate the results for a group of eight students. Results are not returned to students if less than four students complete the task. Table III indicates that Student 2 is performing extremely well although they may be somewhat reserved in the assessment of their own input. Table III also indicates that all students except the fourth student are contributing strongly to the task. An SPA of 0.83 is quite low and indicates that the student is not performing. However, the SAPA of 1.48 indicates that they consider they are contributing far and above what the other group members believe. Such a high score could also indicate the student has just selected 100% in all categories. This student would be interviewed if the SPA was to be used as a moderator.

 

 

 

 

The feedback given by team members may also help to explain the SPA of 0.83. The team is concerned regarding the lack of involvement in the project.

> Good team member, although a higher attendance rate to meetings would be appreciated.

> Has made some solid contributions to the project and the design. plenty of experience

> Whilst not present at meetings, he still provided valuable insight and experience to the team environment.

> Did not come to many of the meetings.

> Seldom attends to the group meeting due to personal reason and hardly hear any valuable information from him.

> Gave effective suggestions on the project.

In addition the last line of Table III indicates that the student has not undertaken the SPARK assessment. The score of 0.96 is a result of their peers' review only.

As previously mentioned, all students see their own scores and feedback. It is expected that students will consider all comments and modify their contribution a little where needed. It is very rare that I need to talk to the group member.

Table IV represents same process undertaken in week 12 after all assessments had been completed. In comparison to the results published after week 6, the indication is that the group contributions have improved, although only slightly in some cases. It is interesting to note that student 4 has improved greatly, with their SPA increasing from 0.83 to 1.0 and their SAPA dropping to a more acceptable level of 1.04. This illustrates how the peer review process is received by students. The comments below on the same student also show the change in effort that appears to have been made. All feedback is released unedited, and I believe is well accepted and obviously can be a great morale booster or an early 'wake-up call'.

> Good team member, great to work with.

> Great bloke to work with, plenty of innovative ideas and experience

> Provided a wealth of experience and advice for the team.

> Provided fantastic contributions to the group.

> Contributed a lot in building of the dragline.

> Good team member, enthusiastic and dedicated.

> Great team member always sparing much time for group given his tight schedule

 

Summary

The data in Table III and Table IV is taken from the 2013 offering of this course. The total cohort was 79 students in 10 groups. After week 12, 77 student SPA results were compared to those obtained after week 6. Two students had been removed as their scores were incomplete. Of the remaining 77, 24 students (31%) had a lower SPA in week 12 compared to week 6. However, the average decrease in SPA averaged only 0.05 points. I am not too concerned regarding this value as 43 (56%) students showed an increase in SPA, with 10 remaining the same. The average overall improvement in SPA was 0.035. Although it cannot be formally confirmed, I believe that the increase in student involvement is a result of the feedback gained at the end of week 6. Peer review is a powerful tool and can be a great morale builder rather than a punishment tool.

Not all courses require this process to be undertaken twice in a semester. However, in ENGG1000 it is run in week 6 as a formative exercise, and again in week 12. The SPA value from week 12 is used as a moderator of a student's mark. This generally works quite well. However, situations have arisen where a student receives a very low score, or very high score that could return a final mark of greater than 100%. Typically, in each course outline made available to students before commencement of the course a limit is advised. For example SPAs are limited to between 0.90 and 1.10 in the case of ENGG1000. If a score is outside these limits I would meet with the student to discuss the issues and determine if there were any extenuating circumstances. In cases where the SPA is very low I find that other individual assessments are often completed poorly and receiving similar SPAs from other group work. This is a good indicator that the student is not coping well with the workload, and the student is encouraged to discuss their progress with a staff member.

Peer review of group work contributions has been a major component of UNSW mining courses for a number of years. Overall, I feel that peer review overcomes some of the traditional issues with group work, particularly being able to determine the contribution of an individual to an assessment task. So often group members receive the group mark, which is not always appropriate. This often promotes discontent within the group and engenders a clear reluctance to undertake further group work assignments.

A clear advantage of introducing peer review and peer assessment in a year 1 course is that students become accustomed to using both approaches in later years, when they become more important.

 

Conclusions

This paper set out to describe initiatives trialled and present some of the challenges of continually developing and managing a course for 1400 students with a strong emphasis on group work and peer interaction, which is uncommon in a typical year 1 course. The main challenges included the process of working together as a team. Many, if not most, students would have had little experience with working in a team and hence the transition to this type of non-individual study can be quite confusing, challenging, and even confronting when students have to work together in a small team to produce an outcome. In addition, the members of the team are required to provide feedback to their team colleagues on the quality of assessable material submitted as well as on commitment to the process of completing it.

I have shown that, overall, the process is a success. Students gain a lot from the experience. However, there are issues that need to be addressed. We need to ensure that students understand the reasons for peer assessment, that they are comfortable with the process, know how to give feedback, are comfortable with assessing another students' submission, and know how to comment on the level of involvement of colleagues in the process. We need to work at showing students that group projects are fair and are a common approach across many courses in engineering.

 

Acknowledgements

I wish to acknowledge the great team of the nine school coordinators who contribute to the success of this course. I am also grateful for the support of David Clements, the Faculty Associate Dean Education, and John Paul Posada, the Faculty Education Technologist for his valuable Moodle Technical support. I would also like to acknowledge the support Mining Education Australia for continued access to SPARK Plus and to Keith Willey at the University of Technology, Sydney for his guidance and support in all aspects of SPARK Plus.

 

References

Beamish, B., Kizil, M., Willey, K., and Gardner, A. 2009. Monitoring mining engineering undergraduate perceptions of contribution to group project work. 20th Annual Conference for the Australasian Associationfor Engineering Education, Adelaide, 6-9 December 2009. Engineering the Curriculum. Barton, A.C.T. Engineers Australia. pp. 318-325.         [ Links ]

Dutson, A., Todd, R., Magleby, S., and Sorensen, C. 1997. Review of literature on teaching engineering design through project oriented capstone courses. Journal of Engineering Education, vol. 86, no. 1. pp. 17-25        [ Links ]

Gibbs, G. and Simpson, C. 2004. Conditions under which assessment supports student's learning. Learning and Teaching in Higher Education, no. 1. 2004-05.         [ Links ]

Hemer, D. 2008. Peer assessment of group-based software engineering projects. 19th Australian Conference on Software Engineering, Perth, Western Australia, 26-28 March 2008. pp. 470-478.         [ Links ]

Kennedy, J.K. 2005. Peer assessment in group projects: is it worth it?' Computing Education 2005. Proceedings of the Seventh Australasian Computing Education Conference (ACE2005), Newcastle, NSW. CRPIT, 42. Young, A. and Tolhurst, D. (eds.). Australian Computer Society, Sydney. pp. 59-65.         [ Links ]

McAlpine, I., Reidsema, C., and Allen, B. 2006. Educational design and online support for an innovative project based course in engineering design. Proceedings of the 23rd Annual Conference of the Australasian Society for Computers in Learning in Tertiary Education (ASCILITE), University of Sydney, 3-6 December 2006.         [ Links ]

McAlpine, I. and Reidsema, C. 2007. The role of student peer review and assessment in an introductory project based engineering design course. Connect-ed2007. International Conference on Design Education, UNSW, Sydney, July 2007.         [ Links ]

Mitra, R., Saydam, S., Daly, C., and Hagan, P. 2009. Enhanced student collaboration in mining engineering through peer review of major projects. International Journal of Learning, vol. 16, no. 11. pp. 501-519. http://ijb.cgpublisher.com/product/pub.30/prod.2494.         [ Links ]

Parker, K.R. and Chao, J.T. 2007. Wiki as a teaching tool, interdisciplinary. Journal of Knowledge and Learning Objects, vol. 3.         [ Links ]

Parsons, J.R. and Klukken, P.G. 2005. An introductory design and innovation course at the University of Tennessee. Frontiers in Education Conference, Atlanta, GA, November 2005. pp. 3a5, 13-15.         [ Links ]

Race, P. 2001. A Briefing on Self, Peer and Group Assessment. Assessment Series no. 9. LTSN Generic Centre, York, UK.         [ Links ]

Vos, H.J., Frits, P. van B., and Ten Bruggencate, G.C. 2000. Multidisciplinary design projects among both engineering and humanities studies. International Journal of Continuing Engineering Education and Lifelong Learning, vol. 10, no. 1-4. pp.314-326.         [ Links ]

Webb, A. and Willis, L. 2010. Enhancing feedback for engineering students. Higher Education Academy Engineering Subject Centre, Loughborough University, UK.         [ Links ]

White, F., Lloyd, H., Kennedy, G., and Stewart, C. 2004. Effective Management and Assessment of Group Work. Final Report: Teaching Improvement Fund 2003, Faculty of Science, University of Sydney.         [ Links ]

Willey, K. and Freeman, M. 2006. Improving teamwork and engagement: the case for self and peer assessment. Australasian Journal of Engineering Education, vol. 12. http://www.aaee.com.au/journal/2006/willey0106.pdf        [ Links ]

Willey, K. and Gardner, A. 2006. Improvements in the self and peer assessment tool SPARK: do they improve learning outcomes? ATN Assessment 08, UTS; Engaging students with assessment.         [ Links ]

Willey, K. and Gardner, A. 2008. Using self and peer assessment for professional and team skill development: do well functioning teams experience the benefits? ATN Assessment 08, UTS; Engaging students with assessment.         [ Links ]