GENERAL ARTICLES

 

Metacognitive learning: Skills development through gamification at the Stellenbosch learning factory as a case study

 

 

M. Henning; D. Hagedorn-Hansen^*; K.H. von Leipzig

Department of Industrial Engineering, Stellenbosch University, South Africa

 

 

---

ABSTRACT

South Africa has a high-cost, low-performance education system, which ultimately leads to unemployment and a skill shortage in the country. In order to bridge the skill shortage gap at a tertiary level, the Stellenbosch Learning Factory (SLF) was established. Learning factories involve experiential learning in a production environment through 'learning by doing'. Gamification, one of the teaching methods used in the SLF, is investigated as a possible answer to South Africa's educational problems. Learning factories can be used to train employees: the knowledge transfer resulting from real production conditions is favoured because process improvements can be implemented or practised without any real production downtime. The aim of this study was to determine the learning contribution of the games implemented at the SLF. This was accomplished by developing a three-d imensional matrix that employs a revised version of Bloom's taxonomy to measure the learning success of the educational games at the SLF.

---

OPSOMMING

Suid-Afrika het 'n hoë koste, lae uitkoms onderwysstelsel, wat bydra tot die werkloosheid en die tekort in vaardigheid in die land. Om die vaardigheidsgaping op 'n tersiêre vlak te oorbrug, is die 'Stellenbosch Learning Factory' (SLF) gestig. Leer fabrieke behels eksperimentele onderrig in 'n produksie omgewing deur gebruik te maak van 'n 'leer deur doen' benadering. 'Gamification', een van die onderrigmetodes wat gebruik word in die SLF, word ondersoek as 'n moontlike verbetering op die onderrig probleem in Suid-Afrika. Leer fabrieke kan gebruik word om werknemers op te lei, omdat die kennisoordrag wat ontstaan vanuit die werklike produksietoestande bevoordeel word, aangesien prosesverbeteringe implementeer en beoefen kan word sonder onderbreking van produksie. Die doel van hierdie studie was om die leerbydrae van die opvoedkundige speletjies wat geïmplementeer is by die SLF te bepaal. Dit is bereik deur die ontwikkeling van 'n drie-dimensionele matriks wat die hersiende weergawe van Bloom se taksonomie gebruik om die leersukses van die opvoedkundige speletjies by die SLF te bepaal.

---

 

 

1 INTRODUCTION

Education is fundamental to contemporary human development. South Africa has a high-cost, low-performance educational system that ultimately leads to a skill shortage in the country [1]-[3]. Dr Martin Prew [2], Director (2010) of the Centre for Education Policy Development (CEPD), has stated that "a total review of the current Schools Act is a priority". Business leaders also argue that the new generation of employees do not have enough practical experience when they step into the workplace, requiring a large amount of training to rectify this problem [4]. In order to rectify the skill shortage in South Africa, the government established the Skills Development Act (SDA), which was created to address this problem by establishing 23 Sectoral Education and Training Authorities (SETAs). SETAs focus on increasing access to highly relevant and quality training opportunities and skill development to set the path for substantive change [5]. Although the SETAs were stated to be on a positive trajectory in 2007 [6], the current education and training methods need to change with the times.

The current learning environment in South Africa can, in certain instances, be described as having an absence of dynamic and constructive interaction between learners and educators. The result of this environment is that factual learning is prominent, resulting in an atmosphere where excitement, emotional appeal, and personal engagement is sometimes absent or not always evident [7]. A recent concept, known as gamification, can contribute a solution to this problem. Gamification is defined by Lamprinou and Paraskeva [8] as "the use of game design elements in non-gaming contexts". Gamification promotes the use of games and elements of games in education to provide a learning environment to which the youth of today could possibly relate to. With gamification, a learning environment for teachers and students, either in the workplace or in the classroom, can be developed to challenge and build on their existing knowledge.

Learning factories can also use gamification in a production environment through 'learning by doing'; this is investigated as a possible solution to the educational problems and skill shortage in the country. Learning by doing has been shown to lead to greater application and retention possibilities than the traditional methods of education such as lectures [7], [9]-[11]. Due to the knowledge transfer resulting from real production conditions, favoured over a purely theoretical exposition, learning factories can also be used to train employees. As opposed to real production sites, the process improvements can be practised without cost pressure and in a lower risk environment [7],[9], [11].

With a greater demand for better forms of learning, it is predicted that the number of learning factories used in universities and companies could increase greatly in the future [6, 10]. According to Abele, Metternich and Tisch et al. [9], it is important that efficient and effective learning factory configurations are developed and identified systematically. To do this, it is fundamental that learning success is measured in a valid but simple way, and that the gap between formal education and the workplace is adequately bridged [13]. The aim of this study was to determine whether the learning contribution of the games at the Stellenbosch Learning Factory (SLF), situated at the Industrial Engineering department, is significant and measurable.

 

2 METHODOLOGY OF ANALYSING GAMES

To evaluate learning success, the study used the taxonomy of educational objectives, also known as 'Bloom's taxonomy', as the point of departure. The self-determination theory (SDT) and human needs hierarchy theories were also investigated to quantify a key aspect of gamification and learning factories: the fun element or, in classic terms, learning motivation. Based on these three theories, a multi-axis, multilevel approach to quantifying the learning success of the different games used at the SLF was developed in Rstudio. Emotional immersion was plotted on the z-axis, abstraction on the y-axis, and knowledge on the x-axis, to measure learning success. The results were determined by questionnaires that consisted of questions based on the learning theories. Using the questionnaires, the knowledge that the students had gained through participating was determined; their emotional immersion was determined by whether the participants were intrinsically or extrinsically motivated to participate; and the preference for gamification was determined by whether the students were in favour of the gamification and learning factory approach over the conventional lecturing approach. The abstraction dimension rankings were not determined by the questionnaires, but were based on an in-depth understanding and analysis of the games.

The learning success of five games - the Beer Game, the LEGO Car Game, the Train Game, the Off-Roader LEGO Car Game, and The Fresh Connection (TFC) simulation game - was determined through the questionnaires with 195, 78, 12, 11, and 72 participants respectively. The dimensions of the resultant matrix consisted of four levels ('1' being the worst and '4' being the greatest learning success). Each level is based on the need theories and on Bloom's taxonomy, and is specified in Table 1. The criteria of the knowledge and abstraction dimensions are a summary of [14]-[18], which focus on the revised Bloom's taxonomy. The criteria of the emotional immersion dimension summary are based on the self-determination theory (SDT) [8], [19]-[21 ] and the two need theories - Maslow's need hierarchy and Alderfer's existence, relatedness, and growth (ERG) theory [22]. The revised version of Bloom's taxonomy suggested that certain words be replaced and the top three levels flattened, as illustrated in Error! Reference source not found..

 

Figure 1

 

The revised Bloom's taxonomy (RBT) also re-conceptualised the single dimension into two dimensions (knowledge and cognitive), illustrated in Figure 2. The RBT views the six stages as a 'cognitive processing' dimension rather than as the cumulative hierarchy envisaged in the original taxonomy. For example, 'applying' can be achieved without the need to master the 'remembering' dimension.

 

 

The SDT can be divided into two human motivation groups: intrinsic motivation and extrinsic motivation. Intrinsic motivation is when players perform a certain action for their personal development and not to receive something [21]. Extrinsic motivation means that the player performs a certain action because (s)he is told to do it, through pressure, or it can lead to an external outcome or 'extrinsic reward'. The three core characteristics of intrinsic motivation are competence, autonomy, and relatedness. Competence creates confidence, as it makes players feel that they can achieve something and that they earn their achievement [20]. Autonomy is a feeling of being able to direct one's own actions; people thus need to experience their actions as self-determined rather than as controlled through an outside source [19]. Relatedness makes the player feel connected or related to others [21].

Maslow's need hierarchy theory proposed that motivation is a function of physiology, safety, belonging, esteem, and self-actualisation. Alderfer's ERG (existence, relatedness, and growth) theory reduced the dimensions to three, with each letter of ERG representing a type of need. The existence need (E) is comparable to Maslow's physiological and safety needs. Relatedness (R) is similar to Maslow's external esteem and social needs. Alderfer's growth need (G) covers Maslow's self-actualisation and self-esteem needs. Alderfer's ERG needs can be summarised as follows: The existence need is concerned with basic material-existence requirements, and includes various forms of physiological, safety, and material needs. Relatedness is a need to maintain significant relations, and includes a sense of belonging, respect, interaction, and security. The final need, growth, is concerned with the development of potential, and represents an intrinsic need for personal development. It also includes supporting the growth of others and the ability to realise one's potential [22].

Table 1 is a summary of the different levels used to evaluate the questionnaires. Each level is based on the need theories and Bloom's taxonomy, summarised in the previous sections.

 

3 RESULTS

The results from the questionnaires for the Beer Game, LEGO Car Game, Train Game, Off-Roader LEGO Car Game, and The Fresh Connection (TFC) simulation game were analysed, and are presented in graphical format in Figures 3 to 5. The overall response percentages of each game, based on the results of the distributed questionnaires, are plotted on the y-axis against the different levels discussed in Table 1. The results for the overall knowledge can be seen in Figure 3.

The knowledge results in Figure 3 indicate that the students have learned certain terms or theories through the experience, as they mostly selected the level 3 and 4 options. The results also indicate that the gamification approach increased their understanding of the terms due to experiencing the impact these theories or terms had on the game. The overall emotional immersion results are depicted in Figure 4.

Overall the students were intrinsically motivated to learn and participate, as the emotional immersion results from Figure 4 indicate that the greater majority selected the level 3 and 4 options. For the majority of the games, the students agreed with most of the questions' level 3 response - in-between. This indicates that the students were very motivated to participate in the experience and wanted to learn from it, but they were also driven to beat the other groups that were participating. Although they were driven to win, the students also felt part of a group and responsible for their own choices, which again satisfies intrinsic motivation. The overall preference for gamification results can be observed in Figure 5.

The overall preference for gamification results indicates that the students are in favour of this new approach to learning, and prefer it over a one- to two-hour lecture. For all of the games, almost 100 per cent of the students selected the level 3 and 4 options, indicating that they were in favour of the experience and would like to do other parts of their course using a similar gamified approach. This could be interpreted as a positive level for emotional immersion.

The overall rankings for the emotional immersion dimensions were based on the emotional immersion and the overall preference for gamification questionnaire results. By analysing the overall results it is clear that the level 3 and 4 responses were mostly selected for the different questions. The Beer game was also tested through two iterations with two different sample groups. Although it is the same game, the overall results from the three figures differ for the two different sample groups. Thus, the questionnaire results could also have been influenced by the players' study interests, and not only by the game.

The next section will illustrate and discuss the three-dimensional matrix. The section will also discuss the main factors that influenced the results of the three dimensions and the reasons behind the outcomes for the most and least successful games.

 

4 OVERALL LEARNING SUCCESS MATRIX

The three-dimensional matrix (Error! Reference source not found.) illustrates the relationship of the knowledge, abstraction, and emotional immersion between the games. The SLF games were ranked according to the analyses of the questionnaire results and criteria discussed in the previous sections. The results of the emotional immersion dimensions were influenced by the number of rounds that were played, and the period of time for which the students had to participate. The games that were stretched out over a longer period than a day therefore had greater emotional immersion results. The knowledge dimension's rankings were also influenced by the number of rounds that were played, as through each round the influence of the improvements implemented could be experienced. The ranking was also influenced by the duration period that the students participated, as a shorter period could lead to a high exposure of information or information overload. The abstraction dimension's rankings were influenced by whether the students had to 'create' the improvements, or whether the students implemented improvements based on the lecturer's instructions. The games where the participants had to 'create' their own improvements to implement had a higher abstraction ranking (closer to level 4).

Through the matrix in Error! Reference source not found., each game's learning success could be evaluated against the other games, and overall, the most successful game can be identified by its position relative to the different axes.

 

Figure 6

 

From Error! Reference source not found. it is clear that, overall, the TFC simulation game has the highest learning success, as it is ranked the highest according to the criteria of the three dimensions. This is because the game requires greater autonomy and in-depth engagement over a longer period of time. This is where the most learning takes place. In another study, TFC was also found to be a cost-effective way to optimise team performance and improve business results while promoting a fun learning environment [23].

The results indicated that, for games with multiple rounds and terms being presented, the learning success is more successful when played over a longer period (when possible) than a day, as this can cause information overload and the students' motivation decreases. The matrix also clearly indicates that the Off-Roader LEGO Car and Beer games' rankings in the three dimensions are the lowest of all the games. Due to participating over a shorter period of time, less in-depth engagement takes place; thus less learning takes place. This is a relative observation, as both of these games can still be considered successful learning experiences.

For future game development at the SLF, the three-dimensional matrix could be used to measure the learning success of the games. The tool can be used to test a game's learning success before finalising the development of a particular game, or to improve the learning success of existing games.

 

5 CONCLUSION

In conclusion, one can state that there is a definite need for alternative educational approaches in South Africa. The education system has not kept up with the people's need to satisfy their educational curiosity and intellectual proclivities to motivate them to learn. Gamification and learning factories both contribute to more effective teaching methods, as the students can participate in interactive learning, which motivates them to learn and to participate actively. The results indicated that the majority of the students preferred the gamification approach to conventional lecturing methods. Therefore, the gap between theoretical education and practical professional work can be bridged by learning factories, as the learning environment is based on reallife situations and on applying theory in practice. This is achieved through gamification, as it makes use of Bloom's taxonomy to transfer learning material by practical implementation.

The goal to determine whether the learning contribution of the games at the Stellenbosch Learning Factory (SLF) is significant and measurable was achieved. A three-dimensional matrix with different criteria has been developed as a tool. The questionnaire results contributed to determining the learning success ranking of the SLF games. The overall ranking of the games satisfies each dimension's level 3 and 4 criteria, indicating that a high level of learning success is achieved through each game that was investigated. This showed that the learning contribution from implementing gamification in learning factories is significant. The results also clearly indicated that the students are in favour of this approach, and that they do not view the game-like approach as unnecessary.

 

REFERENCES

[1] Dias, R. & Posel, D. 2007. Unemployment, education and skills constraints in post-apartheid South Africa. DPRU Working Paper No. 07/120. [Online]. Available:SSRN: https://ssrn.com/abstract=982046 or http://dx.doi.org/10.2139/ssrn.982046 [Accessed: 19-Mar-2017].         [ Links ]

[2] Department of Education. 2009. Challenges facing education in South Africa. [Online]. Available: http://www.cepd.org.za/files/pictures/The%20Challenges%20Facing %20Education% 20Interview%20Nov%2009.pdf [Accessed: 18-Mar-2017].         [ Links ]

[3] Daniels, R.C. 2007. Skills shortages in South Africa: A literature review. University of Cape Town.         [ Links ]

[4] Jaschik, S. 2015. Well-prepared in their own eyes. [Online]. Available: https://www.insidehighered.com/news/2015/01/20/study-finds-big-gaps-between-student-andemployer-perceptions [Accessed: 17-Nov-2016].         [ Links ]

[5] 'merSETA'. [Online]. Available: http://www.merseta.org.za/AboutUs.aspx [Accessed: 10-Mar-2017].         [ Links ]

[6] Singizi CC. 2007. Employment promotion programme: SETA Review, Department of Labour.         [ Links ]

[7] Kreimeier, D., Morlock, F., Prinz, C., Krückhans, B. & Bakir, D.C. 2015. Holistic learning factories: A concept to train lean management, resource efficiency as well as management and organization improvement skills. Procedía CIRP, 17, pp. 184-188.         [ Links ]

[8] Lamprinou, D. & Paraskeva, F. 2015. Gamification design framework based on SDT for student motivation. Proc. 2015 Int. Conf. Interact. Mob. Commun. Technol. Learn. IMCL, pp. 406-410.         [ Links ]

[9] Abele, E., Metternich, J., Tisch, M., Chryssolouris, G., Sihn, W., El Maraghy, H., Hummel, V., & Ranz, F. 2015. Learning factories for research, education, and training, Procedia CIRP, 32(Clf), pp. 1 -6.         [ Links ]

[10] Reiners, T. & Wood, L. 2013. Immersive virtual environments to facilitate authentic education in logistics and supply chain management, in Learning management systems and instructional design: Best practices in online education, Y. Kats, ed. Hershey: IGI Global, pp. 323-343.         [ Links ]

[11] Jan Cachay, R.T., Wennemer, J. & Abele, E. 2012. Study on action-oriented learning with a learning factory approach. Int. Conf. New Horizons Educ. INTE2012, 55, pp. 1144-1153.         [ Links ]

[12] Abele, E., Tenberg, R., Wenemer, J. & Cachay, J. 2010. Production skills development in learning factories, J. Econ. Fact. Oper., 105, pp. 909-913.         [ Links ]

[13] Eraut, M. 2009. Transfer of knowledge between education and workplace settings, in Knowledge, values and educational policy: A critical perspective, 65, pp. 1 -17.         [ Links ]

[14] Hattinger, M. & Treurnicht, N. 2011. Learning aspects for manufacturing enterprises within aerospace industry entering into quality Standard AS9100, 24th ICDE World Conf., pp. 1-11.         [ Links ]

[15] Healey, M. & Jerkins, A. 2000. Kolb's experiential learning theory and its application in geography in higher education, J. Chem. Inf. Model., 99(5), pp. 185-195.         [ Links ]

[16] Krathwohl, D.R. 2002. A revision of Bloom's taxonomy: An overview, Theory Pract., 41(4), pp. 212-218.         [ Links ]

[17] Mayer, R.E. 2002. Rote versus meaningful learning, Theory Pract., 41(4), pp. 226-231.         [ Links ]

[18] Pickard, M.J. 2007. The new Bloom's taxonomy: An overview for family and consumer sciences. J. Fam. Consum. Sci. Educ., 25(1), pp. 45-55.         [ Links ]

[19] Mekler, E.D., Brühlmann, F., Tuch, A.N. & Opwis, K. 2015. Towards understanding the effects of individual gamification elements on intrinsic motivation and performance. Comput. Human Behav., 71, pp. 1 - 10.         [ Links ]

[20] Kapp, K.M., Blair, L. & Mesch, R. 2014. The gamification of learning and instruction fieldbook. San Francisco: Wiley.         [ Links ]

[21] Van den Berg, C. 2014. Gamify a contact center, MSc Business studies, University of Amsterdam.         [ Links ]

[22] Sinding, K., Waldstrcpm, C., Krietner, R. & Kinicki, A. 2014. Organisational behaviour, 5th ed. UK: McGraw-Hill Education.         [ Links ]

[23] Cotter, J., Forster, G. & Sweeney, E. 2009. Supply chain learning: The role of games. J. Natl. Inst. Transp. Logist., 10(3), pp. 32-36.         [ Links ]

 

 

* Corresponding author devonh@sun.ac.za