PAPERS OF GENERAL INTEREST

 

An evaluation framework for virtual reality safety training systems in the South African mining industry

 

 

E.A. van Wyk^I; M.R. de Villiers^II

^IFaculty of ICT, Tshwane University of Technology, South Africa
^IISchool of Computing, University of South Africa

Correspondence

 

 

---

SYNOPSIS

Mining companies strive to increase and maintain production, while simultaneously remaining competitive in the global economy. Furthermore, they must ensure workers' safety and maintain good safety records. The use of virtual reality (VR) facilitates the development of tools and systems for various purposes that can improve knowledge and understanding of the work environment. VR is a rapidly growing technology that uses the ever-increasing power of computing to simulate real-world and imaginary environments and situations with a high degree of realism and interaction. The availability of 3D modelling tools and simulation programming engines that work effectively with a mid-range desktop PC and standard 3D graphics card, make VR technology viable and attractive for mainstream training applications. The design, development, and implementation of interactive VR training systems is proposed as an innovative approach to augment safety training. However, in order to assess the impact of such VR training systems it is of particular importance to determine the effectiveness of the design of such systems. This article proposes an evaluation framework for this vital purpose. This framework comprises criteria to assess VR training systems, specifically relating to usability, instructional design, VR systems design, and mining industry context-specific aspects. Although the framework was developed as an evaluation tool to assess effectiveness of the design of such systems, it can equally well be used as a set of design principles to inform the design of VR training systems for mining.

Keywords: virtual reality, mine safety training, design-based research, evaluation framework.

---

 

 

Introduction

Mining in South Africa has been the main driving force behind the history and development of Africa's richest and most advanced economy (Coka, 2012). The South African mining industry is frequently criticised for its poor safety record and high fatality rate (Stoddard and Skweyiya, 2016). Inadequate or insufficient training is often cited as a root cause of accidents (Van Wyk and De Villiers, 2009; Tichon and Burgess-Limerick, 2011). Virtual reality (VR) is a rapidly growing technology that utilizes the ever-increasing power of computing to simulate real-world and imaginary environments and situations with a high degree of realism and interaction (Webber-Youngman and Van Wyk, 2013). VR is currently being used or investigated as a training solution in a variety of industries. VR technology has developed rapidly and costs have fallen to levels where it can now be considered for mainstream training applications. The availability of 3D modelling tools and simulation programming engines that work effectively with a mid-range desktop PC and a standard 3D graphics card, make VR even more attractive to mine training centres (Van Wyk and De Villiers, 2009).

The design, development, and implementation of interactive VR training systems is proposed as an innovative approach to augment safety training. However, in order to assess the impact of such VR training systems it is of particular importance to determine the effectiveness of the design of such systems. An approach is therefore required that evaluates the appropriateness and effectiveness of the VR systems design within the context of mine safety training. This article proposes an evaluation framework for this vital purpose.

Current evaluation frameworks are limited because they are either confined to evaluation of a specific type of virtual environment or they focus on a particular aspect of such environments. Hence this article reports on four cycles within a design-based research process, which led to the implementation of two interactive desktop VR safety training systems and the generation of an evaluation framework for evaluating such systems. Due to the increase of VR training in the mining sector and other industrial domains, the evaluation framework makes an important contribution in evaluating the effectiveness of training systems. Moreover, the framework can be applied not only to evaluate such systems, but its criteria can also serve as design aids.

 

Background

Current mining training relies mainly on repetitive classroom-style learning with some instruction situated in a physical mock-up of an underground workplace, followed by on-the-job training. But under classroom conditions, workers do not make safety decisions under the same situations of stress and pressure they experience underground. Depending on the stress levels of the real working situation, the decisions taken in the authentic underground workplace may differ significantly from those taken under more relaxed circumstances in the classroom. To enhance the effectiveness of training, a training design is required that simulates threats as closely as possible (Van Wyk, 2015).

VR-based training tools can provide realistic simulations of such threats. Using VR, trainees can experience simulated working conditions in a virtual environment, yet without being subjected to the risks associated with the real environment. VR is currently being used and investigated in training solutions for a variety of fields, such as the military, medical, power generation, and aircraft industries (Van Wyk and Prinsloo, 2015). VR has features that are well suited to training for the mining environment and, in particular, for hazard recognition and associated remedial safety action. The most relevant features are: the facility to expose trainees to simulated hazardous situations without putting them in actual danger; the ability to present simulated hazardous situations more frequently than they are encountered in the real world; the simulation of situations that have not previously occurred, but which could be encountered in the workplace; and showing the possible consequences of actions taken by trainees.

 

Virtual reality

VR technology is evolving rapidly, and it would be risky to define VR in terms of specific devices that might fall out of favour in the short term (La Valle, 2017). In general, VR refers to a technology where a user interacts with a three-dimensional computer-simulated environment that is perceived as comparable to real-world objects and events. As computer hardware and software improve and technology is frequently updated, the ease with which interactive simulations can be developed and deployed has improved considerably and lower-cost, high-quality development tools have become available. Based on the level of immersion, one can distinguish between three categories of VR system.

Immersive VR systems

Immersive systems are the most technically advanced. The user is essentially isolated from the outside world and fully enveloped within a computer-generated environment. Immersive VR systems require users to wear a head-mounted display (HMD) unit that presents an image directly in front of each eye and magnifies it to fill a wide field of view. This creates the impression of actually being inside an environment, rather than observing a screen (Daden, 2016). The view is based on computer-generated images that react to the position and orientation of the user's head. HMD technology has certain disadvantages, including encumbrance due to wearing a headset with cables, a sense of isolation when viewing the virtual world while not knowing what is happening around you in the real world, high cost, and occasional simulator sickness and disorientation (Rogers, Sharp, and Preece, 2011; Stone and Knight, 2012). Various approaches to HMDs are emerging:

► The integrated approach uses a headset with all displays and functionality built in, while a computer generates the graphics. Examples are Oculus Rift, HTC Vive, and Sony Morpheus.

► The smartphone approach uses a smartphone as the display, processor. and movement detector. The phone resides within a holder with lenses, such as the Google Cardboard or Samsung Gear.

►Google is currently developing a standalone VR headset, called Daydream, that will not require a separate computer, smartphone, or cables (Google, 2017).

Semi-immersive VR systems

In semi-immersive VR systems, computer-generated images are displayed on large screens by stereo projection and are viewed via special stereo eyewear. Interaction with onscreen menus occurs via a remote keypad, while other input is handled by devices such as 3D controllers or joysticks.

The use of multiple projection-based systems can result in substantial cost, which can run in the order of millions of rands, but high-resolution images can be produced. The configuration in which the user is surrounded by projection screens is sometimes referred to as a CAVE (cave automatic virtual environment). A CAVE creates the illusion of immersion by projecting high-resolution stereo images on the walls and floor of a room-sized cube. Several persons wearing lightweight stereo glasses can enter and walk freely inside the CAVE (Van Wyk, 2015). Although their fixed display position and limited display area restrict the user's range of interactions, projected VR systems have the advantage that the user is not constrained within a headset and can communicate freely with other participants.

Desktop VR systems

Since not all applications require immersion to the extent described in the above categories, more affordable, non-immersive VR systems provide practical alternatives. An important feature of VR is the provision of a sense of actual presence in the simulated environment. Presence refers to the subjective experience of 'being' in the computer-generated environment, rather than in the actual real-world environment. In desktop VR systems the user remains visually aware of the real world, but can also observe the virtual world on a high-resolution monitor. In comparison to immersive systems, desktop VR provides a lower level of presence, but the lower capital cost of hardware, software, and peripherals make desktop VR systems an attractive and realistic substitute. Desktop systems utilize standard computer hardware. Input devices include a keyboard, mouse, 3D controller, joystick, and track pad or voice to interact with and manipulate the virtual environment. The sense of subjective immersion in desktop VR systems can be enhanced through stereoscopic glasses, which provide three-dimensional depth.

The boundaries between these types of VR systems are not clear-cut. The creative use of display and audio peripherals in desktop or semi-immersive systems can promote a sense of presence as experienced in immersive systems, even without the ability to fully control the virtual environment.

 

Virtual reality and mine training

Mining in the 21st century is a high-technology industry. Mining companies strive to increase and maintain production, while simultaneously remaining competitive within the global economy. Importantly, they should ensure workers' safety and maintain a good safety record, aiming towards zero harm. VR offers opportunities to develop tools and systems for a variety of purposes that can improve knowledge and understanding of the work environment. VR systems for the mining industry have been developed for mine planning and design, use of equipment, and training applications.

A primary objective of developing VR systems for mining is to allow personnel to practise and experience situations, activities, and processes that are encountered in the day-to-day operations at a mining site. Using VR, customized simulations of mine layouts and comprehensive virtual environments can be set up, allowing users to move around the virtual mine and take decisions. The consequences of both correct and incorrect decisions can be immediately fed back to trainees, giving them the opportunity to learn from their mistakes. VR also allows trainees to experience conditions that would be difficult or impossible to re-create in the real world. VR simulations can provide a wide range of possible training scenarios without incurring the high costs and risks to personnel and equipment (Van Wyk and Prinsloo, 2015). Moreover, VR is ideal for training workers who perform tasks in dangerous or hazardous environments. Trainees can practise procedures in a risk-free virtual situation, while being exposed to 'life-threatening' scenarios in a safe and controlled situation (Webber-Youngman, 2014).

 

Literature review

Different types of VR training systems have already been developed for the mining industry. and will be discussed in detail in the sections to follow.

Incident reconstruction simulations

Mining incidents are unfortunately a regular occurrence, due to the inherently hazardous nature of mining. With the aim of preventing recurrences, VR can be used to simulate the circumstances relating to previous serious incidents. Such incident reconstruction simulations help to emphasise the significance of unsafe acts and to promote a strong safety culture. They enable the workers to understand how and why an incident happened, how it could have been prevented, and how injuries or fatalities could have been avoided. Simulating a range of incident scenarios on a computer screen and viewing them from multiple angles enables investigators and workers to understand the underlying causes of an incident (STS3D, 2017). Continuous employee education and training, with the aim of establishing a strong safety culture in the industry, is a major factor in preventing fatal and non-fatal mining incidents (Schafrik, Karmis, and Agioutantis, 2003). VR training can play a meaningful role in the transfer of this knowledge (Webber-Youngman and Van Wyk, 2011).

Hazard awareness

As previously stated, VR applications for training in hazard awareness enhance traditional training methods without unnecessarily exposing trainees to actual hazardous situations. Various systems have been developed that focus on simulating hazards relating to specific machinery or particular situations in the workplace, e.g. conveyor belts (Lucas, Thabet, and Worlikar, 2007) and trackless mobile machines (STS, 2013). Orr, Mallet, and Margolis (2009) developed a VR system for enhanced fire escape training for mine workers. VR technology has also been used to simulate hazards relating to the underground ventilation system (Ji-zu et al., 2009).

Training related to mining equipment

Many global mining equipment manufacturers have developed simulators of their more advanced equipment. Sandvik and Atlas Copco have drilling simulators, while Bucyrus International introduced simulation training for electric mining shovels (Chadwick, 2009). Caterpillar and Volvo supply simulators of their heavy machinery, including wheel loaders and excavators. These simulators feature state-of-the-art software with advanced 3D graphics to reproduce the operational movements of the real machines (CAT, 2014; Oryx, 2014).

Evaluation of VR training systems

Tsiatsos, Andreas, and Pomportsis (2010:67) point out the 'need/or a detailed theoretical framework/or VR-based learning environments that couldguidejuture development'. They propose a framework for a specific category of collaborative virtual environments, which entail group work in a single immersive environment. This evaluation approach is not appropriate to the present study, as it relates to individualized desktop VR training.

Hanna, Nader, and Richards (2014) propose an evaluation framework for VR, but this similarly focuses on collaborative virtual environments, which are outside the scope of this study. Earlier frameworks include the work by Bowman, Koller, and Hodges (1998), who presented a framework for the analysis and evaluation of travel techniques in immersive virtual environments. This system, however, is limited to viewpoint motion control techniques. Bowman, Gabbard, and Hix (2002) overviewed usability evaluation of virtual environments, but focused on issues that differentiate evaluation of virtual environments from evaluation of traditional user interfaces. Gang, Jun, and Yingzhen (2006) proposed an evaluation framework for evaluating virtual geographic environments, but this framework evaluates only three aspects, namely the reality portrayed, immersion, and usability of such environments.

To summarize, existing evaluation frameworks are limited. They are either designed to evaluate a specific type of virtual environment or they focus on a restricted aspect of such environments. This research addresses the gap in tools for evaluating desktop VR training systems for the mining industry. This is done through a framework that investigates the design and development of such systems meticulously and comprehensively from the perspectives of instructional design, usability, and VR systems design, situated in the context of mining. These varying perspectives are integrated into a single framework, providing a multi-faceted evaluation approach.

 

Research methodology

The underlying research paradigm of this study is design research, which is currently a maturing research methodology within a number of disciplines. Two streams exist within design research:

► Design science research (DSR) in the discipline of information systems

► Design-based research (DBR) in the domain of educational technology.

DBR was selected for this study because of its cyclic nature of design, evaluation, and redesign, and its mandatory production of both theory and actual solutions in real-life contexts, in this case, the context of VR training systems for the mining industry.

The iterative research process is depicted in the DBR model shown in Figure 1. Each cycle comprises various steps (Van Wyk and De Villiers, 2016):

► Analyse the problem in context: An authentic problem is identified in a complex environment. Literature is reviewed to examine the problem and identify appropriate theory. Researchers and practitioners collaborate in analysing the problem and establishing research goals.

► Design the solution: Influenced by contextual limitations and the complexity of real-world interactions and settings, an initial design is proposed to address the problem.

►Develop the solution: A prototype is developed that serves the research purpose. Development is informed by existing design principles and technological innovations.

► Evaluate in practice: The artefact is tested in an authentic setting. Data is collected and analysed to answer the research questions and to construct principles or theory.

► Reflection, leading to dual outcomes:

• Practical real-world contribution - As reflection occurs upon the data, the solution is enhanced. New designs can be developed, implemented, and improved, leading to an ongoing subcycle of design-reflection-design.

• Theoretical contribution - Design principles should be continuously and cumulatively documented in order to be transferable and utilized by others in similar settings.

Figure 1 shows the process flow from problem to solution, where each blue oval represents an occurrence of the DBR cycle. The red blocks indicate actions, and the green blocks indicate the artefacts or theory deliverables that are the outcomes of the DBR process.

 

Development and results

The first steps in the design research process were the development and trial-in-practice of two VR safety training systems. They required theoretical evaluation and refinement. This confirmed the lack of a suitable tool for evaluating VR for safety training systems, therefore a framework to address this gap was synthesised from the literature. The two VR training systems were then evaluated using the evaluation framework. This process served the purpose of identifying their inadequacies and also tested the evaluation framework itself in a design research process. After this practical application, the evaluation framework was formally and finally evaluated in a meta-evaluation. The next subsections describe the iterative evaluate-and-improve DBR process.

DBR cycle 1

The research process commenced by defining the real-world problem. More than 100 workers die annually in the South African mining industry and thousands are injured (Webber-Youngman and Van Wyk, 2013). An aim of this study was to propose, model, prototype, and evaluate two novel electronic training interventions to improve the safety of mineworkers. These e-learning (also termed e-training) systems were implemented at a South African mine, using VR technology to simulate underground conditions and potential hazards, and to supplement conventional classroom training.

First, a desktop VR training prototype, Look, Stop and Fix (LSF), was designed and developed based on the problem analysis and literature reviews. This prototype simulated the underground working areas, incorporating potential hazards. Mineworkers had to spot the hazards, identify them correctly, and indicate appropriate actions to be followed in response to each hazard. Incorrect identification of a hazard or failure to indicate the correct action to deal with it prompted an animation. This showed the possible consequences of ignoring it or responding incorrectly. The LSF prototype was evaluated in practice by user surveys at a large platinum mine, which resulted in several design problems that needed attention being identified.

Evaluation framework and DBR cycle 2

Following cycle 1, a formal evaluation framework called the Desktop Virtual Reality Evaluation Framework (DEVREF) and a set of criteria were developed for evaluating desktop VR training applications. This was done by means of heuristic evaluation by experts. During cycle 2, the design of LSF was refined and further evaluated using DEVREF. This cycle included several internal DBR subcycles to determine inadequacies and problems in LSF and led to an improved version of LSF.

Case study and DBR cycle 3

Upon reflection, the mine indicated that the LSF prototype, which focuses on generic hazards, should be expanded to focus on major causes of incidents. A case study was done to identify such causes. This culminated in DBR cycle 3 in the design and development of a new geological conditions prototype called Interactive Simulated Geological Conditions (ISGC). In the ISGC prototype, trainees had to identify 21 different geological conditions occurring in that particular mine. They also had to specify the associated risks and controls for each condition. ISGC animations show possible consequences of ignoring or incorrectly addressing the geological conditions. DEVREF was then used to evaluate ISGC and problematic issues were identified and addressed:

► Some trainees struggled with orientation in the virtual environment and it was decided to add orientation labels.

► Additional visual learning material was required to better explain some of the geological conditions.

► Scenes containing combinations of more than one geological hazard, as is practically experienced underground, were added to the system.

Improvement of evaluation framework and DBR cycle 4

In the process of evaluating the LSF and ISGC prototypes, inadequacies emerged in the DEVREF evaluation framework itself. To fix these, modifications were made to some criteria and additional evaluation statements were added to the framework to address important aspects not fully covered. After improving the prototypes, a further DBR cycle (cycle 4) was applied to improve DEVREF itself, so as to strengthen future evaluations. To evaluate the appropriateness and effectiveness of the evaluation framework, a meta-evaluation was undertaken. This process is discussed in the section Evaluating the evaluation framework. The findings of this process were used to enhance DEVREF and its functionality.

Outcomes of the DBR process

Figure 1 indicates the outcomes resulting from the processes. In line with design-based research, there are dual outcomes involving a practical real-world solution in the form of novel desktop VR training systems and a theoretical contribution which is the evaluation framework, DEVREF, for evaluating desktop VR training systems.

DEVREF - a synthesised framework for heuristic evaluation of VR training applications

As stated above, the DBR process had dual outcomes:

the production of real-world innovative products and the development of theory. The practical outcome - the desktop VR training systems, LSF for hazards recognition and ISGC for geological conditions, evolved from prototypes to products, becoming real-world training systems currently in use at various mines (Van Wyk and De Villiers, 2014). The focus of this article, however, is the theoretical contribution made by the new evaluation framework, DEVREF.

Various factors determine the theoretical foundations for investigating e-learning applications. No single paradigm is appropriate for all situations, since domain, context, and content must all be considered (De Villiers, 2005). To find an appropriate approach, technological issues, educational theories, and usability should be taken into account. Costabile et al. (2005) advise that evaluation of educational systems should investigate pedagogical effectiveness and usability aspects. When integrating usability into learning, usability features that support the achievement of educational goals should be addressed (Rogers, Sharp, and Preece, 2011).

Ardito et al., (2006), advocate that specific custom-designed guidelines should be provided for the evaluation of e-learning, rather than using sets of general criteria. In response, this study synthesises a new set of guidelines and a single, integrated evaluation framework, specifically customized to evaluate interactive desktop VR training systems for safety in the mining industry. Therefore, the design and development of such systems are investigated meticulously and comprehensively from four different perspectives, integrated into a single framework, namely DEVREF, as presented in Table I. DEVREF provides a multi-faceted approach, with four categories of heuristics (performance criteria) extracted from an extensive in-depth study of the literature:

• Category 1: Instructional design - heuristics related to pedagogical effectiveness, learning theories, and multimedia learning design.

• Category 2: General usability - interface design, navigation and interaction, and heuristics that support the goals of usability.

• Category 3: Virtual reality system design - heuristics specific to the design of VR systems.

• Category 4: Mining-specific heuristics - heuristics relating to the context and content of the application domain.

 

Evaluating the evaluation framework

The findings of the prototype evaluations indicated some inadequacies in DEVREF and a meta-evaluation was done to strengthen it. No standard meta-evaluation checklist was appropriate for evaluating DEVREF, due to its innovative and extensive nature. A custom-built instrument was therefore developed to evaluate both (i) the criteria that comprise the DEVREF evaluation framework and (ii) the methodology employed in the evaluation. The meta-evaluation was a systematic review conducted by six experts in various aspects, four of them being double experts who had expertise in more than one of the DEVREF categories. Each category was evaluated separately.

The meta-evaluation instrument was designed in the context of evaluating desktop VR training systems. The challenge was determining appropriate criteria for evaluating the merit of evaluation frameworks themselves. First, criteria were selected from the literature by acknowledged experts within DEVREF's four categories. The selected criteria were consolidated into a set of evaluation statements, which resulted in the first part of the meta-evaluation questionnaire used to evaluate the DEVREF evaluation criteria. The second part was used to evaluate the evaluation method employed, in this case heuristic evaluation. This questionnaire was used as the instrument in the meta-evaluation, and the participating expert evaluators were the same evaluators who had conducted the heuristic evaluations of the prototype VR training programs.

The meta-evaluation was done six months after the heuristic evaluation of the second prototype, ISGC. Each expert evaluator was approached individually, the purpose and procedure of the process was explained, and a copy of the DEVREF framework was provided for reference during their meta-evaluation.

The findings of the meta-evaluation clearly indicated that evaluating the evaluation framework itself was a meaningful exercise and that it achieved its aim of determining whether the evaluation statements in DEVREF cover all the pertinent considerations. Using the feedback received from the experts, DEVREF was further enhanced by incorporating six new evaluation statements and rephrasing two others. The framework in Table I is the final improved version of DEVREF.

 

Discussion and recommendations

DEVREF, developed and evaluated in this research, applies specifically to desktop VR. The reasons for using desktop technology instead of immersive technology are as follows.

►Desktop VR systems run smoothly on standard desktop PCs, using input devices such as a keyboard and mouse. At most, an additional graphics card with onboard memory may be required for effective delivery.

► The lower capital cost of hardware, software, and peripherals makes desktop VR systems an attractive and realistic alternative.

► The general low literacy level of employees in the mining industry exacerbates the problem of safety training. By starting with desktop VR, such training can be introduced gradually.

► The application of immersive systems is usually highly individualized since each trainee requires separate equipment to interact with the system. The high cost of such equipment makes it infeasible to train high numbers of trainees simultaneously. This study aims at augmenting safety training of the underground mining workforce, hence the focus is on proposing solutions that can cater for large numbers. A non-immersive training solution is therefore more viable and attractive at this stage.

The training material in the LSF and ISGC prototypes covered generic and geological hazards. The same design principles can be applied to evaluate other learning content, for example, generic induction, site-specific induction, equipment operation, drilling and blasting, loading and hauling, and various mining methods. As immersive training becomes more feasible over time, DEVREF can be extended to cover semi-immersive or immersive VR training systems. This can be done by adapting criteria or adding additional criteria.

 

Conclusion

VR offers innovative and versatile possibilities in training and holds potential to increase productivity and improve safety awareness, hence the number of VR applications in the industry is increasing. Due to the novelty of the technology for the local mining industry and the availability and growing acceptance of desktop computer training, the evaluation framework, DEVREF, was developed specifically for desktop VR training systems. Furthermore, DEVREF was meta-evaluated in the DBR process to improve it and strengthen its validity. Because of the feasibility and acceptance of this non-immersive technology, desktop VR training systems are already in use at several mine training sites throughout South Africa and it is envisaged that they will be used at many more sites in the foreseeable future. This ensures the viability and future use of the framework in evaluation.

Although DEVREF is presented as an evaluation framework, its criteria also relate closely to design aspects. Consequently the framework can serve two valuable theoretical purposes.

► In its primary purpose, it can be applied as an evaluation tool comprising criteria/heuristics to assess the effectiveness of existing interactive desktop VR training systems with regard to their usability, instructional design, VR systems design, and mining context-specific aspects.

► Since design principles are implicitly incorporated in the framework, DEVREF can also be used as a set of design principles to inform the design of new VR training systems.

To date, interactive training systems resulting from prototype systems described in this article have been implemented at fifteen training centres at various mines and smelting plants throughout South Africa. These training sites are committed to further development of VR training systems, and DEVREF can play a significant role in the effective design and evaluation thereof.

 

References

Adebesin, F., KoTZE, P., and Gelderblom, H. 2010. The complementary role of two evaluation methods in the usability and accessibility evaluation of a nonstandard system. Proceedings of the Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists (SAICSIT 2010).         [ Links ]

Alessi, S.M. and Trollip, S.R. 2001. Multimedia for Learning: Methods and Development. 3rd edn. Allyn and Bacon, Boston, MA.         [ Links ]

Ardito, C., Costabile, M.F., De Marsico, M., Lanzilotti, R., Levialdi, S., Plantamura, P., Roselli, T., Rossano, v., and Tersigni, M. 2004. Towards guidelines for usability of e-learning applications. User-centered Interaction Paradigmsfor Universal Access in the Information Society - 8th ERCIM Workshop on User Interfaces for All2004. Springer-Verlag, Berlin. pp. 185-202.         [ Links ]

Ardito, C., Costabile, M.F., De Marsico, M., Lanzilotti, R., Levialdi, S., Roselli, T., and Rossano, V. 2006. An approach to usability evaluation of e-learning applications. Information Society, vol. 4, no. 3. pp. 270-283.         [ Links ]

Arendarski, B., Termath, W., and Mecking, P. 2008. Maintenance of complex machines in electric power systems using virtual reality techniques. Proceedings of the 2008 IEEE International Symposium on Electrical Insulation (ISEI) 2008. Vancouver. IEEE, New York. https://ieeexplore.ieee.org/iel5/4544566/4570258/04570378.pdf        [ Links ]

Bennett, L., Stothard, P., and Kehoe, J. 2010. Evaluating the effectiveness of virtual reality learning in a mining context. SimTecT2010 Conference Proceedings. Leigh, D.E. (ed.). Simulation Industry Association of Australia, Brisbane.         [ Links ]

Bowman, D.A., Gabbard, J.L. and Hix, D. 2002. A survey of usability evaluation in virtual environments: Classification and comparison of methods. Presence: Teleoperators and Virtual Environments, vol. 11, no. 4. pp. 404-424.         [ Links ]

Bowman. D.A., Koller, D., and Hodges, L.F. 1998. A methodology for the evaluation of travel techniques for immersive virtual environments. Virtual Reality, vol. 3, no. 2. pp. 120-131.         [ Links ]

Chadwick, J. 2009. Simulators and virtual training. International Mining, September 2009. http://www.infomine.com/publications/docs/InternationalMining/Chadwick2009s.pdf [accessed 22 April 2010].         [ Links ]

Collins, C. 2012. Should virtual environments be realistic? Hypergrid Business Magazine, May 2012 http://www.hypergridbusiness.com/2012/05/should-virtualenvironments-be-realistic/ [accessed 16 January 2015].         [ Links ]

Costabile, M.F., De Marsico, M., Lanzilotti, R., Plantamura, V.L., and Roselli, T. 2005. On the usability evaluation of e-learning applications. Proceedings of the 38th Hawaii International Conference on System Science, 2005, IEEE Computer Society, Washington. pp. 1-10.         [ Links ]

Daden, 2016. Virtual reality without the hype. https://www.daden.co.uk/downloads/Virtual%20Reality%20-%20Without%20the%20Hype%20v1.pdf [accessed 10 October 2017].         [ Links ]

De Villiers, M.R. 2005. e-Learning artefacts: Are they based on learning theory? Alternation, vol. 12, no. 1b. pp. 345-371.         [ Links ]

Edmundson, A. 2003. Decreasing cultural disparity in educational ICTs: Tools and recommendations. Turkish Online Journal of Distance Education, vol. 4, no. 3. http://tojde.anadolu.edu.tr/tojde11/articles/edmundson.htm. [accessed 12 July 2009].         [ Links ]

Gang, W., Jun, G. and Yingzhen, L. 2006. The construction of evaluation framework for virtual geographic environment. Proceedings of the 16th International Conference on Artificial Reality and Telexistence, ICT '06 IEEE, New York. pp. 20-25.         [ Links ]

Google, 2017. Introducing Daydream standalone VR headsets. https://vr.google.com/daydream/standalonevr/ [accessed 15 October 2017].         [ Links ]

Guimarães, M.D.P., and Martins, V.F. 2014. A checklist to evaluate augmented reality applications. Proceedings of the XVI Symposium on Virtual and Augmented Reality, 2014. IEEE, New York. pp. 45-52.         [ Links ]

Hanna, N. and Richards, D. 2014. Evaluation framework for 3D collaborative virtual environments (the CORE). Proceedings of the Pacific Asia Conference on Information Systems. Paper 284. Association for Information Systems. http://elibrary.aisnet.org/Default.aspx?url=https://aisel.aisnet.org/cgi/ viewcontent.cgi?article=1104&context=pacis2014        [ Links ]

Hollender, N., Hofmann, C., Deneke, M., and Schmitz, B. 2010. Integrating cognitive load theory and concepts of human-computer interaction. Computers in Human Behavior, vol. 26. pp. 1278-1288.         [ Links ]

Hvannberg, E.T., Halldórsdóttir, G., and Rudinsky, J. 2012. Exploitation of Heuristics for Virtual Environments. Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design. Association for Computing Machinery, New York. pp. 308-317.         [ Links ]

Ji-zu, L., Li-mei, A., Jin-yun, W., and Xiao-li, Y. 2009. Research on simulation of safety behavior in coal mines based on the 3D and VIRTOOLS technologies. ICCSIT 2009: Proceedings of the 2nd IEEE International Conference on Computer Science and Information Technology. IEEE, New York. pp. 477-480.         [ Links ]

La Valle, S.M. 2017. Virtual Reality. http://vr.cs.uiuc.edu/ [accessed 12 October 2017].         [ Links ]

Lau, R.W.H., Yen, N.Y., Li, F., and Wah, B. 2014. Recent development in multimedia e-learning technologies. World Wide Web Journal, vol. 17, no. 2. pp. 189-198.         [ Links ]

Lucas, J., Thabet, W., and Worlikar, P. 2007. Using virtual reality (VR) to improve conveyor belt safety in surface mining. ITC Digital Library. http://itc.scix.net/cgi-bin/works/Show?w78_2007_42 [accessed: 10 June 2009].         [ Links ]

Magner, U.I.E., Schwonke, R., Aleven, V., Popescu, o., and Renkl, A. 2013. Triggering situational interest by decorative illustrations both fosters and hinders learning in computer-based learning environments. Learning and Instruction, vol. 29. pp. 141-152.         [ Links ]

Mason, R., Cooper, G., and Wilks, B. 2015. Using cognitive load theory to select an environment for teaching mobile apps development. Proceedings of the Seventeenth Australasian Computing Education Conference ACE2015, Sydney, Australia, January 2015. Australian Computer Society. https://epubs.scu.edu.au/cgi/viewcontentxgi?article=1664&context=busjourism_pubs        [ Links ]

Mayer, R.E. 2014. Incorporating motivation into multimedia learning. Learning and Instruction, vol. 29. pp. 171-173.         [ Links ]

Morrison, B.B., Dorn, B., and Guzdial, M. 2014. Measuring cognitive load in introductory CS: Adaptation of an instrument. Proceedings of the Tenth International Conference on Computing Education Research (ICER2014), Glasgow, Scotland. Association for Computing Machinery, New York. pp.131-138        [ Links ]

Munoz, R. and Chalegre, V. 2012. Defining virtual worlds usability heuristics. Proceedings of the Ninth International Conference on Information Technology: New Generations (ITNG), Las Vegas, 2012. IEEE, New York. pp. 690-695.         [ Links ]

Olsen, D.R. 2010. Building interactive systems: Principles for human-computer interaction. Course Technology, Cengage Learning. Boston, MA.         [ Links ]

Orr, T.J., Mallet, L.G., and Margolis, K.A. 2009. Enhanced fire escape training for mine workers using virtual reality simulation. Mining Engineering, vol. 61, no. 11. pp. 41-44.         [ Links ]

Oryx, 2014. Oryx launches Volvo simulators. http://www.miningmagazine.com/ technology/simulation/ [accessed: 10 January 2015].         [ Links ]

Oviatt, S. 2006. Human-centered design meets cognitive load theory: Designing interfaces that help people think. Proceedings of the 14th Annual ACM International Conference on Multimedia. pp. 871-880. Association for Computing Machinery, New York.         [ Links ]

Patel, H., Stefani, O., Sharples, S., Hoffmann, H., Karaseitanidi, I., and Amditis, A. 2006. Human centred design of 3-D interaction devices to control virtual environments. International Journal of Human-Computer Studies, vo. 64, no. 3. pp. 207-220.         [ Links ]

Rogers, Y., Sharp, H., and Preece, J. 2011. Interaction Design - Beyond Human-Computer Interaction. 3rd edn. Wiley, Chichester, UK.         [ Links ]

Rusu, C., Muñoz, R., Roncagliolo, S., Rudloff, S., Rusu, v., and Figueroa, A. 2011. Usability Heuristics for Virtual Worlds. Proceedings of AFIN 2011: the Third International Conference on Advances in Future Internet, -Nice, France. 21-27 August 2011. Curran Associates, Red Hook, NY. pp. 16-19.         [ Links ]

Schafrik, S.J., Karmis, M., and Agioutantis, Z. 2003. Methodology of incident recreation using virtual reality. Proceedings of the SME Annual Meeting, Cincinnati, Ohio, February 2003. Society for Mining, Metallurgy & Exploration, Littleton, CO.         [ Links ]

Schofield, D. 2014. A virtual education: Guidelines for using games technology. Journal of Information Technology Education: Innovations in Practice, vol. 13. pp. 25-43.         [ Links ]

Ssemugabi, S. and de Villiers, M.R. 2007. Usability and learning: A framework for evaluation of web-based e-learning applications. Proceedings of Ed-Media 2007: World Conference on Educational Mulãmedia,, Hypermedia & Telecommunications, Vancouver, Canada, 25 June 2007. Association for the Advancement of Computing in Education. Waynesville, NC. http://hdl.handle.net/10500/13207        [ Links ]

Stoddard, E. and Skweyiya, S. 2016. SA mine deaths rise after years of improving safety/ https://www.iol.co.za/business-report/economy/sa-mine-deaths-rise-after-years-of-improving-safety-2063416 accessed 26 November 2017].         [ Links ]

Stone, R.J. and Knight, J.F. 2012. 3D Displays: A Human-Centred Review. BAE Systems, Yeovil, Somerset, UK: https://www.birmingham.ac.uk/Documents/college-eps/eece/research/bob-stone/HFI-DTC-publication-3D-displays.pdf        [ Links ]

STS3D, 2017. Accident reconstruction. http://sts3d.co.za/_workplace-safety/. [accessed 10 October 2017].         [ Links ]

Su, K., Wang, H., Wu, Y. and Kuo, N. 2013. The interface design and usability evaluation of interactive virtual reality navigation system. Applied Mechanics and Materials, vol. 302. pp. 635-639.         [ Links ]

Sweller, J., Ayres, P., and Kalyuga, S. 2011. Cognitive Load Theory. Springer, New York.         [ Links ]

Terãs, Η. and Herrington, J. 2014. Neither the frying pan nor the fire: In search of a balanced authentic e-learning design through an educational design research process. International Review of Research in Open and Distance Learning, vol. 15, no. 2. http://www.irrodl.org/index.php/irrodl/article/view/1705/2835        [ Links ]

Tichon, J. and Burgess-Limerick, R. 2011. A review of virtual reality as a medium for safety related training in mining. Journal of Health and Safety Research and Practice, vol. 3, no. 1. http://burgess-limerick.com/download/a54.pdf        [ Links ]

Tsiatsos, T., Andreas, K., and Pomportsis, A. 2010. Evaluation framework for collaborative educational virtual environments. Educational Technology and Society, vol. 13, no. 2. pp. 65-77.         [ Links ]

Van Wyk, E.A. 2015. An evaluation framework for virtual reality safety training systems in the South African mining industry. PhD thesis, University of South Africa.         [ Links ]

Van Wyk, E.A. and de Villiers, M.R. 2009. Virtual reality training applications for the mining industry. Proceedings of the ACM Afrigraph Conference 2009, Pretoria. Association for Computing Machinery, New York. http://hdl.handle.net/10500/13155        [ Links ]

Van Wyk, E.A. and de Villiers, M.R. 2014. Applying design-based research for developing virtual reality training in the South African mining industry. Proceedings of SAICSIT 2014, Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists. Pretoria. pp. 70-81.         [ Links ]

Van Wyk, E.A. and de Villiers, M.R. 2016. Applying educational design research to virtual reality safety training in mines. Proceedings of ECRM 2016: the 15th European Conference on Research Methodology for Business and Management Studies. Kingston University, London, UK, 9-10 June 2016. Academic Conferences and Publishing International Limited.         [ Links ]

Van Wyk, E.A. and Prinsloo, M.W. 2015. Applying virtual reality to prevent repeat incidents. Proceedings of Virtual Reality and Spatial Information Applications in the Mining Industry, University of Pretoria, July 2015. Southern African Institute of Mining and Metallurgy, Johannesburg.         [ Links ]

Vrasidas, C. 2004. Issues of pedagogy and design in e-learning systems. Proceedings of the 2004 ACM Symposium on Applied Computing, Cyprus, March 2004. Association for Computing Machinery, New York. http://dl.acm.org/citation.cfm?id=968086        [ Links ]

Webber-Youngman, R.C. 2014. Students need to become 'imagineers', academic urges. Mining Weekly. http://www.miningweekly.com/article/students-need-to-become-imagineers-academic-urges-2014-09-16 [accessed: 9 January 2015].         [ Links ]

Webber-Youngman, R.C.W. and Van Wyk, E.A. 2013. Incident reconstruction simulations-potential impact on the prevention of future mine incidents. Journal of the Southern African Institute of Mining and Metallurgy, vol. 113, no. 6. pp. 519-528.         [ Links ] ♦

 

 

Correspondence:
E.A. van Wyk
Email: vanwykea@tut.ac.za

Received: 7 Mar. 2018
Revised: 26 Sept. 2018
Accepted: 29 Oct. 2018
Published: May 2019

 

 

ORCiD ID: E.A. van Wyk. https://orchid.org/0000-0002-0436-2741
M.R. de Villiers. https://orchid.org/0000-0002-9849-6723