GENERAL ARTICLES

 

Evaluating the ability of decision-makers to estimate risks effectively in industrial applications

 

 

W. Pieterse^{I, *, 1}; S. Grobbelaar^II; J.K. Visser^III

^IDepartment of Engineering and Technology Management. Graduate School of Technology Management. University of Pretoria, South Africa. werner_pieterse@tuks.co.za
^IIDepartment of Engineering and Technology Management. Graduate School of Technology Management. University of Pretoria, South Africa. sgrobbelaar@york.co.za
^IIIDepartment of Engineering and Technology Management. Graduate School of Technology Management. University of Pretoria, South Africa. krige.visser@up.ac.za

 

 

---

ABSTRACT

There are many factors to be considered in risk management that influence the effective control of risk either beneficially or detrimentally. An improvement in the quality of risk analysis will enable risk managers and decision-makers to make more accurate decisions that would benefit all stakeholders. The aim of this research was to determine the effectiveness of the most commonly employed risk analysis methods. Two decision problems and two hypothetical risk analysis problems were presented to groups of respondents who were asked to choose a correct answer from the given options. The results showed no significant association between a person's age or experience and their choice or decision about risk management. The only association was found between a higher level of professional qualification and the correct choice, when a decision problem was given without providing a decision tree.

---

OPSOMMING

Verskeie faktore moet in ag geneem word in risikobestuur wat die effektiewe beheer van risikos voordelig of nadelig beïnvloed. 'n Verbetering in die kwaliteit van risiko analise kan risikobestuurders en besluitnemers help om meer akkurate besluite te neem tot voordeel van alle belanghebbers. Die doel van hierdie studie was om die effektiwiteit van die mees algemene risiko bepaling metodes te evalueer. Twee besluitnemingsprobleme en twee risiko-analise probleme is vir groepe respondente voorgehou, en respondente is versoek om 'n korrekte keuse te maak uit 'n lys moontlike oplossings. Die resultate toon dat ouderdom of ervaring nie 'n betekenisvolle invloed op 'n effektiewe besluit of keuse met betrekking tot risikobestuur het nie. 'n Betekenisvolle assosiasie is wel waargeneem tussen 'n hoër vlak van professionele kwalifikasie en die korrekte keuse, wanneer 'n besluitnemingsprobleem sonder dat 'n besluitboom verskaf is.

---

 

 

1 INTRODUCTION

1.1 Introduction and background

In the mechanical maintenance environment of a petrochemical plant, Section 2.1 of the General Machinery Regulation (GMR) of the 1993 Occupational Health and Safety Act states that an appointee is responsible for the integrity of the equipment allocated to him/her. The GMR 2.1 appointee has to make decisions about maintenance schedules in order to adhere to legal requirements. The law states (under the Pressure Equipment Regulation (PER), of the 1993 Occupational Health and Safety Act) that pressure equipment - that is, everything with a design pressure of 50 kPa or higher - and its safety devices (e.g., pressure and temperature safety valves) must undergo statutory inspections every three years.

A company could decide to delay a particular piece of equipment's statutory inspections. In most cases, the reason for delaying the inspection is the fear of jeopardising production. Sections 11 and 12 of the PER stipulate the detailed legal requirements. Section 12 pertains specifically to the requirements for voiding the inspections, and the mitigating actions that must be in place (e.g., a Risk-Based Inspection (RBI) process approved by the chief inspector). The output from this RBI study is the possible failure modes and an appropriate risk value. General practice in the petrochemical industry is to evaluate the risk value (probability and consequence) and make a decision whether to delay the inspection, based on the equipment's risk value.

The above-mentioned example clearly illustrates typical difficult risk-based decisions that are required of managers in industry. Many managers in different disciplines are frequently faced with difficult decisions. These disciplines include financial, marketing, procurement, logistics, construction, manufacturing, project, and engineering management, to name a few. Apart from making a decision based on the information presented to the risk bearer, there is also a possibility that the manner in which the data was analysed, synthesised, interpreted, and quantified was insufficient or incorrect. This will inevitably lead to an increased risk of making an incorrect or sub-optimal decision.

1.2 Rationale of the study and problem statement

Within the petrochemical industry, various decision-making methods are used to make decisions about process hazards and risks. The methods used to determine such risks often lack consistency. The outcome of the methods used is also determined by factors such as the risk appetite of the risk bearer, and the objectivity and experience of the assessor. These factors often lead to inconsistent outcomes from the risk analysis. The associated research questions were therefore formulated as follows:

• Do the outcomes of risk analysis depend on the experience, objectivity, and other factors of the assessors?

• Do workers and other people quantify risks correctly?

• Does a higher level of education increase the ratio of correct risk analysis?

1.3 Research propositions

The following propositions were formulated from the research questions above:

a) The outcome of risk analysis will vary depending on the experience, objectivity, and other factors related to the risk assessors.

b) People tend to quantify incorrectly the risks pertaining to more complex systems.

c) The higher the assessor's level of education, the more likely the assessor is able to quantify risks correctly.

 

2 LITERATURE

2.1 What is risk analysis, and why is it conducted?

Risk analyses are conducted to determine the probability of a situation occurring and the severity of that situation. According to Koller [1] and Zio and Aven [2], two parameters are associated with risk:

• The probability of the risk event occurring; and

• The consequence or impact if the event should occur (in terms of criteria such as monetary loss, injury to personnel, or environmental damage).

According to Moubray [3], a third parameter, which is the most contentious, is also present:

• Whether the risk is acceptable or not (to the risk bearer).

Vose [4] summarised his risk management process in the following seven steps:

1. Formulate the problem (this can take the form of a project, new construction, a process safety hazard, an environmental hazard, etc.).

2. Identify the risks, the risk drivers, and the risk management options.

3. Define quantitative scenarios to assist in selecting different options.

4. Review and scrutinise the data that is available.

5. Design, simulate, and validate the model.

6. Review the results.

7. Apply risk management, through either abatement or mitigation strategies.

2.2 Naturalistic decision-making method

According to Yanqing et al. [5], the recognition-primed decision-making model is a primary naturalistic decision-making method. The emphasis is on how decision-makers handle difficult and complex situations in various challenging environments. The more experienced a decision-maker is, the better-suited the person is to assess the situation and make a decision based on previous experience and encounters. A more experienced decision-maker, for example, is able to recognise patterns through experience and careful observation. Mosier and Fischer [6] state that a critical aspect of expertise is the aptitude to synthesise and construe information into coherent knowledge. The successful interpretation of information is a factor for success in resolving complex situations and undertakings.

Yanqing et al. [5] describe the recognition-primed decision-making model in the following four steps:

1. Recognise the current situation to identify which course of action would make the most sense. This is termed 'situation-experience matching', where current situations are matched with past experiences.

2. Evaluate the course of action by mental simulation.

3. Structure the process as part of a decision-action cycle and not just a single conclusion.

4. Adapt and refine the decision through a feedback loop and account for changing situations.

In the second process (the course of action), each possible action is evaluated and measured for how it will evolve. The decision-maker can modify the action through the course of the occurrence, or reject the action either initially or during the course of the action, and pursue alternatives.

According to Carvalho et al. [7], a naturalistic decision-making process involves the recognition of patterns from the decision-maker's tacit knowledge. This can also take the form of rules, where if a certain condition is present, a certain action should be followed (condition-action rules). According to Klein [8], naturalistic decision-making is how people use their experience to make decisions in real situations: this is also termed the 'recognition-primed decision model' [8]. The typical action steps of this model or process are indicated in Figure 1.

 

 

2.3 The Monty Hall problem

The Monty Hall problem is a puzzle of probabilities where a decision must be made, based on partial information [9, 10]. This problem originated from the game show, 'Let's Make a Deal', where the show host's name was Monty Hall. In this game, there are three doors: behind two of them are goats, and behind the third is a car or other prize. The game show's host knows which door reveals the car, and the contestant is asked to pick a door. After the contestant has picked a door, the game show's host then opens one of the two remaining doors to reveal a goat. The contestant is then given the option of switching between the door originally chosen (which is still closed) and the other remaining closed door, or to stay with his/her original choice of a door. Figure 2 illustrates the Monty Hall puzzle/game.

 

 

It can be proven mathematically that the chance of the contestant winning the car (or other prize) if he/she chooses to switch would be 2/3, versus 1/3 if the contestant does not switch. The decision diagram or tree for the Monty Hall game or puzzle is illustrated in Figure 3.

 

 

3 CONCEPTUAL METHOD

3.1 Theories used in this study

3.1.1 Basic probability theory; series and parallel systems

Probability can be defined as "the ratio of the number of outcomes in an exhaustive set of equally likely outcomes that produce a given event to the total number of possible outcomes" [12].

For systems, independent components with known failure rates can be grouped into either series or parallel, depending on the interaction between them.

For components in series, all the components in series must be working in order for the system to work. The probability of the system functioning is given by Equation 1 :

where P_i is the probability that component i is functioning and n is the number of components in series.

For components in parallel, one of the components must be working in order for the system to work. The probability of the system functioning is therefore given by Equation 2.

where P_iis the probability that component i is functioning and n is the number of components in parallel.

3.1.2 Basic reliability theory and reliability of components

Reliability is defined as "the probability that a system or a component in the system will perform its intended function without failure, for a specified time period, under certain conditions" [13].

For a constant hazard rate, h, the following equations can be used to determine the reliability (Equation 3) or unreliability (Equation 4) of a component:

where t is the time since the start of the operation.

3.1.3 Using the recognition-primed decision-making model

In a petrochemical facility, systems are complex and intricate. When evaluating process risks, the more experienced the evaluators and the more attention that is given to detail, the better the chance of obtaining accurate evaluations.

The benefit of the naturalistic decision-making model lies in its feedback loops, where the results obtained are first put under scrutiny before they are finalised. This process is illustrated in Figure 4.

 

 

If the result of a decision is found to be incorrect or inaccurate, the evaluation must be repeated with the new information that is available.

 

4 RESEARCH METHODOLOGY

4.1 Theoretical approach to the investigation, and chosen research tools

According to Scandura and Williams [14], no study is flawless. However, methods exist to increase the accuracy of results and hence the correctness of the conclusions. One such method is triangulation - a concept that originates in the use of multiple methods to study problems in order to examine the discriminant and the convergent validity of measures. Triangulation improves the external validity (generalisation) of the findings. In this study, formal theory, sample surveys, and experimental simulations were used to increase the accuracy of the results.

4.2 Research methodology

A literature survey was used to determine which models were available for improved risk analysis. The hypotheses were developed based on the literature survey, the research problem, the research questions, and the research propositions. The hypotheses were tested through sample surveys that were completed by respondents. Hypothetical risk analysis problems were developed that represent the experimental simulations, and increase the precision in measurement and realism of context of the study. Reliability theory was used to calculate the correct risk analysis values for the hypothetical risk analysis problems/systems.

A probability puzzle was also derived from the theory (i.e., the Monty Hall puzzle). This problem was used as a representation of a complex probability puzzle that can be similar to a complex probability and risk analysis faced by risk bearers. The probability puzzle and the hypothetical risk analysis problems were used in a survey to determine the correctness of the answers, and to determine whether there was any association between the correct assessments and a higher level of education, age, or experience.

4.3 Surveys

Three surveys were presented to different groups of respondents. Survey results were obtained by posting the links on social media platforms, e-mailing them to prospective participants, and purchasing audiences through SurveyMonkey® [15]. The Monty Hall problem was presented to two groups: one group had a decision tree to aid in the decision-making process, while the other group did not have a decision tree and the answers were left to deductive reasoning. The third survey comprised the calculation of the probability of failure for a hypothetical pressure safety valve (PSV) and a complex system, given some reliability data. The complex system comprised three pressure safety valves and a pressure control valve. This survey was presented to technicians and engineers in the energy, utility, and extraction industries.

4.3.1 Age distribution

The age distribution of the respondents who participated in the three surveys (the Monty Hall problem without a decision tree, Monty Hall problem with a decision tree, and a risk analysis) is shown in Figure 5.

 

 

A large proportion of the respondents for the first survey (Monty Hall problem without a decision tree) were in the 20-35 age group. The distribution of age for the second and third surveys was more balanced, with quite a number of respondents in the older age groups.

4.3.2 Experience distribution

The distribution of the number of years' experience of the respondents who participated in the three surveys is shown in Figure 6.

 

 

More than half of the respondents in the second survey (Monty Hall problem with a decision tree) and third survey (risk analysis) had in excess of 15 years' experience. The experience level for the first survey (Monty Hall problem without a decision tree) was more balanced, with nearly half in the 6-15 years' experience group.

4.3.3 Education level distribution

The distribution of the different education levels of the respondents who participated in the three surveys is shown in Figure 7.

 

 

For all three surveys, the respondents with Bachelor's degrees were by far the largest proportion - about 40 per cent. Persons with higher degrees were only represented well in the survey for the Monty Hall problem without a decision tree (about 47 per cent for all postgraduate education).

 

5 RESULTS

5.1 Overview

Statistical analysis was done for the three surveys mentioned above. In order to test for any significance between age, level of experience, and level of education on the one hand, and a correct choice in any of the surveys on the other, the P-values determined by means of the Chi-square statistical test were used. The CHISQ.TEST function provided by MS Excel® [16] was used for this purpose. According to Page and Meyer [17], the P-value is defined as the chance that the null hypothesis is true. If the P-value is less than the accepted percentage (5 per cent in this case), then it is unlikely that the null hypothesis is true, and the null hypothesis must be rejected: this relates to a significant test result. Alternatively, if the P-value is greater than 5 per cent, then the null hypothesis is not unlikely and the null hypothesis cannot be rejected. This relates to a non-significant test result.

5.2 Results of surveys

The results of the three surveys are discussed in separate sections below. The first survey evaluated the Monty Hall problem without a decision tree, the second survey evaluated the Monty Hall problem with a decision tree, and the third survey evaluated the risk analysis for both a pressure safety valve (PSV) and a complex technical system.

5.2.1 Monty Hall problem without a decision tree

This survey was sent to South African respondents. The response was good, with 274 usable responses. The results of a Chi-square test to compare the actual number of correct choices against the expected number of correct choices are shown in Tables 1 to 3.

Test for association pertaining to age

 

 

Test for association pertaining to level of experience

 

 

Test for association pertaining to level of education

 

 

The results listed in Tables 1 to 3 indicate no association between age or level of experience and the correct choice for the Monty Hall problem without a decision tree. However, the results indicate an association between higher levels of education and the correct choice for the Monty Hall problem without a decision tree.

5.2.2 Monty Hall problem with a decision tree

"The survey was sent initially to South African respondents. Groups that have not been requested to participate in the survey without a decision tree was approached originally." However, the response rate was not good enough to compare it with the survey without a decision tree: there were only about 100 responses, which was not even half of the response rate obtained for the survey without a decision tree. Therefore more responses were purchased through SurveyMonkey® [15]. A total of 212 respondents (including the unusable responses) were obtained from people located in the United States of America. A small number of responses from South Africa were also obtained (approximately 30, including the non-usable responses). A total of 219 usable responses were obtained for this survey. The results of the Chi-square-test to compare the actual number of correct choices against the expected number of correct choices are given in Tables 4 to 6 below.

The results listed in Tables 4 to 6 indicate no association between age, level of experience, or higher levels of education and the correct choice for the Monty Hall problem with a decision tree provided.

Test for association pertaining to age

 

 

Test for association pertaining to level of experience

 

 

Test for association pertaining to level of education

 

 

5.2.3 Risk analysis of a pressure safety valve (PSV)

For this survey, 22 responses were obtained from a South African petrochemical company. Only mechanical engineers and mechanical technicians participated. A total of 53 additional responses were purchased through SurveyMonkey® [15]. The 53 respondents were engineers from various disciplines in the utility, energy, and extraction sections from the United States of America. The results of the Chi-square-test to compare the actual number of correct choices against the expected number of correct choices are given in Tables 7 to 9.

Test for association pertaining to age

 

 

Test for association pertaining to level of experience

 

 

Test for association pertaining to level of education

 

 

The results listed in Tables 7 to 9 indicate no association between age, level of experience, or level of education, and a higher incidence of the correct risk analysis for the hypothetical PSV.

5.2.4 System risk analysis

The same population that was used for the risk analysis of the PSV was also requested to determine the probability of failure for a complex process system that comprises a number of valves arranged in a parallel configuration. The total number of responses was therefore also 75. The results of the Chi-square-test to compare the actual number of correct choices against the expected number of correct choices are given in Tables 10 to 12.

P-value test for association pertaining to age

 

 

P-value test for association pertaining to level of experience

 

 

P-value test for association pertaining to level of education

 

 

The results in Tables 10-12 indicate no association between age, level of experience, or level of education and a higher incidence of the correct risk analysis for a complex system.

5.3 Summary of results

The P-values, as determined by means of the Chi-square statistical test for the three surveys discussed in the previous section, are summarised in Table 13.

 

 

No association was found for age, level of experience, or level of education and the correct choice for two decision problems and two risk analyses - except for the Monty Hall problem without a decision tree and the level of education.

 

6 CONCLUSIONS AND RECOMMENDATIONS

6.1 Proposition (a)

Proposition (a) stated that "the outcome of risk analysis will vary depending on the experience, objectivity, and other factors of the assessors".

In both the Monty Hall surveys, the respondents were requested to write a short discussion of why their choice was made. Some answers included a detailed description of the mathematics; some were correct and some were incorrect. This indicates that people approach choices such as these differently. Some people will go according to a feeling, and others will weigh the options and attempt to calculate the correct answer. The comments that some respondents provided in the questionnaire revealed that bias, risk appetite, and indifference played a role in their decision-making. The difference in responses and answers is most likely attributable to a difference in the use of decision-making methods. People will make decisions based on the methods known to them or to which they are accustomed. This is an indication that certain factors play a role in decision-making, and that these factors tend to vary in prominence in individuals. In none of these surveys did experience indicate a higher incidence of a correct choice.

6.2 Proposition (b)

Proposition (b) stated that "people tend to quantify incorrectly the risks pertaining to more complex systems".

The Monty Hall problem is a challenging probability puzzle, and the correct choice is not clear. Although this problem is not analogous to a specific situation that might be presented in industry, it is fair to say that managers and decision-makers are faced with difficult probability problems on a regular basis. The correct choices are not always clear, and careful consideration must be given and deliberation undertaken before decisions are made. There was no association between age, level of education, or level of experience and a higher incidence of a correct choice of the risks in any of the surveys, except in the ase of the Monty Hall problem without a decision tree and a higher level of education. The results of this research indicated that experience does not necessarily lead to the correct choice in decision problems or in quantifing risk for complex systems correctly. An in-depth theoretical knowledge of decision-making and risk analysis is required to quantify risk in complex systems correctly.

6.3 Proposition (c)

Proposition (c) stated that "the higher the assessor's level of education, the more likely the assessor is able to quantify risks correctly".

The survey presented to the respondents from the United States of America revealed that there is no association between a higher level of education and a higher incidence of a correct choice of the Monty Hall problem, even if a decision tree is provided to aid in the decision-making process. The South African respondents did, however, indicate a higher incidence rate of a correct selection of the Monty Hall problem with a higher level of education. The results of the Monty Hall problem without a decision tree indicated an association between the level of education of the respondents and a higher incidence rate in a correct choice. The results have proven that a higher level of education can result in a higher likelihood of correctly quantifying risk, but that it is not always true. There was no association between level of education and a higher incidence of the correct risk analysis made on the hypothetical PSV and complex system, nor for the Monty Hall problem presented with a decision tree.

6.4 General recommendations

Companies should implement better and improved systems for decision-making about operational risk. In the petrochemical industry, although the risk assessment and analysis process has been used extensively, it lacks substance and the assurance of correct results. A typical risk analysis process pertaining to financial risks in a petrochemical industry follows a much more stringent approach, and results are governed more extensively. These financial analyses employ tools such as Monte Carlo analyses and scrubbing sessions, where experts review the assumptions and calculations. Financial risk analyses processes used in the petrochemical industry follow a much more robust process, very similar to the recognition-primed decision-making process, where there are reviews and feedback loops to ensure that the correct results are obtained through iteration. Similar processes should be developed for process and operations risk analyses.

This study did not indicate any association between level of experience and an increase in correct decisions about the questions presented in the survey. This indicates that there is a need for improved procedures, where choices and conclusions made during risk analysis are first put under scrutiny and evaluated by an expert panel. Experts are regarded as people with extensive knowledge in the specific subject matter at hand. The expert panel should agree to the validity of the results before they are presented to the risk bearer for decision-making. The panel will evaluate a 'Go' or 'No Go' situation and direct the way forward. There was an association between a higher level of education and a higher incidence of a correct decision in the Monty Hall problem without a decision tree provided. Higher levels of education relate to greater knowledge and better reasoning abilities [18]. Therefore experts will be able to assist in proper decision-making. However, in the cases of the PSV and system risk analyses, higher education levels did not produce improved decision-making.

The problems that were hypothetically constructed for this study are simple situations that are typically faced by decision-makers on a daily basis. However, determining the correct risk value related to each system is not straight forward, and it was shown that people could not correctly identify the risk values. This suggests that decisions such as these must be presented to experts who will review the systems and processes properly, apply the correct analyses tools, and present the results to the decision-makers. This is all in an effort to establish improved operational risk management processes and procedures.

Processes and procedures should not be too cumbersome or lengthy, but should be concise and reliable in providing information. According to Mosier and Fischer [6], by exerting excessive control a slow build-up of tension could occur. Excessive control also makes it difficult to get work done timeously. People will therefore find workarounds that will enable the system to work despite itself, disguising failure until the system fails catastrophically. For this reason, better and improved processes should not be too cumbersome.

The recognition-primed decision-making model places emphasis on the experience of the person who must make a decision. Although there is proof that the recognition-primed decision-making model will enable decision-makers to be better suited to making correct decisions, no association was found in any of the surveys between age and level of experience. The only association was between level of education and a correct choice of the Monty Hall problem without a decision tree provided. The other surveys (Monty Hall with a decision tree, and the PSV and system risk analysis) also showed no association between a higher incidence of a correct choice and the level of education.

The effectiveness of the recognition-primed decision-making model could not be proven in the analyses (although it was not part of the original propositions). The recognition-primed decision-making model should be better suited for decision-making because it has a feedback loop where results are first subjected to analysis. It is suggested that more research be conducted where hypothetical situations are constructed and persons with expertise in the subject of the situation can participate in the study.

6.5 Recommendations for further research within specific industries

To determine whether a more robust approach to the analysis of risks and the mitigation thereof would be beneficial, a study into a specific industry is suggested. Different companies within the same industry should be reviewed in terms of their processes followed for operations risk assessments. Their incident rates should also be reviewed. Incident rates can be measured as:

• Recordable cases (Occupational Safety).

• Number of fires, explosions, and releases (Process Safety).

If the results indicate that a more robust process that which is better governed leads to a reduction in incidents, then the results can be used to persuade similar companies to follow suit.

6.6 Recommendations for further research into cumulative risk

General practice in the industry is to evaluate the risk value and make a decision on delaying an inspection, based on the single equipment's risk value (as an example of a risk analysis). However, what is not taken into consideration is the cumulative risk of all the risks identified in the area. It must be determined whether a holistic approach should be followed, and whether the total risk should be taken into consideration before a decision is made. Delaying the statutory inspection of equipment might not be advisable if the cumulative risk is already high, even though the appropriate risk value of the individual equipment might be low. This field has not yet been researched extensively, and further investigation might be advisable.

 

REFERENCES

[1] Koller, G. 2005. Risk assessment and decision making in business and industry. United States of America: Chapman and Hall/CRC.         [ Links ]

[2] Zio, E. & Aven, T. 2013. Industrial disasters: Extreme events, extremely rare. Some reflections on the treatment of uncertainties in the assessment of the associated risks. Process Safety and Environmental Protection, 91(1,2), pp. 31-45.         [ Links ]

[3] Moubray, J. 1997. Reliability-centered maintenance. New York, United States of America: Industrial Press Inc.         [ Links ]

[4] Vose, D. 2008. Risk analysis, a quantitative guide. 3^rd edition, England: John Wiley & Sons.         [ Links ]

[5] Yanqing, J., Massanari, R.M., Ager, J., Yen, J., Miller, R.E. & Ying, H. 2007. A fuzzy logic-based computational recognition-primed decision model. Information Sciences, 177(20), pp. 4338-4353.         [ Links ]

[6] Mosier, K.L. & Fischer, U.M. 2011. Informed by knowledge. Expert performance in complex situations. United States of America: Taylor and Francis Group.         [ Links ]]

[7] Carvalho, P.V.R., Dos Santos, I.L. & Vidal, M.C.R. 2005. Nuclear power plant shift supervisor's decision making during micro-incidents. Industrial Ergonomics, 35(7), pp. 619-644.         [ Links ]

[8] Klein, G. 1999. Sources of power: How people make decisions. Massachusetts, United States of America: The MIT Press.         [ Links ]

[9] Baratgin, J. 2009. Updating our beliefs about consistency: The Monty-Hall case. Mathematical Social Science, 57(1), pp. 67-95.         [ Links ]

[10] Granberg, D. 1999. A new version of the Monty Hall Dilemma with unequal probabilities. Behavioural Processes, 48(1,2), pp. 25-34.         [ Links ]

[11] Anonymous. 2014. Monty Hall Problem. URL: http://en.wikipedia.org/wiki/Monty_Hall_problem. Accessed on 5 November 2014.         [ Links ]

[12] Encyclopedia Britannica. No date. Miriam Webster Dictionary. URL: http://www.merriam-webster.com/dictionary/probability. Accessed on 5 November 2014.         [ Links ]

[13] Lewis, E.E. 1995. Introduction to reliability engineering. 2^nd edition, Wiley.         [ Links ]

[14] Scandura, T.A. & Williams, E.A. 2000. Research methodology in management: Current practices, trends, and implications for future research. Management Journal, 43(6), pp. 1248-1264.         [ Links ]

[15] SurveyMonkey®. URL: http://www.surveymonkey.com. Accessed on 5 November 2014.         [ Links ]

[16] Microsoft Excel®, http://www.microsoft.com Accessed on 5 November 2014.         [ Links ]

[17] Page, C. & Meyer, D. 1999. Applied research design for business and management. England: McGraw-Hill.         [ Links ]

[18] Thomas, A. & Thorne, G.. How to increase higher order thinking. URL: http://www.readingrockets.org/article/34655. Accessed on 5 November 2014.         [ Links ]

 

 

* Corresponding author
1 The author was enrolled for an M Eng (Engineering Management) degree in the Department of Engineering and Technology Management, University of Pretoria