GENERAL ARTICLES

 

Criteria for selection and gate reviews of technology innovation projects

 

 

B. Binneman^{I, 1} ; H. Steyn^{II, *}

^IDepartment of Engineering and Technology Management Graduate School of Technology Management University of Pretoria, South Africa
^IIDepartment of Engineering and Technology Management Graduate School of Technology Management University of Pretoria, South Africa. Herman.Steyn@up.ac.za

 

 

---

ABSTRACT

This paper describes appropriate project selection criteria and gate review-point criteria for government agencies that fund technology innovation projects in South Africa. Data was gathered via a Delphi survey in which seven expert respondents participated. The most significant project selection criteria and relevant gate reviewing criteria were established. The proposed basic framework consists of fourteen selection criteria, their associated weights, and forty relevant gate reviewing criteria. Commercial risk and feasibility was considered the most important criterion, followed by the team's skills and competence in second place. The use of these criteria when considering investment in technology innovation projects could improve the likelihood of commercial success.

---

OPSOMMING

Hierdie artikel beskryf, vir staatsinstansies wat projekte vir tegnologiese innovasie in Suid Afrika befonds, toepaslike kriteria vir die seleksie van projekte en vir oorsig by hersieningspunte aan die einde van projek fases. Data is ingewin deur middel van 'n Delphi-opname waaraan sewe deskundige respondente deelgeneem het. Die mees beduidende kriteria vir seleksie en oorsig is bepaal. Die voorgestelde raamwerk bestaan uit veertien seleksie-kriteria, relatiewe gewigte vir elk, en veertig kriteria vir hersiening by oorsigpunte. Kommersiële risiko en doenbaarheid word beskou as die belangrikste kriterium, wat gevolg word deur die span se vaardigheid in die tweede plek. Die gebruik van hierdie kriteria wanneer investering in projekte vir tegnologiese innovasie oorweeg word, kan die waarskynlikheid van kommersiële sukses verhoog.

---

 

 

1 INTRODUCTION

1.1 Selection of projects for technological innovation

Technical innovation is important for economic growth and for enhanced competitiveness [1,2,3,4]. As innovation and entrepreneurship are important drivers of productivity and economic growth, South African government funding agencies aspire to stimulate and commercialise technological innovation. For agencies to achieve their objectives successfully, they have to select projects for investment carefully and strategically in the high risk environment where these technology innovation projects are to be found. There are many instances where projects are carefully and strategically selected, and planned and executed on time and within budget, but still fail to produce actual socio-economic benefit, impact, revenue, and/or profit. They also often fail to establish new ventures or the expected small, medium, and micro enterprises (SMMEs); and they have difficulty in commercialising the final product/service/process [5]. Management of research and development (R&D) projects is quite complex due to the substantial uncertainties in a project life-cycle. Each project phase poses different risks and uncertainties [6]. Despite the range of available solutions and guidelines, a benchmarking study indicated that project selection and prioritisation is one of the weakest facets of new product management activities [7].

In the competitive globalised market, efforts are made to promote the commercialisation of university research [8], to reform their research system to focus on new and innovative technologies, and to set up structures to support activities aimed at increasing commercialisation and SMME start-ups from publicly-funded research. One of the difficulties is that the concept of 'project success' has not been well-defined in the project management literature. Failure is also an ill-defined term that is used by practitioners and in the literature [9]. For the purposes of this paper, 'project success' implies successful commercialisation.

The complex task of commercialisation, to bring highly uncertain technologically innovative and R&D projects to the market, has prompted many researchers over recent decades to investigate the critical factors leading to the success or failure of these projects [10]. It is hard to explain why so many of these projects fail, or never reach the commercial markets; and so very few do, in fact, succeed. To a large degree, the high level of uncertainty of innovative projects implies high risk that often results in failure [11]. Successful project management techniques contribute towards the achievement of project success, but project management, in itself, cannot ensure that a project will necessarily succeed [12].

In the literature, significance is given to three main factors that contribute to the success of R&D, new product, and technology innovation projects:

• Market strength is based on the analysis of (a) the potential size of the market, (b) the expected market share, and (c) the expected profitability of the new product or service [13]. Finding interesting projects is often hard for government financiers, and the uncertainty makes it hard to select high-potential projects.

• The technology, its patentability, and factors that influence the decision to commercialise a patent were identified as possible factors for selecting a project in which to invest [14]. Svensson's [14] results showed that the larger the share of the patent-owners' costs covered by government financing, the lower the probability of patents being commercialised. This is likely to be dependent on the soft terms on which government loans are provided (e.g. whether the patent owner can avoid paying back the loan if the patent is never commercialised), or the government's ability to select promising projects. Cooper and Kleinschmidt [15] found that the relationship between innovativeness and commercial success is U-shaped rather than linear.

• The organisational planning and scheduling process for the introduction of a new product from an R&D project is considered very important in a number of studies. Using quantitative techniques for project selection was deemed important by Liberatore and Titus [16]. Their study found that firms using such techniques had better results. Others, such as Mansfield and Wagner [17] and McGinnis and Ackelsberg [18], found that quantitative project selection was negatively correlated with economic success.

According to Nicholas and Steyn [19], there are two phases to project selection: pre-screening, analysis, and screening of new project proposals; and ranking, selecting, and on-going review of current projects. The strategic and high-level project selection criteria are determined by top management, but the ultimate decisions about whether to approve, deny, or cancel projects rest with the project review board. Technology innovation and R&D projects are usually selected from a pool of projects that are competing for funds, and are evaluated based on various criteria that traditionally imply commercial success. Projects are generally selected based on their expectation of meeting economic goals. There are various qualitative, quantitative, single, and multiple criteria methods for selecting projects in terms of benefit, cost, risk, resource requirements, and strategic objectives [19,20]. These models do not represent a logical or structured solution to the best selection of projects, and are unable to account for factors such as the maturity of the technology for commercialisation. Flaws in technical and commercial feasibility are often overlooked, and in some cases technical feasibility is ranked higher than commercial feasibility, which in turn increases risk and cost to the investor. Balachandra and Friar [10] proposed a contingency framework for the new product and R&D project models. The difficulty of obtaining a solution for selecting projects is exacerbated by the existence of a disproportionately large number of commercial objectives that often include profit, expected benefit, impact, and return on investment (ROI).

According to a best practice study conducted by Cooper et al. [21], it was found that 77.3 per cent of businesses make use of financial scoring methods, dominating 40.4 per cent of the businesses. Popularity does not necessarily equate with effectiveness, since most of these businesses fared poorly. Ironically, using the most rigorous techniques and a variety of financial tools yields the worst results. This is not because the methods are flawed, but because financial data are often missing at the very point where selection decisions are made. These methods yield an unbalanced portfolio of lower value projects, which in many instances lack strategic alignment. Therefore, scoring models that yield a balanced portfolio appear to be best for selecting high value projects. Best practice businesses tend to use multiple criteria to select their projects.

The uncertainty and risk associated with the success of a technology innovation project can pose a dilemma if the selection leads to an implementation or investment failure for an agency. The selection of high-impact, technologically- innovative projects for commercialisation and for possible new venture/SMME establishment are integral to ensuring the commercialisation and national rollout of these projects.

The objectives of this study were to establish (a) a basic framework of selection criteria that should be used for investment decision-making, and (b) gate review criteria that should be used at the development gates indicated in Figure 1 to improve the likelihood of the commercial success of technology innovation projects. The use of appropriate selection and gate review criteria could increase the commercial and long-term success rate for projects.

1.2 Project selection methodology for technological innovation

Technology innovation projects are selected for development and commercialisation in phases. But this does not guarantee that all new technology innovation products/services/processes projects will deliver the planned outcome of having a new product/service, of reaching the market, or of creating other expected benefits. The well-established project management methodologies of project phases and gates (Figure 1) form the basis from which an agency's specific methodologies are derived.

According to Steyn et al. [24], project phases typically include: (i) clarifying the need for the project; (ii) feasibility studies; (iii) definition or design and development; (iv) implementation or project execution; (v) handover to the client and project closure; and (vi) support and maintenance. Risk and uncertainty have implications for investment commitments, especially during the early stages. A certain number of company resources are invested in the different stages to manage a full range of activities that impact on the project outcomes of marketing and operational, technical, commercial, and financial feasibility [23].

A specific phase of a project is usually only authorised once there has been satisfactory progress in the previous phase and the detailed plans for the next phase to be worked on are well-planned and approved. A detailed plan for the next phase forms the baseline from which to work, as well as a summary of the succeeding stages or a full summary of the entire project. This is sometimes referred to as 'rolling wave planning' [25]. The decision points that precede every stage are called gates, which act as formal review points where specific criteria need to be met for approval for the project to continue. The criteria used can reoccur as the project progresses, and should be treated as the entry point to the next stage. Once the feasibility of the overall project has been assessed, the decision may be made.

The early stages of a project are fundamental to its success; 30 to 50 per cent of the emphasis of a project's life is typically given to the investigative stages, with far-reaching consequences for the remainder of the project [23]. At each stage the full range of work should be done to cover the entire scope of the functional inputs required, and all aspects (marketing, commercial, operational, and technical) of the project should be addressed in parallel. It is essential that forecasts and reforecasts of the benefits, resources, and costs required to complete the project are taken into account throughout each stage [23].

Expenditure and progress during the first period of the project is slow, but quickly accelerates once work gets underway. It eventually slows down towards the end of the project, leading to a slow-rapid-slow progress towards the project goal, forming a common S-shaped curve over the project life-cycle [24,26,27]. At the same time, the number of new and innovative products/services/processes decreases at each stage (Figure 2) as the ideas with the greatest potential for commercialisation are pursued [28]. The objective of each phase is to reduce the risk to the subsequent phases in a cost-effective manner [29]. During the appraisal of the completed phase, a decision can be taken to terminate a project if there are no feasible risk mitigation plans or steps to follow during the subsequent phases. The identification of potential risks for technology innovation projects, for both technical and commercial feasibility, could ensure that the risks are quantified, mitigated, and addressed well in advance during the life-cycle.

 

 

The criteria for results to be achieved at each gate, according to the project management literature, are established by the stakeholders, who take into account the deliverable(s) from the previous phase, a well-documented execution plan for the next phase, and an updated high-level plan for the remainder of the project. Based on these, resources and funding are approved or rejected for the next phase. If the specific criteria of a phase have not been met, mitigation plans should be in place and work for the next phase should not be started; the project might even have to be terminated. The project can progress to the next phase only once all criteria have been met, and detailed plans for the next phase (as well as an updated, overall, high-level plan for the rest of the project) have been accepted. Provided that the stakeholders accept the inherent risks, they might also decide to 'fast track' the project, by allowing it to proceed to the next phase without all the criteria being met. If the criteria have not been met, that has to be highlighted and documented, or the project could be terminated [24,29].

 

2 RESEARCH METHOD

The analytical network process (ANP) method to investigate the criteria for selecting the projects to obtain superior value for investors is adapted from a method used by Mohanty, Agarwal, Choudhury and Tiwari [30] and Meade and Presley [31]. Saaty [32] developed the ANP as another approach to selecting R&D projects, and is a generalisation of the analytic hierarchy process (AHP), one of the most widely-used multiple criteria decision-making methods. Where the AHP decomposes a problem into several levels, making up a hierarchy in which each decision element is supposed to be independent, the ANP extends the AHP to problems with dependence and feedback, and allows for more complex interrelationships among decision elements by replacing a hierarchy with a network [33]. The ANP network presents a relationship between decision elements and the relative weightings of each decision element.

The ANP framework consists of:

1. Different project phases:

Decision-makers have different views of the phases that influence the selection process, but of great importance are the various attributes and criteria used at each phase. The phases include: (i) proof of concept, (ii) development, (iii) product/ service/process prototype, (iv) technology and market validation, (v) market accepted product/service/process, (vi) market launch, and (vii) trading business. During the early phases of the project, the focus is mainly on the technological viability and attributes, whereas during the later phases, the commercial viability and attributes are of greater importance [30].

2. Different factors affecting decision-making:

2.1 Merit - Funding agencies consider projects that suit their needs or mandate and that provide the relevant expected benefits to be realised, taking into account the various inherent constraints. The merits of projects are assessed on a set of attributes that are classified into five categories:

1) project,

2) organisational,

3) technological,

4) market, and

5) environmental.

2.2 Risk - Risk in the technology innovation space is very high, and a level of comfort should be reached in order for such a project to be selected. The risks are classified into three categories:

1) technical,

2) economic, and

3) commercial.

2.3 Category - Due to the strategic needs and preferences of the agencies that are directed by their mandate, their inclination is towards certain categories of projects. The classification of the projects based on research is:

1) basic

2) fundamental, and

3) advanced.

The selected ANP model encompasses the attributes of technology innovation project selection.

Various criteria that are relevant to project selection, and are obtained mainly from the literature, were evaluated (Table 1). According to Mohanty et al. [30] and Meade and Presley [31], the objective of modelling the ANP framework is to select the project from the available options that would best support the realisation of the end goal. Gate review criteria were also identified for use at each gate.

 

 

It is important when doing research within a technology innovation project context that the methodology includes a multi-disciplinary approach. Considering technical and commercial feasibility within the technology innovation investment context in a developing market like South Africa, and to obtain consensus, the Delphi method was considered most appropriate by a panel of experts, according to Mullen's [34] guidelines. Beukman and Steyn [29] concur that, in situations where less information is available, consensus methods such as Delphi, focus groups, or brainstorming are called for. Although Delphi is a relatively simple technique, many studies have shown it to be an effective method [35, 36].

The participants selected as members of the expert panel had: (i) to be part of an organisation that supports technology innovation and commercialisation in South Africa, and (ii) to have experience in the fields of technology innovation, commercialisation, or start-up/establishment of new ventures/SMME, and in the selection of projects. These experts were identified, contacted individually, and personally requested to participate as expert panellists. They had to be available and willing to participate in both Delphi rounds. This was followed by discussions with some of these identified expert panellists on attributes and criteria over and above those identified from the literature (Table 1); these were then included in the study. From the identified attributes only the most applicable criteria from the secondary data were selected for use during a two-round Delphi study in order to collect the primary data. The identified attributes were used to identify the selection criteria relating to those factors, their weights, and gate review criteria during the Delphi study.

In the first-round of the Delphi survey the experts were asked to provide their biographical information and to rank and rate the critical factors that were provided (Table 1). The experts also identified and listed the criteria that were best suited for review at the gates between the project phases, and gave their opinions on the statements about the critical factors to identify the most widely-used criteria.

All seven experts who agreed to take part in the Delphi survey completed and returned their Delphi survey questionnaires. They had a combined industry experience of more than 120 years, with an average of more than 17 years per expert, and a total of 65 years experience with technology innovation and innovation projects/start-up of new ventures/SMMEs. Furthermore, they had 70 years' combined experience working with R&D, technology innovation, setting-up new ventures, and project selection. They had been involved jointly in the approval of projects, project selection, and portfolio management for more than 110 years. In addition, they had successfully commercialised or established more than 65 new ventures. The expert respondents therefore had significant experience with project selection for commercial success and new venture/SMME start-up required for the Delphi survey.

The statements on which no consensus was achieved in the first round were analysed and fed back to the panel during the second round of the Delphi survey. While the first survey asked the panel to rank and provide the respective weights for the identified selection criteria, in the second round the panel were provided with the results of the ranking and the gate review criteria, and had to indicate whether or not they agreed with the results. Three different options for the associated weights were provided from which to choose. The additional criteria suggested by the panel for consideration were included in the second round for analysis.

Only six of the seven respondents took part in the second survey. The questions on which no consensus was reached were regarded as less important project selection criteria, and were not used in the basic ANP framework. The gate review criteria with which the panel did not agree were removed. It is believed that the requirements set for a Delphi survey were satisfied, and that the results are a true reflection of the consensus reached.

 

3 RESULTS

The most important project selection and gate review criteria for selecting and reviewing technology innovation projects were identified. The Delphi panel identified fourteen statements about frequently-used selection criteria that should be considered when deciding to invest in technology innovation projects for commercial success:

i. The value proposition of the project in order to determine the value of solving the problem to the customer; a customer should be prepared to pay for the solution (product/service or process) that is to be developed.

ii. Competitors' size and the alternative products they offer should be identified, to estimate the degree of competitive intensity and barriers to entry, including the ease of adoption that might affect successful market entry.

iii. There should be a clear link between the milestones and deliverables and the specific expected outcomes of the project.

iv. There should be opportunity for market success.

v. The project should have a sustainable competitive advantage, such as intellectual property.

vi. The project team should show skills and competencies.

vii. The route to market and market penetration ability for the intended business should be measured.

viii. There should be a degree of intellectual property protection.

ix. Risks (technical, commercial, and developmental) should be controlled throughout the project life-cycle.

x. There should be opportunity for technical success.

xi. The technical risk of the product/service or process being developed should be assessed.

xii. Strategic commercial partnerships are an important factor.

xiii. Regulatory hurdles and challenges should be addressed early on - from project selection.

xiv. The project team's drive and determination and an entrepreneurial mind-set are crucial.

These criteria were used as the measures and criteria for the ANP framework (Table 2).

 

 

The most critical factors to consider during project selection/early project phases for a project to achieve commercialisation success are commercial risk and feasibility, followed by the team's skills and competence. No consensus was achieved about the third factor. This indicates that the experts have varying opinions on the importance of the third factor. The fact that the criteria differ between industries and individuals might be the reason for the limited selection criteria published for commercialisation success of this type of projects.

The three least important factors to consider during project selection/early project phases for a project to achieve commercialisation success were found to be product availability at launch of organisation, followed by investment management, and alignment with strategy as the third of the least important factors. These factors are therefore not as important for the selection of projects according to the experts, and do not have to be considered as selection criteria for technology innovative projects.

The weight of importance allocated to each of the critical factors from Table 1 was found to be within the minimum and maximum ranges, as indicated in Table 3. This confirms the results from the Delphi survey that commercial risk and feasibility is the most important factor, followed by team skills and competencies. These factors thus play a critical role as selection criteria for technology innovation projects.

 

 

When these selection criteria are considered, there is a greater possibility that commercial success will be achieved; but the project will need to be monitored and evaluated throughout its life-cycle. The new-product stage-gate system in Figure 1 was used to identify the review criteria to be used at each of the gates. The results for the gate for each respective gate from the Delphi survey are indicated in Table 4.

 

 

4 CONCLUSIONS AND RECOMMENDATIONS

There are various methodologies to select projects. Information on project selection criteria - specifically for technologically innovative projects aimed at commercial success and the possible establishment of an SMME - is, however, very limited. This study contributes to filling this gap by (i) revealing some insights into project selection criteria, their associated weightings, and gate review criteria, based on the experience of experts in the field of technology innovation and its realisation of commercial success; (ii) providing a basic foundation in the form of an ANP framework and gate review criteria to develop a project management methodology; (iii) providing a better understanding of what it takes to select technology innovation projects to ensure commercial success; and (iv) contributing to the bodies of knowledge on project management for technology innovation projects geared towards commercial success.

Recommendations about the selection criteria for technology innovation projects and their weightings in South Africa: (i) the selection criteria should be employed, together with the gate review criteria from which a proper pilot feasibility study should be conducted on funding and supporting agencies for technology innovation projects; (ii) monitoring and evaluating the pilot study should indicate whether the desired outcome (commercialisation and/or SMME establishment) was achieved; and (iii) clear guidelines should be developed for each relative effort.

Areas for further research include finding the exact weighting of the identified criteria, and developing a generic technology innovation stage-gate methodology that could be applied and used throughout the development of the projects. Case studies can be used to compare the actual methodologies used by different funding agencies, and to see what effect these different methodologies have on the commercial success or failure of technology innovation projects.

 

REFERENCES

[1] McFarlan, F.W. 1984. Information technology changes the way you compete. Harvard Business Review, 62, pp 98-103.         [ Links ]

[2] Porter, M.E. & Millar, V.E. 1985. How information gives you competitive advantage. Harvard Business Review, 63, pp 149-160.         [ Links ]

[3] Reddy, R. 1990. Technology and organizations: A technological perspective, Technology and organizations. Jossey-Bass, San Francisco, CA, pp 232-253.         [ Links ]

[4] Sproull, L.S. & Goodman, P.S. 1990. Technology and organizations: Integration and opportunities, Technology and organizations. Jossey-Bass, San Francisco, CA.         [ Links ]

[5] Dvir, D., Raz, T. & Shenhar, A.J. 2003. An empirical analysis of the relationship between project planning and project success. International Journal of Project Management, 21(2), pp 89-95.         [ Links ]

[6] Hemantha, S.B. & Park, C.S. 2002. Multi-stage capital investment opportunities as compound real options. The Engineering Economist, 47(1), pp 1-28.         [ Links ]

[7] Cooper, R.G. & Kleinschmidt, E.J. 1996. Winning business in product development: Critical success factors. Research Technology Management, 39, pp 18-29.         [ Links ]

[8] Rasmussen, E. 2008. Government instruments to support commercialisation of university research: Lessons from Canada. Technovation, 28, pp 506-517.         [ Links ]

[9] Rae, T. & Eden, C. 2002. On project success and failure in major engineering projects. EURAM.         [ Links ]

[10] Balachandra, R. & Friar, J.H. 1997. Factors for success in R&D projects and new product innovation: A contextual framework. IEEE Transactions on Engineering Management, 44 (3), pp 276-287.         [ Links ]

[11] Wang, J., Lin, W. & Huang, Y. 2010. A performance-oriented risk management framework for innovative R&D projects. Technovation, 30, pp 601-611.         [ Links ]

[12] Munns, A.K. & Bjeirmi, B.F. 1996. The role of project management in achieving project success. International Journal of Project Management, 14(2), pp 81-87.         [ Links ]

[13] Cooper, R.G. 1979. The dimensions of industrial new product success and failure. The Journal of Marketing, 43, pp 93-103.         [ Links ]

[14] Svensson, R. 2007. Commercialisation of patents and external financing during the R&D phase. Research Policy, 36, pp 1052-1069.         [ Links ]

[15] Cooper, R.G. & Kleinschmidt, E.J. 1991. New product processes at leading industrial firms. Industrial Marketing Management, 20(2), pp 137-147.         [ Links ]

[16] Liberatore, M.J. & Titus, G.J. 1983. Synthesizing R&D planning and business strategy: Some preliminary findings. R&D Management, 13(4), pp 207-216.         [ Links ]

[17] Mansfield, E. & Wagner, S. 1975. Organizational and strategic factors associated with probabilities of success in industrial R&D. The Journal of Business, 48, pp 179-198.         [ Links ]

[18] McGinnis, M.A. & Ackelsberg, M.R. 1983. Effective innovation management: Missing link in strategic planning? Journal of Business Strategy, 4(1), pp 59-66.         [ Links ]

[19] Nicholas, J.M. & Steyn, H. 2012. Project management for business, engineering, and technology: Principles and practice. 4^th edition. Routledge, New York and London.         [ Links ]

[20] Coffin, M.A. & Taylor, B.W. 1996. Multiple criteria R&D project selection and scheduling using fuzzy logic. Computers & Operations Research, 23(3), pp 207-220.         [ Links ]

[21] Cooper, R.G, Edgett, S. & Kleinschmidt, E. 2001. Portfolio management for a new product development: Results of an industry practices study. R&D Management, 31(4), pp 361-380.         [ Links ]

[22] Cooper, R.G. 1990. Stage-gate systems: A new tool for managing new products. Business Horizons, May-June, pp 44-54.         [ Links ]

[23] Buttrick, R. 2000. The project workout, 2^nd edition. Financial Times Management, London.         [ Links ]

[24] Steyn, H. (ed.) 2012. Project management: A multi-disciplinary approach, 3^rd edition. FPM Publishing, Pretoria.         [ Links ]

[25] Collyer, S. & Warren, C.M.J. 2009. Project management approaches for dynamic environments. International Journal of Project Management, 27, pp 355-364.         [ Links ]

[26] Meredith, J.R. & Mantle, S.J. 1995. Project management: A managerial approach. 3^rd edition. John Wiley and Sons, Inc., United States.         [ Links ]

[27] Bhuiyan, N. 2011. A framework for successful new product development. Journal of Industrial Engineering and Management, 4(4), pp 746-770.         [ Links ]

[28] Booz, Allen & Hamilton. 1982. New product management for the 1980s. Booz, Allen & Hamilton, Inc, New York.         [ Links ]

[29] Beukman, E. & Steyn, H. 2011. Phasing technology transfer projects for sustainable socio-economic development. South African Journal of Industrial Engineering, 22(2), pp 40-53.         [ Links ]

[30] Mohanty, R.P., Agarwal, R., Choudhury, A.K. & Tiwari, M.K. 2005. A fuzzy ANP-based approach to R&D project selection: A case study. International Journal of Production Research, 43(24), pp 5199-5216.         [ Links ]

[31] Meade, L.M. & Presley, A. 2002. R&D project selection using the analytic network process. IEE Transactions on Engineering Management, 49, pp 59-66.         [ Links ]

[32] Saaty, T.L. 1980. The analytical hierarchy process: Planning, priority setting, resource allocation. McGraw-Hill, New York.         [ Links ]

[33] Jung, U. & Seo, D. W. 2010. An ANP approach for R&D project evaluation based on interdependencies between research objectives and evaluation criteria. Decision Support Systems, 49, pp 335-342.         [ Links ]

[34] Mullen, P.M. 2003. Delphi: Myths and reality. Journal of Healthy Organization and Management, 17(1), pp 37-52.         [ Links ]

[35] Turoff, M. 1970.The design of a policy Delphi. Technological Forecasting and Social Change, 2, pp 149-171.         [ Links ]

[36] Barry, M.-L., Steyn, H. & Brent, A. 2008. Determining the most important factors for sustainable energy technology selection in Africa: Application of the Delphi technique, IAMOT.         [ Links ]

[37] Yap, C.M. & Souder, W.E. 1994. Factors influencing new product success and failure in small entrepreneurial high-technology electronics firms. Journal of Product Innovation Management, 11, pp 418-432.         [ Links ]

[38] Cooper, R.G. 1994. New Products: The factors that drive success. International Marketing Review, 11(1), pp 60-76.         [ Links ]

[39] Cooper, R.G. & Kleinschmidt, E.J. 2001.Stage-gate process for new product success. Innovation Management U3, pp 1 -9.         [ Links ]

[40] O'Connor, G.C. & Veryzer, R.W. 2001. The nature of market visioning for technology-based radical innovation. The Journal of Product Innovation Management, 18, pp 231-246.         [ Links ]

[41] Cooper, R.G. 2008. Perspective: The stage-gate idea-to-launch process-update: What's new, and NexGen system. The Journal of Product Innovation Management, 25, pp 213-232.         [ Links ]

[42] Hart, S., Hultink, E.J., Tzokas, N. & Commandeur, H.R. 2003. Industrial companies' evaluation criteria in new product development gates. The Journal of Product Innovation Management, 23, pp 22-36.         [ Links ]

[43] Coldrick, S., Longhurst, P., Ivey, P. & Hannis, J. 2005. An R&D options selection model for investment decisions. Technovation, 25, pp 185-193.         [ Links ]

[44] Cooper, R.G. & Kleinschmidt, E.J. 1993. Screening new products for potential winners. Long Range Planning, 26(6), pp 74-81.         [ Links ]

[45] Shenhar, A.J., Dvir, D., Lipovetsky, S. & Lechler, T. 2002. Refining the search for project success factors: A multivariate, typological approach. R&D Management, 32(2), pp 111-126.         [ Links ]

[46] Fortune, J. & White, D. 2006. Framing of critical success factors by a systems model. International Journal of Project Management, 24, pp 53-65.         [ Links ]

[47] Belassi, W. & Tukel, O.I. 1996. A new framework for determining critical success/failure factors in projects. International Journal of Project Management, 14(3), pp 141-151.         [ Links ]

[48] Sohn, S.Y. & Moon, T.H. 2003. Structural equation model for predicting technology commercialisation success index (TCSI). Technology Forecasting and Social Change, 70, pp 885-899.         [ Links ]

[49] Cooper, R.G., Edgett, S.J. & Kleinschmidt, E.J. 1999. New product portfolio management: Practice and performance. The Journal of Product Innovation Management, 16, pp 333-351.         [ Links ]

[50] Palmberg, C. 2006. The source and success of innovation - Determinants of commercialisation and break-even times. Technovation, 26, pp 1253-1267.         [ Links ]

[51] Ahn, M.J., Zwikael, O. & Bednarek, R. 2010. Technological innovation to product innovation: A project management approach. International Journal of Project Management, 28, pp 559-568.         [ Links ]

[52] Danneels, E., & Kleinschmidt, E.J. 2001. Product innovativeness from the firm's perspective: Its dimensions and their relation with project selection and performance. The Journal of Product Innovation Management, 18, pp 357-373.         [ Links ]

[53] Lim, C.S., & Mohamed, M.Z. 1999. Criteria for project success: An exploratory re-examination. International Journal of Project Management, 17(4), pp 243-248.         [ Links ]

[54] Iyer, K.C., & Jha, K.N. 2005. Factors affecting cost performance: Evidence from Indian construction projects. International Journal of Project Management, 23, pp 283-295.         [ Links ]

[55] Carbonell-Foulquie, P., Munuera-Aleman, J.L. & Rodriguez-Escudero, A.I. 2004. Criteria employed for go/no-go decisions when developing successful highly innovative products. Industrial Marketing Management, 33, pp 307-316.         [ Links ]

 

 

* Corresponding author.
1 At the time this research was conducted, this author was enrolled for the MSc (Project Management) degree at the Graduate School of Technology Management, University of Pretoria.