SciELO - Scientific Electronic Library Online

 
vol.29 número1Impact of implementation of total quality management: An assessment of the Saudi IndustryTesting flexible grippers for geometric and surface grasping conformity in reconfigurable assembly systems índice de autoresíndice de assuntospesquisa de artigos
Home Pagelista alfabética de periódicos  

Serviços Personalizados

Artigo

Indicadores

Links relacionados

  • Em processo de indexaçãoCitado por Google
  • Em processo de indexaçãoSimilares em Google

Compartilhar


South African Journal of Industrial Engineering

versão On-line ISSN 2224-7890
versão impressa ISSN 1012-277X

S. Afr. J. Ind. Eng. vol.29 no.1 Pretoria  2018

http://dx.doi.org/10.7166/29-1-1675 

GENERAL ARTICLE

 

Project management assessments (PMAs): An empirical study

 

 

R. Siriram*

School of Mechanical, Industrial and Aeronautical Engineering, University of the Witwatersrand, Johannesburg, South Africa

 

 


ABSTRACT

The objectives of this research are to: a) compare project management assessments (PMAs) from four firms across four industry sectors (75 PMAs are evaluated using known statistical techniques); b) validate the results through an expert panel; c) apply Pearson's correlation analysis to find links in the PMA constructs and see how these links relate to the overall project result; and d) identify areas for further research. Three hypotheses are tested, and reveal differences and similarities in project management practices. The results provide interesting opportunities for researchers and for project management practitioners. Finally, a discussion, the limitations of the study, and suggestions for further research are presented.


OPSOMMING

Die doelwitte van hierdie navorsing is eerstens om projekbestuur-assesserings van vier firmas in vier industriesektore te vergelyk (vyf-en-sewentig projekbestuurassesserings is evalueer met bestaande statistiese ontledingstegnieke). Tweedens is die doel om die resultate deur 'n kennerpaneel te valideer en derdens om Pearson se korrelasie analise toe te pas om ooreenstemminge in die projek-bestuurassesserings op te spoor en om te sien hoe hierdie dit verband hou met die algehele projek uitslag. Laastens word areas van verdere navorsing identifiseer. Drie hipotese is getoets en toon ooreenkomste en verskille in projekbestuurpraktyke. Die resultate lewer interessante geleenthede vir navorsers en projekbestuurders. Ten slotte word 'n bespreking, die beperkings van die studie en voorstelle vir verdere navorsing voorgehou.


 

 

1 INTRODUCTION

The aim of this research is to make a contribution to knowledge by providing empirical evidence on project management assessments (PMAs). Seventy-five PMAs are used to compare project management practices between four firms and four industry sectors. The research is significant in that it shows that there is a need for an industry-wide project management assessment (PMA) tool that can be used by project management practitioners. The results of the research show similarities and differences between industry sectors. From the four industry sectors it is shown that the electronics and electrical engineering, and information and communication technology (ICT ) industries show significant differences in their means for the overall project result, whereas the services and consulting engineering industries show no significant differences between the means for the overall project result. Comparisons are also drawn between the different PMA constructs. To understand the need for empirical PMA research better, the need for project management needs to be understood better.

Given the rapid pace of business evolution, firms are expected to become increasingly competitive. Siriram [1] states that old ways of doing business are becoming obsolete, and that the rules of the game change at a bewildering rate. Some firms (buyers) expect firms (sellers) to become more competitive than other firms (other suppliers) in the production of goods and services. Siriram [2] says that firms are in a race to be better, or they are at risk of becoming less competitive. Firms need to develop their capabilities to outsmart other firms. Some firms are project-driven, and in these firms project management is seen as a capability to develop competitive advantage. Grant [3] sees the firm's capabilities as resources needed to develop competitive advantage, which is necessary to out-smart competitors. It is important that firms that are project-driven see project management as an integral part of their business strategy. Srivannaboon and Milosevic [4] say that many firms see project management as a building block for strategy, and that aligning project management and strategy may help firms to become more competitive. Therefore, one may see project management as a capability needed for competitive advantage.

To develop project management as a capability, researchers such as Fringsdorf, Zuo and Xia [5] identify critical success factors for project efficiency; they say that these factors need to be developed to ensure successful project outcomes. They add that these factors are not limited to individual project factors, but extend to include project interdependencies and external customers. Maylor, Brady, Cook-Davies and Hodgson [6] argue that, for the past decade, firms have been changing their competitive strategy from an operational strategy to a project management strategy; more recently, Maylor, Turner and Murray-Webster [7] show how operational strategies may be adapted in certain cases as project-based operations. Moreover, Thiry and Deguire [8] speak of project-based firms that can shape or re-shape strategies. Others, such as Canonico and Soderlund [9], say that more firms are being managed through projects, and that it is difficult to imagine a firm that is not managed through some kind of project activity. This is also shown by Kujala, Artto, Aaltonen and Turkulainen [10], who point out that business models in project-based firms focus on service delivery and integrated solutions across the project life cycle. In further support of this idea, Wikstroöm, Artto, Kujala and Söderlund [11] show the diversity in business models for project-based firms. Buys and Stander [12] highlight the need to link projects to business strategy. On the other hand, Meskendahl [13] suggests that business strategy describes the way in which a firm opts to compete in the market compared with its competitors. Based on this research evidence, it may be concluded that developing project management as a capability is necessary to drive competitive advantage in project-based firms.

Given the need to link business strategy to projects and to competitive advantage, Young, Young, Jordaan and O'Connor. [14] point out deficiencies in the way projects are currently selected and managed, thereby limiting the capability to realise strategic goals. Young and Grant [15] also found that projects contribute little to strategic goals; other than in some stable environments, projects have made some contribution to strategic goals. Given that project management can shape and reshape a firm's strategies [8], firms are looking for integrated business models [10] and the complexity that arises from the diversity of business models [11], not to mention the deficiencies in project execution [14]. It may be stated that project management is an underdeveloped capability, and more investigation and attention needs to be devoted to project management. To develop project management as a capability, some researchers, such as Andersen and Jessen [16], see project maturity as important. Golini, Kalchschmidt and Landoni [17] define project management maturity as a firm's capabilities in relation to the project management process (management of time, scope, quality, etc.). Given the importance of project management maturity, Mittermaier and Steyn [18] see the need for project management maturity and assessment; and Cao and Hoffman [19] speak of assessing project performance and evaluation to achieve project management maturity. Grant and Pennypacker [20] argue that the project management community is actively involved in developing methods to assess and improve project management maturity. Tahri and Drissi-Kaitouni [21] argue that there is generally no agreed definition of what a mature project organisation looks like, and therefore a growing number of maturity models are being developed to assess project management maturity. Furthermore, Grant and Pennypacker [20] argue that there is little research evidence supporting empirical research investigating project management assessments. Therefore, based on these observations, it would be beneficial to develop a tool to assess the maturity of project management within organisations. Pennypacker and Grant [22] point out that assessment tools allow an organisation to assess and compare its own practices against best practices, or against those employed by competitors, with the intention to map out a structured path to improvement. Further support of this is given by Qureshi, Warraich and Hijazi [23], who argue that good project management assessment tools are required to make organisations the 'best of the best'. Therefore, in this research paper, drawing on empirical research, a project management assessment (PMA) tool is proposed. Clearly, project management is a capability that needs to be further developed. In this paper, research on project management assessments (PMAs) is conducted, as PMAs are the basis for project management maturity. PMA can therefore be defined as a tool that an organisation can use to assess its project management maturity, comparing its own practices against best practices with the intention of mapping out a path to improvement.

PMA data was collected from four firms and four industry sectors: the electronics and electrical engineering industry, the information and communication technology (ICT) industry, the consulting engineering industry, and the service industry. This paper makes a contribution to knowledge by meeting the following objectives:

a) Carrying out empirical research to compare project management practices using 75 project management assessment (PMAs), involving four firms across four industry sectors to obtain a better understanding of the similarities and differences using known statistical techniques.

b) Validating the results through an expert panel.

c) Using Pearson's correlation analysis to find links in the PMA constructs, and seeing how these links relate to the overall project result.

d) Identifying areas for further research.

The research draws on the experiences of an expert panel to validate the results of the empirical analysis, and provides important insights for researchers and project management practitioners.

 

2 LITERATURE SURVEY AND HYPOTHESIS DEVELOPMENT

Many firms have reported having poor project management results. Miller and Lessard [24] refer to 60 large-scale projects with an average capital value of US$ 1 billion undertaken between 1980 and 2000; 18 per cent of them incurred extensive cost overruns. They also showed that almost 40 per cent of the projects performed so badly they were totally abandoned or were restricted after experiencing financial crises. Morris and Hough [25] also provide a comprehensive list of cost overruns on large complex projects (LCPs). In further support, Merrow, McDonnell and Arguden [26] studied 47 LCPs and found that four finished on budget, with an average cost overrun of 88 per cent, and that 26 (72 per cent) failed to achieve their profit objectives.

According to an article by Evans [27] in the Economist (June 2005) entitled 'Project management, overdue and over budget, over and over again', many firms have reported on overdue and overbudget projects. The article reflects on project inadequacies, from silo approaches to shortcomings in project phases (i.e., initiation, planning, execution, control, and closure). The article quotes the Journal of the American Planning Association, examining 210 big rail and road projects in 14 countries, and referring to poor results being attributed to inaccurate demand forecasts and the importance of project management being ignored. More recently, Shehu, Endut, Akintoye and Holt [28], in a study of 359 projects, also point out that 55 per cent of Malaysian projects experienced cost overruns. It is clear that projects are burdened with poor project outcomes, and that further attention is required.

Even though much research has been done on project management, many projects have poor outcomes. It is therefore important to see what research has been done. Covering the full extent of project management research would be impossible; however, it is important to look at some of the research evidence in the project management body of knowledge, such as the following research on project management: Kloppenborg and Opfer [29] on the current state of project management; Grundy and Brown [30] on strategic project management; Crawford and Bryce [31] on project monitoring and evaluation; Kadefors [32] on trust in project relationships; Cheung, Wong, Fung, and Coffey. [33] on predicting project performance; Hyväri [34] on project management effectiveness; Shenhar and Dvir [35] on the challenges and opportunities in project management; Kolltveit, Karlsen and Gronhaug [36] on different perspectives on project management, including leadership, task, and business perspectives; Kwak and Smith [37] on project risk management; Blichfeldt and Eskerod [38] on project portfolio management; Aubry, Muller, Hobb and Blomquist [39] on project management offices; Choi, Chung and Lee [40] on risk perception analysis; Shi [41] on rethinking implementation of project management; Yang, Huang and Wu [42] on project manager leadership styles, team work, and project success; Cao and Hoffman [19] on project performance evaluation systems; Davies [43] on stakeholders and their perceptions of project success; and Pinto [44] on project management governance and the normalisation of deviance. More recently, Fernandes, Ward and Araújo [45] give further support to improve and embed project management practices in organisations.

Further research, specifically of factors affecting poor project outcomes and how to mitigate those outcomes, is given by the following researchers, among others: Baker and Fisher [46], identifying factors affecting project success; Atkinson [47] on the project management focus on cost, time, and quality; Chapman and Ward [48] on risk efficiency for best practice projects; Chapman [49] on risk and uncertainty management; Atkinson, Crawford and Ward [50] on fundamental uncertainties in projects and the scope of project management; Braimah and Ndekugri [51] on factors in delay analysis; Ahsan and Gunawan [52] on cost and schedule performance; Van Os, Berkel, Gilder, Dyck and Groenewegen. [53] on project risk; and Hazir [54] on analytical models and decision-support tools for project monitoring and control. From the research evidence it is clear that poor project outcomes are a concern, and mitigation of poor project outcomes is needed. The following authors, among others, provide some guidance: Crawford [55] suggests that project management maturity models provide a path to project management excellence; Jugdev and Thomas [56] see project management maturity models as the silver bullets for competitive advantage; Crawford and Bryce [31] propose project monitoring and evaluation as a method to enhance the efficiency and effectiveness of project implementation; Jaafari [57] sees project and programme diagnostics as a systematic approach to project management maturity and evaluation; Mittermaier and Steyn [18] focus on project management maturity models; Qureshi, Warraich and Hijazi [23] propose project management assessment models; and Guangshe, Yuting, Xiangdong, Jianguo, Jiming, and Kewei. [58] also support project organisational maturity. Clearly there is a need for some investigation into project management assessments (PMAs), which may be seen as the basis for project management maturity and the path to project management excellence.

Notwithstanding the importance of PMAs, little empirical work has been done on them. Some empirical work has been done by the following researchers, among others: Ika, Diallo and Thuiller [59] on critical success factors for world bank projects; Teller and Kock [60] on project risk management and its influence on project portfolio success; Mir and Pinnington [61] on evidence linking project management performance and project success; Basu [62] on quality in projects; and Xu and Yeh [63] on a performance-based approach to project assignment and performance evaluation. Clearly further empirical work on PMAs will contribute to the project management body of knowledge.

The focus of this paper is on further empirical research in project management assessments (PMAs). The focus is on collecting PMA data to conduct empirical research. Seventy-five projects from four firms across four industry sectors are studied. Given the importance of project management maturity, the role of PMAs in achieving project management maturity, and the need to improve and embed project management practices, further empirical evidence on PMAs specifically may assist researchers and practitioners in the study of project management. Using the PMA data, comparisons are drawn using known statistical techniques. The following three hypotheses are formed:

1. H1: Hypothesis one:

a) Null hypothesis: H0, there are no significant differences between means for PMA constructs within each industry sector.

b) Alternative hypothesis: H1, there are significant differences between means for PMA constructs within each industry sector.

2. H2: Hypothesis two:

a) Null hypothesis: H0, there are no significant differences between variances for PMA constructs within each industry sector.

b) Alternative hypothesis: H1, there are significant differences between variances for PMA constructs within each industry sector.

3. H3: Hypothesis three:

a) Null hypothesis: H0, there are no significant differences between means for the overall PMA project result within each industry sector.

b) Alternative hypothesis: H1, there are significant differences between means for the overall PMA project result within each sector.

This research also lays a basis for further research in using PMAs as a tool that may be used to develop project management as a capability. The research methodology is discussed next.

 

3 RESEARCH METHODOLOGY

3.1 Data collection and sample

The research methodology consisted of a seven-step process, depicted in Figure 1. Each of the steps in Figure 1 is discussed next.

 

 

1. Step [1]: Identification of firms participating in the research. The firms were chosen specifically for their willingness to participate in the research. It proved to be difficult to encourage firms to participate in this research, as the research includes an in-depth analysis of the firm's project management practices and results; and some were not willing to divulge project-specific information. However, the author was able to elicit the support of four firms to participate in the research. The four firms studied operate in the electronics and electrical engineering industry; the information and communication technology industry; the service industry; and the consulting engineering industry. The firms opted to be anonymous contributors to the research; therefore, for the purposes of this research, the firms are identified as Alpha (α), Beta (β), Gamma (γ), and Phi (Φ). The demographics of the firms are shown in Table 1.

2. Step [2]: Collection of project management assessments (PMAs). Seventy-five PMAs were collected across four firms. The PMAs were in-house project assessments that each firm used to evaluate project management maturity. The PMAs may also been seen as an indication of the firm's project management practices. It was not the intention of this research to evaluate the PMA tools or to develop new PMAs. Rather, the scope of this research was to compare the results of the existing PMAs used by the firms. (See the section under 'Further research' to develop PMA tools.) The PMA tools across the four firms were different; therefore the PMA results had to be normalised to enable comparisons. Normalisation was done using key words and phrases, following which each of the constructs was grouped into three constructs: a) project organisation, b) risk and variation, and c) project planning, engineering design, and quality.

For this research paper, the following descriptors are used for each of the constructs:

a) Project organisation - PMBOK [64] defines project organisation as the human infrastructure of the project that includes the project organisation chart and the roles and relationships of the project team. For this paper, the definition is expanded to include project ownership (project manager agreement and project organisation structure), behaviour of executives (executive involvement and support), financial control (project monitoring and control), and project structure (e.g. human infrastructure).

b) Risk and variation - PMBOK [64] defines 'risk' as the process concerned with risk management planning, identification, analysis, responses, and monitoring and control of a project. For this paper, we expand the definition to include the risk and variation that the project needed to have a proper process in place to cover risk management, change and claims management, involvement of stakeholders (customers, employees, and other stakeholders) to mitigate risk, a proper project close-out process to mitigate risks associated with project creep, and project compensation for the project team.

c) Project planning, engineering design, and quality - PMBOK [64] defines project planning as a formal set of approved documents used to guide both project execution and project control. In this paper, project control includes project quality plans and engineering design. For this construct, the following is included: a project plan (bar or network chart, work packages, dependence analysis, etc.), a project quality plan, and the engineering design, which includes a consulting and an innovation component.

The normalisation process (step 4) was necessary, initially to set up the three constructs, and then to compare them. The three constructs consisted of 14 subscales. Grouping the subscales into each of the PMA constructs was done through the normalisation process - this was achieved by all eight executives in the expert panel agreeing to the grouping; when there was no agreement, the subscale was dropped from the analysis. The grouping of the PMA constructs and the subscales are shown in Table 2.

3. Step [3]: Setting up the expert panel. Expert guidance in project management was required to assist with the normalisation process. Two executives from the project management area in each firm were selected. They had more than ten years' experience in project management, and held leadership positions within their firms. The experts also had to be knowledgeable about the project management practices, policies, and guidelines within their firms, and knowledgeable about project management as a discipline.

4. Step [4]: Normalisation of the PMA results. This involved normalising the PMA results across the four firms; this was accomplished through in-depth interviews with six executives from the participating firms. The expert panel reviewed the PMA tools from the different firms and, by using key words and phrases from each of the PMAs, a common template was developed. The PMA results were then used as the input into this template, and analysed. The PMA template consisted of a five-point Likert scale, where '1' was rated as the lowest level of implementation, and '5' was rated as a world class level of implementation.

5. Step [5]: Statistical analysis. This was performed on the normalised PMA results, using SPSS version 20.0.

6. Step [6]: Expert panel review of the statistical analysis. The results were reviewed during two three-hour round-table discussions held with the expert panel.

7. Step [7]: Report on research findings. The research results are discussed next.

 

4 DISCUSSION OF RESULTS

4.1 Measurement model

The data consisted of a sample size of 75 PMAs. After the normalisation process, a PMA template was created, resulting in three categories: a) project organisation, b) risk and variation, and c) project planning, engineering design, and quality. This consisted of 14 subscales in total. The constructs are shown in Table 2. Cronbach's alpha tests were performed for the 75 PMA results to ensure that they were correctly categorised into the three constructs. The reliability tests are shown in Table 3.

The values of Cronbach's alpha for a) project organisation, b) risk and variation, and c) project planning, engineering design, and quality are 0.711, 0.735, and 0.749 respectively, all of which are greater than or equal to the recommended level of 0.70 [65]. Others, such as Siriram and Snaddon [66], have used values above 0.6. In addition, item-total correlations are high, indicating that they have high internal consistency to minimise the contribution of random error. There is also very little difference if items are deleted, showing a good level of internal consistency.

Reliability and validity were assessed as follows: reliability was assessed by the coefficient alphas; and the factor structure and the measurement quality were assessed by exploratory factor analysis. Tables 3 and 4 show the results for reliability and factor analysis respectively.

To understand the factor structure and measurement quality, a principal component analysis with Varimax rotation was conducted, with the results shown in Table 4. To identify the number of factors to retain, an evaluation of the eigenvalues was used. Throughout this process, all items were loaded into three different factors. Three factors in which eigenvalues were greater than one (1) represent a) project organisation; b) risk and variation; and c) project planning, engineering design, and quality. Furthermore, the percentage of explained variance by the three factors is 28.86 per cent, 20.87 per cent, and 18.19 per cent respectively. And all the factor loadings are above the acceptance level of 0.50 [67]. These results therefore indicate the uni-dimensionality of the three constructs.

Because estimation methods are easily affected by the distribution variables, the descriptive statistics of the variables were also assessed. All variables were within the acceptance levels, indicating that the data is normally distributed, Table 5 (Shapiro Wilk Sig. values > 0.05).

4.2 The research hypotheses

Table 6 shows the results for the three hypotheses.

1. H1: Hypothesis one: For the differences between means for the PMA constructs in all four industry sectors (electronics and electrical, information and communications technology, services, and consulting engineering) all show no significant differences between the means for the three PMA constructs - i.e. a) project organisation, b) risk and variation, and c) project planning, engineering design, and quality. Therefore, the null hypothesis is fail to reject i.e. H0: There are no significant differences between the means (P (2-tailed values < 0.05). Table 6 also shows whether the results for the PMA constructs are significant or not significant.

2. H2: Hypothesis two: For the differences in variances for the PMA constructs, ANOVA testing will be used. For ANOVA F tests for F values > F critical value, the null hypothesis is rejected. ANOVA tests were performed at a 0.05 level of significance. For the information and communication and services industries, H0 was rejected. In the two other sectors (electrical and electronics, and consulting engineering), H0 varied and either failed to be rejected or was rejected; details are given in Table 6. Lavene's test statistics for homogeneity of variances and the ANOVA F tests for the PMA constructs - a) project organisation, b) risk and variation, and d) project planning, engineering design, and quality - are also given in Table 6. For the Lavene's test statistics, significant values < 0.05, the null hypothesis is rejected (Lavene's significant values in the third column of Table 6 are used).

3. H3: Hypothesis three: For the differences in the overall PMA project result, the hypothesis for the electronics and electrical and the information and communications technology industries both show significant differences between the means for the projects; thus the null hypothesis is rejected i.e. H0: There are no significant differences between the means (P 2 tailed values < 0.05). In the services and consulting engineering industry, no significant differences are found between the means; thus the null hypothesis cannot be rejected (P 2 tailed values > 0.05). Table 6 also shows whether the results for the PMA constructs - a) project organisation, b) risk and variation, and c) project planning, engineering design, and quality - are significant or not significant.

4.3 The expert panel results

Once the statistical tests had been completed, a round table discussion was held with the expert panel. This involved two sessions of three hours each - a total of six hours. The sessions were held in one week, on two alternate days. Comments from the expert panel about the hypotheses were as follows:

1. H1: For the differences between the means for the PMA constructs, the expert panel felt that the fact that there were no significant differences in the results was an indication of bias in the way that the PMAs were administered, because all of the PMAs were administered by the project teams themselves. PMAs were used as a self-assessment tool. They also felt that in future the PMAs should be administered independently. However, they did comment that project management in the electronics and electrical and the consulting engineering industries was more mature in implementation, and that therefore this result was not surprising.

2. H2: For the differences in variances in the information and communication and the services industries, these were significant. Here the expert panel felt that this was because project management practices were not being well embedded in these two industries. Project management was seen more as a coordination and escalation function. Regarding the electronics and electrical engineering and the consulting engineering industries, and the possible reasons for the differences in the variances, the expert panel felt that the implementation of project management was more mature in some areas and weaker in other areas; hence the inconsistency in the results.

3. H3: For differences in the overall project result, the electronics and electrical and the information and communications technology industries show significant differences. Here the expert panel felt that, for the electrical and electronics industry, this was due to some poor projects and some good projects, since the project management was at a more mature level.

For the information and communications technology industry, they felt that project management was not well embedded and was seen more as a coordination function; this finding is consistent with the H2 differences in variances. For the services and consulting industry, no significant differences were found. For the consulting engineering industry, the expert panel felt that project management was at a more mature level of implementation, and that the results were consistent with H2. Regarding the services industry, for which the expert panel felt that project management was still at a low level of maturity, there were no differences in means because of bias; this is consistent with significant differences in variances in the services industry. They also indicated that the services industry was not really functioning as a project management organisation, which is reflected in the low level of maturity.

4. In addition, the expert panel felt that the PMAs were limited in scope; they pointed out that the following areas, at least, should be included in further PMAs:

a) Project categorisation - e.g., A, B, C class projects, where A is high in complexity and involves long implementation periods (say, three years or more). B is medium levels of complexity and requires one to two years; and C is low levels of complexity and a duration of less than eight months. However, this will vary per industry sector. The important point is that some level of project categorisation exists that may be used to determine the level of PMA (A, B, or C) to be implemented.

b) Involvement of executive management in high-level projects. Executives can provide leadership and guidance, especially in A and B projects. Therefore executive involvement is important.

c) The handover from sales to project teams. The expert panel pointed out that a poor handover from sales teams may lead to scope creep, and is an area that needed further attention. Another point is project manager agreement: does the project manager agree that he/she can deliver this project as it was sold to the buyer? This is an important point that needs to be noted upfront. Also of importance is handover gates to other functional areas - e.g., engineering, procurement, and quality - that need to be noted in the project structure plan below.

d) The project structure plan consists of at least the following: a proper project plan (e.g., bar chart, network diagram, critical path analysis) that is a realistic representation of the project; clearly separated work packages (handover points, as in c) above); clear lines of communication; and an accurate representation of the stakeholders.

e) Resource planning to include a proper dependence analysis between the different work packages, allocation of the necessary resources (material, personnel, and financial) to the work packages, and definition of the project plan into sensible phases with clear delivery milestones.

f) Trust in project relationships is important. Often team members are aware of risks, but - for fear of failure if they disclose the risks - they remain silent. Therefore, trust in project teams should also be measured. The involvement of the relevant stake holders, such as customer groups and the procurement and supply chain, have different perspectives; so their involvement in project activity is important, and will reduce the risk of untrustworthiness.

5. The expert panel felt that further governance and support was required to ensure that the PMA tool received the correct attention from an executive standpoint. They also felt that the implementation of the PMA should be audited by an independent body, such as quality management or internal audit.

6. The expert panel supported the view of developing an industry 'standard' for a PMA tool that can be developed and used. They felt that merely having and implementing the PMA would not be sufficient; it was more important to have the PMA with the correct constructs, leading to more beneficial implementation. Therefore, an industry 'standard' for a PMA will be advantageous.

Table 7 gives additional comments from the expert panel. (Only the comments for hypothesis one are given because the comments for the other two hypotheses repeated those for the first.)

In summary, the expert panel felt that, while PMAs were implemented in firms and provided valuable insight, further detail for the diversity of PMA constructs is still required. They also felt that, if more constructs (like points 4a to f above) had been included, the results might have been different. Therefore, it is concluded from this analysis that the PMA tools being used by firms are limited and have shortcomings, and that PMA tools need to be further improved. However, PMAs are an important part of project management maturity, which needs to be further investigated.

To find out whether there were any correlations between the four construct categories, a multi-correlation analysis using Pearson's correlation was conducted next.

4.4 Multiple regression analysis

The results of the multiple regression analysis are shown in Table 8. The model is a good fit, with an R-squared value of 0.851 and an adjusted R squared valued of 0.845. ANOVA sum of squares gives the total (sum of squares) as 76.667. The F change is also quite high at 135.060. In terms of the collinearity statistics, both the tolerance values and the variance inflation factors (VIF) are low, which means a lack of collinearity. The VIF values are below the threshold value of 10 [67]. The evidence for multicollinearity is evident in a) the principal component analysis in Table 4, which shows a better model fit; and in b) Pearson's correlation, where an indication of correlation between dependent and independent variables is shown in Figure 3. In addition, the condition indices are low, indicating a low level of collinearity.

4.5 Pearson's correlation analysis

To show the links between the different PMA constructs, a correlation analysis using Pearson's correlation was also performed (see Figure 2). Pearson's correlation analysis shows that all correlations are significant at the 0.01 level of significance. The correlation diagram reflects the fact that all of the categories - a) project planning and organisation, b) risk and variation, and c) project planning, engineering design, and quality - affect the project result. Poor performance in any of the PMA constructs is likely to affect the overall project result.

 

 

5 LIMITATIONS OF THE RESEARCH

The research was limited to four firms across four industry sectors and 75 projects. It would be beneficial to add more firms and projects to the analysis. However, obtaining such in-depth project information is problematical because most firms are not willing to disclose such detailed information. Given the same data set, it would be beneficial to feed these results back to the firms studied, and then monitor the results in a few years' time to validate and understand any further improvements and maturity in project management practices.

As PMA assessments were done at a particular time (a 'snapshot'), it would be interesting to obtain PMA assessments through the life cycle of the project and to compare the results at different phases. While the research participants included project management personnel, it would be beneficial to include other stakeholders in the research, such as product managers, marketing managers, engineering managers, and service managers. Including customer viewpoints would also be interesting.

A further limitation of the research was the use of existing PMAs, as these had too few constructs - i.e., a) project planning and organisation, b) risk and variation, and c) project scheduling, engineering design, and quality. Other constructs could also be included, as suggested by the expert panel in section 4.3.

 

6 FURTHER RESEARCH

This research has made a contribution to the knowledge that shows that differences in project management practices exist across different industry sectors. It would be interesting to find out why there are significant differences in the electronics and electrical engineering and the information and communication technology industries, while there are no significant differences in the service and consulting engineering industries. There is also an opportunity to develop a 'standard' PMA tool that can be used to measure project management practices across different industry sectors.

The research was conducted with four firms within the electronics and electrical engineering, ICT, services, and consulting engineering industries. Opportunities exist for further research by increasing the number of projects within these industries to allow more robust statistical testing of larger and more diverse sample sizes, which may allow other statistical testing methods such as structural equation modelling (SEM). Finally, additional research could investigate the relationship between project management and operations management practices, as this was an issue raised in the services industry. A possible question to be asked is: Is the service industry more suited to a project-based or an operations-based organisation?

 

7 CONCLUSION

In terms of the objectives of the research, the following criteria have been satisfied:

1. Using empirical data, 75 PMAs across four firms and four industry sectors were analysed, showing that differences and similarities exist within and between industry sectors.

2. The results were validated by an expert panel.

3. Opportunities for further research have been identified.

4. Pearson's correlation analysis was used to identify links in the PMAs and to show how the different constructs relate to the overall project result.

One of the major shortcomings pointed out by the expert panel was that the PMAs were limited in scope, and that they need to be expanded to include other import constructs, as mentioned in section 4.3. From the research conducted, it may be concluded that further empirical research is still required in respect of project management maturity and evaluation. PMAs are one way of getting a better understanding in this area, and require further investigation. A 'standard' industry

PMA will add value to both researchers and practitioners in project management. The PMA may be used as an instrument to measure project management maturity.

 

REFERENCES

[1] Siriram, R. & Snaddon, D.R. 2004. Linking technology management, transaction processes and governance structures, Technovation, 24, pp. 779-791.         [ Links ]

[2] Siriram, R. 2012. A soft and hard systems approach to business process management, Systems Research and Behavioral Science, 29, pp. 87-100.         [ Links ]

[3] Grant, R.M. 1999. In Zack, H. Knowledge and strategy, Butterworth-Heinemann, Chapter 1: The resource-based theory of competitive advantage: Implications for strategy formulation, pp. 3-23, Woburn Massachusetts.         [ Links ]

[4] Srivannaboon, S. & Milosevic, D.Z. 2006. A two-way influence between business strategy and project management, International Journal of Project Management, 24(3), pp. 493-505.         [ Links ]

[5] Fringsdorf, O., Zuo, J. & Zia, B. 2014. Critical success factors for project success in a defence environment, International Journal of Project Management, 32, pp. 803-814.         [ Links ]

[6] Maylor, H., Brady, T., Cook-Davies, T. & Hodgson, D. 2006. From projectification to programmification, International Journal of Project Management, 24, pp. 663-674.         [ Links ]

[7] Maylor, H., Turner, N. & Murray-Webster, R. 2015. "It worked for manufacturing 1" Operations strategy in project-based operations, International Journal of Project Management, 33, pp. 103-115.         [ Links ]

[8] Thiry, M. & Deguire, M. 2007. Recent developments in project-based organisations, International Journal of Project Management, 25(7), pp. 649-658.         [ Links ]

[9] Canonico, P. & Sonderlund, J. 2010. Getting control of multi-project organizations: Combining contingent control mechanisms, International Journal of Project Management, 28, pp. 796-806.         [ Links ]

[10] Kujala, S., Artto, K., Aaltonen, P. & Turkulainen, V. 2010. Business models in project-based firms: Towards a typology of solution-specific business models, International Journal of Project Management, 28, pp. 96-106.         [ Links ]

[11] Wikstroöm, K., Artto, K., Kujala, J. & Söderlund, J. 2010. Business models in project business, International Journal of Project Management, 28, pp. 832-841.         [ Links ]

[12] Buys, A. & Stander, M. 2010. Linking projects to business strategy through project portfolio management, South African Journal of Industrial Engineering, 21(1), pp. 59-68.         [ Links ]

[13] Meskendahl, S. 2010. The influence of business strategy on project portfolio management and its success: A conceptual framework, International Journal of Project Management, 28, pp. 807-817.         [ Links ]

[14] Young, R., Young, M., Jordaan, E. & O'Connor, P. 2012. Is strategy being implemented through projects? Contrary evidence from a new leader in public management, International Journal of Project Management, 30, pp. 887-900.         [ Links ]

[15] Young, R. & Grant J. 2015. Is strategy being implemented through projects? Disturbing evidence in the state of NSW, International Journal of Project Management, 33, pp. 15-28.         [ Links ]

[16] Andersen, E.S. & Jessen, S.A. 2003. Project maturity in organizations, International Journal of Project Management, 21, pp. 457-461.         [ Links ]

[17] Goloni, R., Kalchschmidt, M. & Landoni, P. 2015. Adoption of project management practices: The impact on international development projects of non-governmental projects, International Journal of Project Management, 33, pp. 650-663.         [ Links ]

[18] Mittermaier, H. & Steyn, H. 2009. Project management maturity: An assessment of maturity for developing pilot plants, South African Journal of Industrial Engineering, 20(1), pp. 95-107.         [ Links ]

[19] Cao, Q. & Hoffmann, J.J. 2011. A case study approach for developing a project performance evaluation system, International Journal of Project Management, 29(2), pp. 155-164.         [ Links ]

[20] Grant, K.P. & Pennypacker, J.S. 2006. Project management maturity: An assessment of project management capabilities among and between selected industries, IEEE Transactions, 53(1), pp.59-68.         [ Links ]

[21] Tahri, H. & Drissi-Kaitounib, O. 2015. New design for calculating project management maturity (PMM). Procedia - Social and Behavioural Sciences, 181, pp. 171-177.         [ Links ]

[22] Pennpacker, J.S. & Grant, K.P. 2003. Project management maturity: An industry benchmark, Project Management Journal, 34(1), pp.4-11.         [ Links ]

[23] Qureshi, T.M., Warraich, A.S. & Hijazi, S.T. 2009. Significance of project management performance assessment (PMPA) model, International Journal of Project Management, 27(4), pp. 378-388.         [ Links ]

[24] Miller, R. & Lessard, D. 2009. The strategic management of large engineering projects: Shaping institutions, risks and governance. Cambridge, MA: Massachusetts Institute of Technology.         [ Links ]

[25] Morris, P. & Hough, G. 1987. The anatomy of major projects: A study of the reality of project management. Chichester: John Wiley & Sons, Inc.         [ Links ]

[26] Merrow, E., McDonnell, L. & Arguden, L. 1988. Understanding the outcomes of megaprojects: A quantitative analysis of very large civilian projects, The Rand Corporation Publication Series,Santa Monica, CA.         [ Links ]

[27] Evans, M., 2005. Project management: Overdue and over budget, over and over again. The Economist, Available online: https://www.economist.com/node/4065653. Accessed 2018/05/08        [ Links ]

[28] Shehu, Z., Endut, I.R., Akintoye, A. & Holt, G.D. 2014. Cost overrun in Malaysian construction projects: A deeper insight, International Journal of Project Management, 32, pp. 1471-1480.         [ Links ]

[29] Kloppenberg, T.J. & Opfer, W.A. 2002. The current state of project management research: Trends, interpretations, and predictions, Project Management Journal, 33(2), p. 5.         [ Links ]

[30] Grundy, T. & Brown, L. 2002. Strategic project management: Creating organizational breakthroughs. Cornwall: Thompson Learning.         [ Links ]

[31] Crawford, P. & Bryce, P. 2003. Project monitoring and evaluation: A method for enhancing the efficiency and effectiveness of aid project implementation, International Journal of Project Management, 21, pp. 363-373.         [ Links ]

[32] Kadefors, A. 2004. Trust in project relationships: Inside the black box, International Journal of Project Management, 22(3), pp. 175-182.         [ Links ]

[33] Cheung, S., Wong, S., Fung, A.S.Y. & Coffey, W.V. 2006. Predicting project performance through neural networks, International Journal of Project Management, 24(3), pp. 207-215.         [ Links ]

[34] Hyväri, I. 2006. Project management effectiveness in project-oriented business organizations, International Journal of Project Management, 24(3), pp. 216-225.         [ Links ]

[35] Shenhar, A.J. & Dvir, D. 2007. Project management research: The challenge and opportunity, Project Management Journal, 38(2), pp. 93-99.         [ Links ]

[36] Kollveit, B.J., Karlsen, J.T. & Gronhaug, K. 2007. Perspectives on project management, International Journal of Project Management, 25, pp. 3-9.         [ Links ]

[37] Kwak, Y.H. & Smith, B.M. 2009. Managing risks in mega defense acquisition projects: Performance policy and opportunities, International Journal of Project Management, 27, pp. 812-820.         [ Links ]

[38] Blichfeldt, B.S. & Eskerod, P. 2008. Project portfolio management: There's more to it than what management enacts, International Journal of Project Management, 26(4), pp. 357-365.         [ Links ]

[39] Aubry, M., Muller, R., Hobb, B. & Blomquist, T. 2010. Project management offices in transaction, International Journal of Project Management, 28, pp. 766-778.         [ Links ]

[40] Choi, J., Chung, J. & Lee, D. 2010. Risk perception analysis: Participation in China's water PPP market, International Journal of Project Management, 28, pp. 580-592.         [ Links ]

[41] Shi, Q. 2011. Rethinking the implementation of project management: A value adding path map approach, International Journal of Project Management, 29(3), pp. 295-302.         [ Links ]

[42] Yang, L., Huang, C. & Wu, K. 2011. The association among project manager's leadership style, teamwork and project success, International Journal of Project Management, 29(3), pp. 258-267.         [ Links ]

[43] Davies, K. 2014. Different stakeholder groups and their perceptions of project success, International Journal of Project Management, 32, pp. 189-201.         [ Links ]

[44] Pinto, J. 2014. Project management governance and the normalization of deviance, International Journal of Project Management, 32, pp. 376-387.         [ Links ]

[45] Fernandes, G., Ward, S. & Araújo, M. 2015. Improving and embedding project management practice in organizations: A qualitative study, International Journal of Project Management, 33, pp. 1052-1067.         [ Links ]

[46] Baker, B. & Fisher, D. 1998. Factors affecting project success. In: Cleland, D. & King, W. (eds), Project management handbook, New York: Van Nostrand.         [ Links ]

[47] Atkinson, R. 1999. Project management: Cost, time and quality, two best guesses and a phenomenon: It's time to accept other success criteria, International Journal of Project Management, 17(6), pp. 337-342.         [ Links ]

[48] Chapman, C. & Ward, S. 2004. Why risk efficiency is a key aspect of best practices, International Journal of Project Management, 22, pp. 619-632.         [ Links ]

[49] Chapman, C. 2006. Key points of contention in farming assumptions for risk and uncertainty, International Journal of Project Management, 24(4), pp. 303-313.         [ Links ]

[50] Atkinson, R., Crawford, L. & Ward, S. 2006. Fundamental uncertainties in projects and the scope of project management, International Journal of Project Management, 24(8), pp. 687-698.         [ Links ]

[51] Braimah, N. & Ndekugri, I. 2008. Factors influencing the selection of delay analysis methodologies, International Journal of Project Management, 26(8), pp. 789-799.         [ Links ]

[52] Ashan, K. & Gunawan, I. 2010. Analysis of cost and schedule performance of international development projects, International Journal of Project Management, 28, pp. 68-78.         [ Links ]

[53] Van Os, A., Van Berkel, F., De Gilder, D., Van Dyck, C. & Groenewegen, P. 2015. Project risk as an identity threat: Explaining the development and consequence of risk discourse in an infrastructure project, International Journal of Project Management, 33, pp. 877-888.         [ Links ]

[54] Hazir, O. 2015. A review of analytical models, approaches and decision support tools in project monitoring and control, International Journal of Project Management, 33, pp. 808-815.         [ Links ]

[55] Crawford, J. 2002. Project management maturity model: Providing a proven path to project management excellence. Havertown, Pennsylvania: Project management solutions, Inc.         [ Links ]

[56] Jugdev, K. & Thomas, J. 2002. Project management maturity models: The silver bullets of competitive advantage, International Journal of Project Management, 33(4), pp. 4-14.         [ Links ]

[57] Jaafari, A. 2007. Project and program diagnostics: A systemic approach, International Journal of Project Management, 25(8), pp. 781-790.         [ Links ]

[58] Guangshe, J., Yuting, C., Xiangdong, X., Jianguo, C., Jiming, C. & Kewei, T. 2011. Program management organization maturity integrated model for mega construction projects in China, International Journal of Project Management, 29, pp. 834-845.         [ Links ]

[59] Ika, L.A., Diallo, A. & Thuillier, D. 2012. Critical success factors for World Bank projects: An empirical investigation, International Journal of Project Management, 30, pp. 105-116.         [ Links ]

[60] Teller, J. & Kock, A. 2013. An empirical investigation on how portfolio risk management influences project portfolio success, International Journal of Project Management, 31, pp. 817-829.         [ Links ]

[61] Mir, F.A. & Pinnington, A.H. 2014. Exploring the value of project management: Linking project management performance and project success, International Journal of Project Management, 32, pp. 202-217.         [ Links ]

[62] Basu, R. 2014. Managing quality in projects: An empirical study, International Journal of Project Management, 32, pp. 178-187.         [ Links ]

[63] Xu, Y. & Yeh, C. 2014. A performance-based approach to project assignment and performance evaluation, International Journal of Project Management, 32, pp. 218-228.         [ Links ]

[64] Project Management Institute. 2008. A guide to project management body of knowledge, 4th ed. Pennsylvania: Project Management Institute.         [ Links ]

[65] Nunnally, J.C. 1978. Psychometric theory, 2nd ed. New York: McGraw Hill.         [ Links ]

[66] Siriram, R. & Snaddon, D.R. 2005. Verifying links in technology management, transaction processes and governance structures, Technovation, 25, pp. 321-337.         [ Links ]

[67] Hair, J.F. Jr., Anderson, R.E., Tatham, R.L. & Black, W.C. 1998. Multivariate data analysis, Fifth ed. Prentice Hall, New Jersey.         [ Links ]

 

 

Submitted by authors 23 Oct 2016
Accepted for publication 19 Apr 2018
Available online 31 /May 2018

 

 

* Corresponding author raj@alpha-concepts.com

Creative Commons License Todo o conteúdo deste periódico, exceto onde está identificado, está licenciado sob uma Licença Creative Commons