SciELO - Scientific Electronic Library Online

 
vol.33 issue1Perceptions of cyber bullying at primary and secondary school level amongst student teachers in the Eastern Cape province of South AfricaA survey of automated financial statement fraud detection with relevance to the South African context author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Article

Indicators

Related links

  • On index processCited by Google
  • On index processSimilars in Google

Share


South African Computer Journal

On-line version ISSN 2313-7835
Print version ISSN 1015-7999

SACJ vol.33 n.1 Grahamstown Jul. 2021

http://dx.doi.org/10.18489/sacj.v33i1.950 

RESEARCH ARTICLE

 

The current state of measuring return on investment in user experience design

 

 

Trevor ChawanaI; Funmi AdebesinII

IDepartment of Informatics, University of Pretoria, South Africa. revorchawana@gmail.com (corresponding)
IIDepartment of Informatics, University of Pretoria, South Africa. funmi.adebesin@up.ac.za

 

 


ABSTRACT

Incorporating user experience (UX) design into system development processes can give an organisation a competitive advantage over its rivals. Embedding UX design into system development processes requires investment in skilled professionals, who in turn, follow a methodical process to make UX design an integral part of the organisation. Despite the value that can be added by UX design, demonstrating the return on investment (ROI) in UX design remains a perennial challenge because of the difficulty associated with explicitly linking UX design with good ROI, to the exclusion of other system development processes. This paper reports on the emerging perspectives on measuring ROI in UX design in a selection of South African organisations. Interviews were conducted with 33 UX professionals from four organisations to get insight into how ROI in UX design is measured in their respective organisations. The results showed that some participants were aware of the metrics that could be used to measure ROI in UX design. However, none of the four organisations were specifically calculating ROI in UX. Rather, ROI was being calculated on an entire project.
CATEGORIES: · Human-centred computing - Human computer interaction (HCI) · HCI design and evaluation

Keywords: user experience, user experience design, UX, return on investment, ROI, ROI metrics, software development


 

 

1 INTRODUCTION

The global spending on information technology (IT) projects in 2019 was projected at $3.74 trillion. This constituted an increase of 0.6% on the 2018 figures (Gartner, 2019). The 2015 Standish Group CHAOS report also indicated that only 36% of IT projects were deemed successful, 45% were challenged, while 19% failed (Standish Group, 2015). Software developers often spend about 50% of their time on rework that could have been avoided. The cost of this rework can be substantial, especially if errors were identified after deployment (Aleryani, 2020; Ramdoo & Huzooree, 2015).

There is growing evidence that incorporating user experience (UX) practices into the design of applications (apps), websites, and systems can give a business a competitive advantage over its rivals (Boztepe, 2007; Paunovic, 2017; Sward & Macarthur, 2007). Embedding UX design into development processes is especially important in contemporary business environments where consumers can readily switch their loyalty. Present-day consumers expect and consider an optimised user experience to be a basic requirement (Bilgihan, 2016; Minge & Thüring, 2018; Paunovic, 2017). UX can be defined as "a momentary, primarily evaluative feeling (good-bad) while interacting with a product or service" (Hassenzahl, 2008, p. 12). From an organisational perspective, the benefits of good UX include building the right product for users, reduction in call centre volumes to report errors, and increased customer loyalty (Aleryani, 2020). These benefits can translate into increased profitability (Donahue, 2001; Kolbeinsson et al., 2020). Positive UX does not happen by accident, it is the result of the intentional incorporation of UX design practices into an organisation's system development processes (Erdös, 2019).

Despite the benefits of incorporating UX design into system development processes, demonstrating return on investment (ROI) in UX remains a perennial challenge (Graefe et al., 2003). This challenge can be attributed to the fact that the use of traditional methods to demonstrate the value added by UX design processes is no longer sufficient to get the buy-in of decisionmakers in an organisation. It is important to demonstrate the impact of UX design within the context of the organisation (Graefe et al., 2003; Turner, 2011). Complicating the status quo further is the disagreement among researchers on the subject of measuring ROI in UX. Although some authors have suggested metrics that could be used to measure ROI in UX design (Bradner & Sauro, 2012; Reichheld, 2003; Weinschank, 2005; Young, 2014), others (Rosenberg, 2004; Turner, 2011) disagree with the accuracy of such metrics, citing the difficulty of creating a causal link between the measures of ROI and UX design.

The research reported in this paper forms part of a broader study on the optimisation of the UX design process for the timeous development of information systems (Chawana, 2020). However, this paper is specifically focused on measuring ROI in UX design. The main research question for the paper is "What are the perspectives on measuring ROI in UX design in South Africa?". The structure of the remaining sections of the paper are as follows: Section 2 provides an overview of extant literature that are relevant to the paper. This includes the concept of ROI and the different metrics that could be used to measure ROI in UX design. The research design and methodology are discussed in Section 3, while Section 4 contains the research results. The discussion and conclusion of the paper are presented in Section 5.

 

2 OVERVIEW OF RELATED STUDIES

2.1 The concept of return on investment

Organisations spend money to derive benefits that will have positive impacts on their longevity (Layard & Glaister, 1994). Expenditure made to derive a benefit is known as an investment (Hovey, 2002). Due to limited financial resources, organisations often allocate different amounts to various investments and measure the return on such investments (Lyneis, 1975). The amount of financial resources allocated to a specific investment is related to the extent to which the potential investment will increase the organisation's profitability and long-term survival (Lyneis, 1975). Periodically, organisations assess the extent to which their investments are yielding the anticipated returns (Schueler & Loveder, 2020).

Embedding UX design in an organisation's system development processes requires the organisation to follow a methodical process, which in turn, requires specialised skills (Rohn, 2007). The skills and knowledge possessed by individuals are known as human capital (Schultz, 1961). To successfully embed UX design in an organisation, investments in the appropriate human capital are required (Schultz, 1961). Hence, incorporating UX design practices into the system development processes can be seen as an investment, since an organisation has to spend financial resources to recruit skilled personnel or train current employees to achieve the goal of embedding UX design in the organisation.

2.2 Measuring return on investment in UX design

UX design is not exempt from the measurement agenda and there are divergent views on measuring the benefits of incorporating UX design into an organisation's system development processes. Various metrics have also been devised to establish whether a particular investment has yielded the expected returns (Schueler & Loveder, 2020). This section provides an overview of the different viewpoints on measuring ROI in UX design from existing literature. The section also includes some of the metrics that could be used to measure ROI in UX design.

Authors like Bias and Mayhew (2005) and Djamasbi et al. (2014) have reported on organisations that have benefitted financially from incorporating UX design into their system development process. According to Turner (2011), organisations can follow two important steps when measuring ROI in UX design. The first step is to classify the organisation's strategic goals into four perspectives namely: (i) financial, (ii) customer, (iii) internal business, and (iv) innovation and learning. Once the different perspectives of the organisation's strategic goals have been established, the second step is to identify how incorporating UX design into the development process can contribute to each perspective, using a metric(s) that is(are) suited to each perspective.

The financial perspective expresses ROI in monetary terms, using metrics such as the Net Present Value (NPV). The customer perspective measures how customers perceive the organisation, using metrics like the Net Promoter Score (NPS). The internal business perspective is concerned with process improvements and can be measured using the Cost Per Order (CPO) metric. Lastly, the innovation and learning perspective identifies the improvements that are required within an organisation to support the customer and internal business perspectives. The innovation and learning perspective typically contains a list of initiatives that should be undertaken to support the customer and internal business perspectives, rather than a set of metrics.

Researchers like Rosenberg (2004) and Fedele et al. (2017) have expressed scepticism on the accuracy of the metrics that are typically used to measure ROI in UX design. One of the criticisms of the measures of ROI in UX design is that there is insufficient empirical evidence to support the claim that a better user experience results in a good ROI (Rosenberg, 2004). Another source of scepticism over the measures of ROI in UX design stems from the tendency to create causal relationships between UX design practices and the ROI metrics while omitting (intentionally or otherwise) other factors that could have contributed to the calculated ROI. The assumptions that are made when creating these causal relationships can result in inaccuracies, largely because the studies were not conducted in a controlled environment (Fedele et al., 2017; Rosenberg, 2004).

More often than not, decision-makers in an organisation will seek only the metrics that are necessary to make informed decisions (Beach & Mitchell, 1978). This implies that some organisations may not necessarily deem it essential to apply any metric to measure the ROI in UX design within their organisational context. In situations where decision-makers believe that it is important to establish the ROI in UX design, several factors are typically taken into account. These include the type of product being developed or sold (for example, mobile or web-based application), the application domain (for example, e-commerce or e-government), the size of the organisation (for example, small, medium, or large-size), the number of design professionals in the development team, the maturity of the organisation's design process, and the type of client (for example, internal or external client) (Graefe et al., 2003).

2.2.1 Net promoter score

The Net Promoter Score (NPS) is a metric that can be used to measure ROI in UX design from customers' perspectives. It is used to measure customers' loyalty to an organisation, based on the extent to which they are likely to promote the organisation's products or services (Reichheld, 2003). There are four key steps involved in calculating an NPS (Bradner & Sauro, 2012). The first step entails asking customers to rate the likelihood of them recommending an organisation's product or service to other people, on a scale of zero to ten, with zero to indicate "extremely unlikely" and ten to indicate "extremely likely." The second step entails dividing all the responses into three categories, namely promoters, passives, and detractors.

Promoters are responses with ratings of nine or ten, passives are responses with ratings of seven or eight, while detractors are responses with ratings from zero to six. Once the responses have been categorised, the third step involves calculating the percentage of promoters and the percentage of detractors. The last step is to subtract the percentage of detractors from the percentage of promoters (Bradner & Sauro, 2012).

Authors like Bradner and Sauro (2012), Feng and Wei (2019), and Walsh et al. (2014) have linked good user experience to increased customer loyalty. Given that UX is considered to be a key determinant of customer loyalty, organisations can use improvements in the NPS as the ROI in UX design initiatives (Bradner & Sauro, 2012; Reichheld, 2003).

2.2.2 Drop off rate

The drop-off rate is a measure of the number of users that exit a web-based application or website from a specific web page (Young, 2014). A high drop-off rate for a particular page on a web-based application or website can be an indication of poor design (Weinschank, 2005).

UX practitioners typically identify the drop-off rate on a particular user flow by reviewing the statistics that are derived from web analytics software (Young, 2014). Web analytics software records the number of users that exit an application, identifying the web page they last accessed on each user task flow (Young, 2014). A high drop-off rate on a page where users were not expected to exit a particular user task flow may indicate a design flaw (Young, 2014). Once the page with the design flaw has been identified, correcting the design flaw typically involves the creation of one or more alternative designs, followed by testing the designs with end-users, a process known as A/B testing (Gardey & Garrido, 2020). The outcome of an A/B test is a better understanding of the design option that users are most likely going to find easier to use, thereby reducing the drop-off rate (Gardey & Garrido, 2020). A decrease in drop-off rate is one of the metrics that can be used to calculate ROI in UX design (Weinschank, 2005). A decrease in drop-off rate is established by comparing the drop-off rate before a redesign and after the redesign (Weinschank, 2005). This metric falls within the innovation and learning perspective of an organisation's strategic goals. It supports the customer perspective by optimising the flow of users' tasks when they execute a task on an application.

2.2.3 Increase in sales

Good UX is one of the outcomes of a user-centred design process (Weinschank, 2005). When an application is developed with little or no consideration for users' needs, user frustration is often the result (Weinschank, 2005). Good UX can lead to increased sales on e-commerce websites and other software from which organisations generate revenue (Bradner & Sauro, 2012; Weinschank, 2005). An increase in sales is therefore a metric that can be used to measure ROI in UX design and is calculated by comparing the sales revenue before a redesign with the sales revenue after a redesign (Weinschank, 2005). Measuring the increase in sales that results from improvements in UX design falls within the financial perspective of an organisation's strategic goals, given that it is an expression of ROI in monetary terms (Turner, 2011).

2.2.4 Cost reduction

For-profit organisations can maximise their profitability by either increasing sales revenue or reducing costs (Crane, 2017). Cost reduction enables both for-profit and non-profit organisations to improve their long-term sustainability (Crane, 2017). When compared to well-designed applications, poorly designed ones often result in high costs of user training, higher call centre volumes for user support, and the need for more detailed user manuals (Cham & Costa, 2017; Weinschank, 2005). By incorporating UX design into the development process, organisations can reduce the costs associated with user training, user support, and documentation (Weinschank, 2005). UX design practices also support the decision-making related to the selection of the right features to include in an application, thereby avoiding unnecessary and costly rework (Cham & Costa, 2017).

Cost reduction is one of the metrics that can be used to measure ROI in UX design (Weinsch-ank, 2005). The cost reduction that is accrued from the incorporation of UX design into system development processes can be calculated by subtracting the cost of user training, support, and documentation after a redesign from the costs before the redesign (Weinschank, 2005).

Another cost-related metric is Cost Per Order (CPO), which is used to measure the total costs associated with a single customer order (Turner, 2011). A call by a customer to an organisation's customer call centre because of the inability to complete a task increases the costs associated with the order that is placed by the customer (Turner, 2011). If a redesign of the website results in fewer calls to the customer call centre, the overall cost per order will be reduced (Turner, 2011).

Organisations that utilise metrics like CPO are more likely to address the problems that customers face because cost reduction is directly linked to profitability (Turner, 2011). Given that cost reduction measures relate to internal process and user flow improvement, it falls within the internal business perspective of organisational strategic goals.

2.2.5 Net present value

When calculating ROI, it is important to consider the principle of the time value of money, which states that money is worth more right now than in the future (Espinoza, 2014). The Net Present Value (NPV) is a metric that accounts for the time value of money by comparing the current and future costs of investment to future benefits, using a single time frame, which is the present (Erdogmus et al., 2004). NPV can be defined as the present value of future cash inflows, less the present value of the investment amount, and future cash outflows (Anthes, 2003). Simply put, NPV is the net outcome from an investment made over a number of years, expressed in the current value of money (Anthes, 2003). The formula to calculate the present value PV of a future cash flow C that occurs in t years with a discount rate r is (Kahn & Nelling, 2010):

The net present value is the sum of the present value of money for each year that an investment is expected to generate a return (Turner, 2011). The discount rate (r) in the formula represents the amount that is waived today through an investment to earn better returns in the future (Kahn & Nelling, 2010).

In the context of UX design, the NPV metric can be used to establish the ROI in UX by comparing the costs of conducting UX-related activities, such as user research, design, and user testing on a project, with the future cash inflows that are associated with those activities, using today's value of money (Turner, 2011). The NPV metric falls within the financial perspective of organisational strategic goals, given that it expresses ROI in UX in monetary terms (Turner, 2011).

 

3 RESEARCH DESIGN AND METHODOLOGY

This section provides a detailed discussion of the research design and the methodology that was followed to collect data.

3.1 Research design

The research was underpinned by the interpretive research paradigm, which is based on the philosophical assumption that there are multiple realities and these realities are socially constructed through shared meanings (Klein & Myers, 2001). The research employed a qualitative case study research strategy. The choice of a qualitative case study research strategy was motivated by three key factors. Firstly, UX design often occurs in the context of system development projects, hence, the research strategy of choice needed to be suitable for investigating UX design practices within an organisational context. Secondly, the researchers wanted to understand the data that was collected from the study participants' perspectives. Lastly, the investigation needed to be carried out in more than one organisation, with research data elicited within each organisation's natural context. Case study research is a good strategy to use for qualitative studies where the experiences of the actors are critical to the study (Ben-basat et al., 1987). Given that the focus was on more than one organisation, the multiple case study strategy was employed using four different organisations as the cases (Yin, 2011). A key benefit of case study research was maximised by interviewing the key role players in the participating organisations. The potential impact of inconsistent recollection of events was also minimised by interviewing multiple participants per case study organisation. Interviewing multiple participants was beneficial in that interview responses that deviated significantly from otherwise homogenous responses could easily be identified as an exception to the general response (Benbasat et al., 1987).

Semi-structured interviews were conducted in four South African organisations. The choice of participating organisations was based on two key factors. Firstly, the organisation should have UX design practices embedded in its system development process. This was an important consideration, given that such organisations would be more likely to provide information that would provide the answer to the research question. The second consideration was that an organisation should be willing to participate in the study. Given that the selection of participants in the study was based on the specific qualities they possessed, purposive sampling was used to select the organisations that participated in the study (Etikan et al., 2016). UX professionals and UX team leads in the four organisations were interviewed. Individuals in roles that work closely with UX professionals, for example, Business Analysts, and Product Owners were also interviewed.

Data analysis was guided by the five steps to interpretive studies' data analysis of Terre Blanche et al. (2006), which entailed (i) familiarisation with the research data, (ii) identification and documentation of responses that were relevant to answering the research question to gain deeper insight, (iii) grouping similar insights into themes using the bottom-up analysis approach (iv) linking the contents from the interview transcripts to the themes, and (v) interpreting the research findings and reporting on the themes identified. The first step in the data analysis process involved familiarisation with the interview data. This was achieved by listening to all the interview recordings and reading the interview transcripts. While listening to the interview recordings and reviewing the transcripts, responses that could help to answer the research question were identified and documented. The interview transcripts were reviewed multiple times, identifying and documenting the useful responses until no new insights could be derived from the interview recordings and transcripts. Thereafter, similar insights were grouped into themes, using a bottom-up analysis approach. Based on the insights from the interview recordings and transcripts, the two themes discussed in Section 4 were created.

3.2 Research ethics

Ethical clearance for this research was granted by the Faculty of Economic and Management Sciences' Research Ethics Committee at the University of Pretoria. Each of the four participating organisations granted formal permission to involve their employees as study participants. Individual study participants from each organisation also signed informed consent. Participants were assured that the data will only be used for research purposes. Neither monetary nor non-monetary incentive was offered to participants.

 

4 RESULTS

As stated in Section 1, the research reported in this paper is part of a broader study on the optimisation of the UX design process for the timeous development of information systems (Chawana, 2020). As such, a detailed discussion of the results from the broader study is outside the scope of this paper. This section presents the results that are specifically related to the research question that is answered in this paper, namely "What are the perspectives on measuring ROI in UX design in South Africa?".

A total of 33 participants were interviewed from four organisations, hereinafter referred to as Org1, Org2, Org3, and Org4, respectively (see Table 1). Three out of the four development teams interviewed worked in a contract development environment, with only one being an in-house development team. Two people were interviewed at Org1, which is a Johannesburg-based company that specialises in contract development of applications for client organisations in a variety of industries. Org2 is a management consulting company with its head office in Johannesburg and provides software development and management consulting services to organisations in different industries. Nine participants were interviewed from Org2. Seventeen people were interviewed from Org3, which is a financial services organisation that has existed for more than a century. Lastly, five people were interviewed from Org4, which is a management consulting company based in Johannesburg.

Based on analysis of the interview data, neither of the two participants from Org1 identified any metric that could be used to measure ROI in the incorporation of UX design into system development processes. Out of the nine participants from Org2, five identified an increase in sales as a metric that could be used to measure ROI in UX design, while three participants identified drop-off rate reduction and one identified the NPS improvement metric. From Org3, seven out of the 17 participants identified an increase in sales as a metric that could be used to measure ROI in UX, two identified the NPS improvement, while one identified the cost reduction metric. Two participants from Org4 identified an increase in sales as a metric that could be used to measure ROI in UX, with the same number of participants identifying the NPS improvement, cost reduction, and drop-off rate reduction as metrics that could be used to measure ROI in UX design. A summary of the ROI metrics identified by participants from the four organisations is provided in Table 2.

As shown in Table 2, although participants from Org2, Org3, and Org4 identified metrics that could be used to measure ROI in UX design, none of the four organisations were using any of the metrics to calculate ROI in incorporating UX design into their system development processes. According to participants from Org1, the development team did not calculate ROI in UX because their client organisations did not share information related to the benefits that were derived from projects with them. Without this information, it would be difficult to calculate the value added by the incorporation of UX design into system development processes in monetary terms.

The two main themes from the research results that are relevant to the research question addressed in this paper were:

Difficulty with separating the calculation of ROI in UX design from other system development processes.

Participating organisations did not see the need to calculate ROI in UX design.

One of the reasons why the participating organisations did not explicitly calculate ROI for UX design had to do with the difficulty associated with establishing accurate ROI calculations that separate UX design from other systems development activities. System development projects have different components, including requirements analysis, logical and physical design, UX design, coding, testing, as well as user training. Each of these development activities contributes to the overall quality of a system. Ongoing maintenance is also required after deployment throughout the useful life of a system. Study participants were of the view that it is impractical to single out UX design in the calculation of ROI as opposed to doing the calculation for an entire system development project.

According to study participants, explicit calculation of ROI in UX design activities is a challenge for the development team. For example, two participants from Org3 expressed the difficulty as follows:

It's very tangible to measure what was spent on tech, because you have lines of code. [P6]

and

Measuring ROI for UX is kind of a difficult one. [P17]

Another participant from Org4 expressed the difficulty thus:

... It is very difficult to say with 100% accuracy where the value is coming from. [P5]

The difficulty associated with singling out UX design for the calculation of ROI was expressed by study participants from both contract and in-house development environments. As indicated in the opening paragraph of this section, study participants from Org1, Org2, and Org4 worked in contract development environments, while participants from Org3 worked in an in-house development context.

Another reason proffered by participants for not calculating ROI in UX design was the view that there was no need for such metrics. In the case of Org2, which is a contract software development company, the clients created the business case for a system development initiative. At the end of a development project, what would be important to clients would be the extent to which the entire project enabled them to achieve the business case objectives because

... nobody puts together a business case for user experience. [P6]

The view that it is not necessary to calculate ROI in UX design was also shared by participants from Org3, who were part of an in-house development team. The participants were of the view that the value that was derived from the investment in an entire project was more important, not necessarily the cost of the UX design component. If a system failed to meet its development objectives, it is important to know why the entire project has failed because

... nobody cares about the journey map ... it's 'we spent R20 million on building an asset and no one is using it.' [P6]

 

5 DISCUSSION AND CONCLUSION

This study investigated the current perspectives on measuring ROI in incorporating UX design into development processes in South Africa. As stated in Section 2.2, factors like the product type, application domain, organisation size, and the type of client can influence the metrics that are used to measure ROI in an organisation (Graefe et al., 2003). The research findings showed that although the majority of the interviewees in the participating organisations were aware of the metrics that could be used to measure ROI in UX design, none of the organisations were using these metrics to explicitly calculate ROI in UX design. Rather, ROI was being calculated on entire software development projects. This status quo existed in both contract and in-house development contexts. Some of the reasons can be attributed to the challenge associated with singling out UX design activities to measure their impact on the development process.

The research results strengthen the criticism by authors like Rosenberg (2004) and Fedele et al. (2017), who were of the view that there is insufficient empirical evidence to justify the claim that a better user experience results in a good ROI. The authors also believe that the assumptions made when creating a link between UX practices and ROI metrics tend to ignore other factors that could have contributed to the ROI being calculated. The result reported in this paper also supports the position of Turner (2011), who is of the view that the use of metrics should take into account organisational context. The research findings also showed that organisations, irrespective of whether they operate in contract or in-house development environments may not necessarily see the need to calculate ROI in UX design. Measuring ROI in UX design also requires access to the appropriate financial data, which may not be shared with contract development organisations. Even in in-house development contexts where the development team could gain access to the financial data required for calculating ROI in UX design, such calculations may not be deemed as essential. In addition, decision-makers will seek only the metrics that are necessary to make informed decisions (Beach & Mitchell, 1978). For instance, in the context of Org2, decision-makers were interested in the calculation of ROI for their entire projects and as a result, the appropriate metrics were used for this calculation.

The fact that ROI in UX design was not explicitly calculated in any of the case study organisations should not necessarily lead to the conclusion that UX design did not hold strategic importance to the organisations' decision-makers. The research results suggest that this is not likely to be the case, since there was evidence that UX design practices are well-embedded in all the four organisations' system development processes. Another indication that UX design holds strategic importance to all four organisations related to the continued recruitment of UX design professionals. As discussed in Section 2.1, the recruitment of UX professionals can be seen as an investment in UX design. Furthermore, the inclusion of UX design practices like user research in system development processes suggests an implicit understanding of the value added by UX design. A more plausible reason for the status quo is that people in these organisations do not need to be convinced of the value that UX design brings to their organisation.

In conclusion, there is evidence that explicit calculation of ROI in UX design continues to be a challenge in the South African system development environment. From a theoretical point of view, several metrics could be used to calculate the impact of incorporating UX design into system development processes. However, singling out the contribution of UX design to calculate their ROI is practically difficult.

 

References

Aleryani, A. Y. (2020). The impact of the user experience (UX) on the quality of the requirements elicitation. International Journal of Digital Information and Wireless Communications, 10(1), 1-9.         [ Links ]

Anthes, G. H. (2003). Net present value. Computerworld, 37(7), 30.         [ Links ]

Beach, L. R. & Mitchell, T. R. (1978). A contingency model for the selection of decision strategies. Academy of Management Review, 3(3), 439-449.         [ Links ]

Benbasat, I., Goldstein, D. K. & Mead, M. (1987). The case research strategy in studies of information systems. MIS Quarterly, 369-386.

Bias, R. G. & Mayhew, D. J. (2005). Cost-justifying usability: An update for the Internet age. Elsevier.

Bilgihan, A. (2016). Gen Y customer loyalty in online shopping: An integrated model of trust, user experience and branding. Computers in Human Behavior, 61, 103-113. https://doi.org/10.1016/j.chb.2016.03.014        [ Links ]

Boztepe, S. (2007). User value: Competing theories and models. International Journal ofDesign, 1 (2), 55-63.         [ Links ]

Bradner, E. & Sauro, J. (2012). Software user experience and likelihood to recommend: Linking UX and NPS. User Experience Professionals' Association International Conference, Henderson.

Cham, A., Jennifer & Costa, K. (2017). UX Design - Maximising the value of scientific software in life science R&D [Last accessed: 20 Jun 2021]. https://www.ddw-online.com/ux-design-maximising-the-value-of-scientific-software-in-life-science-rd-1795-201708/

Chawana, T. N. (2020). Optimising the user experience design process for timeous systems development: A South African case study (Thesis). University of Pretoria. https://repository.up.ac.za/bitstream/handle/2263/80180/Chawana_Optimising_2020.pdf        [ Links ]

Crane, S. P. (2017). The five operational keys to maximizing business profitability. The Journal of Business Forecasting, 36(1), 10-12.         [ Links ]

Djamasbi, S., McAuliffe, D., Gomez, W., Kardzhaliyski, G., Liu, W. & Oglesby, F. (2014). Designing for success: Creating business value with mobile user experience (UX). International conference on HCI in Business, 299-306.

Donahue, G. M. (2001). Usability and the bottom line. IEEE Software, 18(1), 31-37.         [ Links ]

Erdogmus, H., Favaro, J. & Strigel, W. (2004). Return on investment. IEEE Software, 21(3), 18-22.         [ Links ]

Erdös, F. (2019). Economical aspects of UX design and development. 10th IEEE International Conference on Cognitive Infocommunications (CogInfoCom).

Espinoza, R. D. (2014). Separating project risk from the time value of money: A step toward integration of risk management and valuation of infrastructure investments. International Journal of Project Management, 32(6), 1056-1072. https://doi.org/10.1016/j.ijproman.2013.12.006        [ Links ]

Etikan, I., Musa, S. A. & Alkassim, R. S. (2016). Comparison of convenience sampling and purposive sampling. American Journal of Theoretical and Applied Statistics, 5(1), 1-4. https://doi.org/10.11648/j.ajtas.20160501.11        [ Links ]

Fedele, G., Fedriga, M., Zanuso, S., Mastrangelo, S. & Di Nocera, F. (2017). Can user experience affect buying intention? A case study on the evaluation of exercise equipment. Human Factors and Ergonomics Society Europe, 145-152.

Feng, L. & Wei, W. (2019). An empirical study on user experience evaluation and identification of critical UX issues. Sustainability, 11(8), 2432. https://doi.org/10.3390/su11082432        [ Links ]

Gardey, J. C. & Garrido, A. (2020). User experience evaluation through automatic A/B testing. 25th International Conference on Intelligent User Interfaces Companion.

Gartner. (2019). Gartner says global IT spending to grow 0.6% in 2019 [Last accessed: 20 Jun 2021]. https://www.gartner.com/en/newsroom/press-releases/2019-10-07-gartner-says-global-it-spending-to-grow-06-in-2019

Graefe, T. M., Keenan, S. L. & Bowen, K. C. (2003). Meeting the challenge of measuring return on investment for user centered development. CHI'03 Extended Abstracts on Human Factors in Computing Systems.

Hassenzahl, M. (2008). User experience (UX): Towards an experiential perspective on product quality. 20th International Conference of the Association Francophone d'Interaction Homme-Machine.

Hovey, M. (2002). Is timeshare ownership an investment product? Journal of Financial Services Marketing, 7(2), 141-160.         [ Links ]

Kahn, M. J. & Nelling, E. F. (2010). Estimating the value of medical education: A net present value approach. Teaching and Learning in Medicine, 22(3), 205-208. https://doi.org/10.1080/10401334.2010.488206        [ Links ]

Klein, H. K. & Myers, M. D. (2001). A classification scheme for interpretive research in information systems. Qualitative Research in IS: Issues and Trends (pp. 218-239). IGI Global.

Kolbeinsson, A., Lindblom, J. & Thorvald, P. (2020). Employing UX processes for more focused development in research projects. 6th International Digital Human Modeling Symposium. https://doi.org/10.3233/ATDE200036

Layard, R. & Glaister, R. (1994). Cost-benefit analysis. Cambridge University Press.

Lyneis, J. M. (1975). Designing financial policies to deal with limited financial resources. Financial Management, 13-24.

Minge, M. & Thüring, M. (2018). Hedonic and pragmatic halo effects at early stages of user experience. International Journal of Human-Computer Studies, 109(13-25). https://doi.org/10.1016/j.ijhcs.2017.07.007        [ Links ]

Paunovic, G. (2017). The bottom line: Why good UX design means better business [Last accessed: 20 Jun 2021]. https://www.forbes.com/sites/forbesagencycouncil/2017/03/23/the-bottom-line-why-good-ux-design-means-better-business/

Ramdoo, V. & Huzooree, G. (2015). Strategies to reduce rework in software development on an organisation in Mauritius. International Journal of Software Engineering & Applications, 6(5), 9-20. https://doi.org/10.5121/ijsea.2015.6502        [ Links ]

Reichheld, F. F. (2003). The one number you need to grow. Harvard Business Review, 81(12), 46-55.         [ Links ]

Rohn, J. A. (2007). How to organizationally embed UX in your company. Interactions, 14(3), 25-28.         [ Links ]

Rosenberg, D. (2004). The myths of usability ROI. Interactions, 11(5), 22-29.         [ Links ]

Schueler, J. & Loveder, P. (2020). Understanding the return on investment from TVET: A practical guide. National Centre for Vocational Education Research (NCVER).

Schultz, T. W. (1961). Investment in human capital. The American Economic Review, 51(1), 1-17.         [ Links ]

Standish Group. (2015). Chaos report 2015 [Last accessed: 20 Jun 2021]. https://www.standishgroup.com/sample_research_files/CHAOSReport2015-Final.pdf

Sward, D. & Macarthur, G. (2007). Making user experience a business strategy. AMCIS 2007 Conference.

Terre Blanche, M., Durrheim, K. & Painter, D. (2006). Research in practice: Applied methods for the social sciences. Juta & Company Ltd.

Turner, C. W. (2011). A strategic approach to metrics for user experience designers. Journal of Usability Studies, 6(2), 52-59.         [ Links ]

Walsh, T., Varsaluoma, J., Kujala, S., Nurkka, P., Petrie, H. & Power, C. (2014). Axe UX: Exploring long-term user experience with iScale and AttrakDiff. 18th International Academic Mindtrek Conference: Media business, management, content & services, 32-39.

Weinschank, S. (2005). Usability: A business case [Last accessed: 20 Jun 2021]. https://humanfactors.com/downloads/whitepapers/business-case.pdf

Yin, R. K. (2011). Applications of case study research (3rd edition). Sage.

Young, S. W. H. (2014). Improving library user experience with A/B testing: Principles and process. Weave: Journal of Library User Experience, 1(1). https://doi.org/10.3998/weave.12535642.0001.101        [ Links ]

 

 

Received: 8 February 2021
Accepted: 28 May 2021
Available online: 12 July 2021

Creative Commons License All the contents of this journal, except where otherwise noted, is licensed under a Creative Commons Attribution License