GENERAL ARTICLES

 

Development of a maturity model for telemedicine^#

 

 

L. van Dyk^{I, *} ;C.S.L. Schutte^II

^IDepartment of Industrial Engineering Stellenbosch University, South Africa. lvd@sun.ac.za
^IIDepartment of Industrial Engineering Stellenbosch University, South Africa. corne@sun.ac.za

 

 

---

ABSTRACT

For more than a decade, the South African National Department of Health (DoH) has recognised the potential benefit of information and communication technology (ICT) in the delivery of health care to rural areas. Despite generous funding and proven technology, not many telemedicine systems have proved sustainable after the pilot phase. The purpose of this paper is to develop a maturity model that can be implemented to measure and manage the capability of a health system, for use in the delivery of sustainable health care after the pilot phase of a telemedicine project. The validity of the telemedicine maturity model (TMMM) is tested within the context of the South African public health sector.

---

OPSOMMING

Die Suid Afrikaanse Nasionale Departement van Gesondheid het reeds meer as 'n dekade gelede die voordeel besef wat inligtings- en kommunikasietegnologie kan bied ten opsigte van die lewering van gesondheidsorg in afgeleë gebiede. Ten spyte van ruim befondsing en bewese tegnologie, is daar egter min volgehoue telegeneeskundedienste in die publieke gesondheidstelsel van Suid Afrika. Die doel van hierdie artikel is om 'n volwassenheids-model te ontwikkel wat gebruik kan word om die vermoë van 'n gesondheidstelsel te bepaal en bestuur, ten einde telegeneeskunde loodsprojekte vol te hou. Die geldigheid van hierdie telegeneeskunde volwassenheidsmodel (TMMM) is getoets binne konteks van die publieke gesondheidsektor van Suid Afrika.

---

 

 

1. INTRODUCTION

The purpose of this paper is to develop a maturity model for telemedicine, and to validate this within the context of the South African health sector. To introduce this study, the nature and origin of telemedicine is considered, together with telemedicine applications within the South African context. This is followed by a discussion of the origin and application of maturity models.

1.1. Telemedicine in South Africa

In 1906, a Dutch physiologist published the results of experiments that recorded - with the help of a string galvanometer and a telephone line - the electrical cardiac signals of patients in a hospital one and a half kilometres away [1]. This is considered by many as the first example of telemedicine. The term 'telemedicine' was first coined in 1970, and refers to the delivery of healthcare services (medicine) over distance (tele). Although the use of information and communication technology (ICT) is not included as such in this definition, the development of telemedicine is so intertwined with the development of ICT that reference to ICT is found in almost all definitions of telemedicine.

Of all telemedicine specialisations, teleradiology applications are most often successfully integrated into health systems. In South Africa, most private hospital groups and some networks of public hospitals have fully functional teleradiological services. Typically, digital radiological images are uploaded to a picture archiving and communication system (PACS), and then added to existing health information or previous images for a specific patient. This information can then be viewed and shared among radiographers, radiologists, and other clinicians irrespective of their location.

Less complex telemedicine applications are found in teledermatology. For example, a clinic nurse takes a photo with a mobile phone. The picture is then sent via MMS to the dermatologist, situated elsewhere, who responds via SMS, e-mail, or telephone. Other telemedicine specialisations include tele-ophthalmology, tele-audiology, telecardiology, telepsychiatry, telesurgery, telemonitoring, and tele-rehabilitation [2].

In 1998, the South African national telemedicine strategy was developed and published by the National Department of Health [3]. This strategy focused on improving the accessibility of specialist health care in the rural areas of the country. Since then, the National Department of Health (DoH) has launched many telemedicine projects in the public health sector. However, many of these projects were not implemented successfully, or were not sustained after implementation. Many telemedicine devices are locked up in storerooms, or await maintenance [3], or are being used for a purpose other than the one for which they were originally developed.

Apart from the obvious waste of equipment and human resources, Yellowlees [4] considers the damage to the reputation of telemedicine as an even greater loss. The South African public health sector is already paying a price: in 2010, the Department of Health placed a moratorium on the launching of any new telemedicine projects until a strategy was in place to raise their success rate. Possible reasons identified by some authors for the low success rate of telemedicine projects in South African public health sectors are: lack of support from health professionals, and lack of technical support, training, and site-coordinators [3]. Many international studies acknowledge the importance and challenge of identifying benchmarks that are suitable for evaluating and measuring the success rate of telemedicine projects. These are vital for facilitating the success, sustainability, and optimisation of telemedicine services [5,6,7,8].

1.2. Maturity models

In many respects, telemedicine projects experienced similar problems to those of military projects that the United States implemented during the 1980s. These projects, which involved software contractors, ran over budget, and were completed far later than planned - if at all. To address this, the US Defense Software Engineering Institute (SEI) developed a process maturity framework to aid in evaluating the capability of the software contractors. It was used as part of the contract awarding process [9]. This was the beginning of the development of dozens of so-called maturity models that were applied in various domains and contexts. These models provide both a way of measuring the status quo, and of facilitating an improvement process that best suits the enterprise, while remaining within the prescribed best practice parameters of the particular domain [10].

Maturity models are no longer directed exclusively at the evaluation of software vendors or software development processes. They are now commonly used as a means of benchmarking, self-assessment, change management, and organisational learning [9,11]. The key design features of the maturity models adopted for this study are listed below [11,12,13].

• They are relevant to the organisational structure, culture, and working practices.
• They align the strategic and tactical priorities of the organisation.
• They are independent of the technology and the vendor.
• They consider both technical and non-technical IT infrastructure capabilities.
• They are easy to maintain and extend according to changes in an organisation's needs and priorities.
• They are simple and intuitive to use, and are written in plain English without technical jargon.
• They can be used for internal and external benchmarking, self-assessment, change management, and organisational learning.

 

2. METHODOLOGY

It is proposed that a maturity model for telemedicine would contribute to the success and sustainability of telemedicine projects implemented in South Africa. The purpose of this paper is thus to develop a maturity model for telemedicine, and to validate this within the context of the South African health sector.

The structure of this article is shown in Figure 1. The theoretical frameworks on which this model is based are defined in terms of the general origin and application of maturity models (previous section), and of existing frameworks and instruments within the context of the telemedicine domain (next section). These are then used as input for the design of the telemedicine maturity model (TMMM) proposed in this study. After the presentation of the model design, the validity of this model within the South African context is tested, using focus-group discussions with people from Provincial departments of health that are involved in existing telemedicine services or projects.

 

 

3. TELEMEDICINE AND E HEALTH READINESS AND MATURITY MODELS

Many models are useful in developing the telemedicine maturity model (TMMM). For the purposes of this study, four models were identified as having the most to contribute to such a maturity model. In this section, each of these models is discussed, highlighting the characteristics most relevant to the development of the TMMM. Since no single framework can serve as an effective model, an explanation justifying the development of the TMMM will also be provided.

3.1. Existing health service maturity models

3.1.1 NHS infrastructure maturity model (NIMM)

The NIMM is an IT infrastructure maturity model developed by the NHS Technology Office together with a number of other IT organisations in the United Kingdom. The nHs team worked closely with Atos Healthcare, a consultant company, helping to define and develop the NIMM [12]. The NIMM consists of two sub-models, each with its own categories:

Business sub-model

• Governance
• Procurement
• Financial management
• Business alignment
• People and skills
• Principles, standards, procedures, and guidelines

Technology sub-model

• End-user devices
• Common applications and services
• Operating systems
• Infrastructure and hardware platforms
• Network devices and services
• IT security and information governance

3.1.2 The PACS maturity model

Around the time of NIMM's development, Van Wetering et al. [11] recognised the potential benefit of maturity models for healthcare services. With teleradiology as their specific area of focus, they developed a picture archiving and communication system (PACS) model. However, the PACS maturity model does not include different categories or sub-models. Instead, each level of maturity addresses certain issues related to the sub-models and categories of the NIMM. For example, PACS process redesign, health information systems integration, and technological adoption are attributes of levels 2, 3, and 4 respectively.

3.2. Evaluation frameworks for telemedicine and eHealth

3.2.1 Layered telemedicine implementation model

South Africa is not the only country to experience significantly high failure rates in its telemedicine projects. An international study by Broens et al. [6] confirmed that telemedicine projects after the prototype phase are more likely to fail than prove sustainable. They conducted a systematic literature review to answer the question: "Why is it so difficult [to implement telemedicine], and what goes wrong?" In this study, Tanriverdi and Iacono's theoretical model was used to identify the determinants for the successful implementation of telemedicine.

Broens et al. [12] postulated that different determinants become applicable as telemedicine implementation maturity is gained, and their layered implementation model was developed accordingly. The relationship between each implementation layer and its associated determinants (in brackets) is shown in Figure 2.

 

 

3.2.2 eHealth readiness instruments

eHealth readiness is defined as the "degree to which users, healthcare institutions and the healthcare system itself, are prepared to participate and succeed with implementation" [7]. Jennett et al. [14] specifically refer to eHealth readiness when arguing that time, money, and energy can be saved if the status of an eHealth/telemedicine system context is determined before implementation. Legare et al. [15] identified six different assessment tools that can be used to measure e-readiness within a health context. Information about internal validity and reliability is only available for two of these tools. Khoja's eHealth readiness assessment tool [7] was selected for the purposes of this paper because it is aimed at developing countries.

The instrument covers five categories, each containing a number of statements with which a respondent is asked to agree or disagree, using a five-point Likert scale:

• core readiness (21 statements) deals with aspects of planning and integration;
• technological readiness (10 statements) considers the availability, reliability, affordability of ICT and related infrastructure;
• learning readiness (six statements) addresses the programmes and resources available to provide training in the use of technology;
• societal readiness (11 statements) considers the interaction between various institutions in the region and beyond, including socio-cultural factors; and
• policy readiness (12 statements) deals with policies already in place at government and institutional level, to address common issues such as licensing, liability, and reimbursement.

3.3. Analysis of existing models

Essmann [10] explains that a maturity model's first objective is to establish the capability maturity of an organisation in terms of a specific domain of practice. The strength of the eHealth readiness instrument lies in the fact that it provides us with a set of statements that can be used to measure an organisation's eHealth readiness. The drawback of eHealth readiness tools is that they do not accommodate the second purpose of a maturity model; that of describing the best practices of the domain, and then facilitating the best way of achieving this.

The layered implementation model [6] recognises that a different determinant for successful telemedicine implementation becomes applicable as maturity is gained; but it does not provide a way of evaluating maturity or of benchmarking.

The NHS maturity model [12] was developed in conjunction with all ICT-supported health services of the NHS. Telemedicine services are a subset of this; but the domain of this maturity model also includes health services that are not necessarily delivered over a distance. The domain of the PACS maturity model [13] is teleradiology - a subset of telemedicine. It is clear that a new maturity model needs to be designed to accomplish the purpose of this study. The development of such a model will be explained in the remainder of this article.

 

4. DEVELOPMENT OF A TELEMEDICINE MATURITY MODEL (TMMM)

The TMMM has three dimensions, as shown in Figure 3. Each of these dimensions is discussed individually in the subsections below.

 

 

4.1. First dimension: Maturity categories

This dimension addresses the following key design features set out earlier in this paper:

• it is independent of the technology and the vendor.
• it is simple and intuitive to use, and is written in plain English without technical jargon.
• it considers both technical and non-technical IT infrastructure capabilities; and
• it is relevant to the organisational structure, culture, and working practices.

Some common themes are obvious in the five eHealth readiness categories [7]: the five determinants for telemedicine implementation [6], the categories of the two NIMM submodels [12], and the implicit maturity indicators of the PACS maturity model [13]. For example, technology is a theme found in all the frameworks, while policy/governance features are found in three. Five new categories were developed for the TMMM, encompassing all the aspects of the theoretical frameworks, and adhering to the design features stated above:

• Machine: Technology
• Money: Finances
• Man: Users
• Method (operations): Work protocols
• Method (strategy): Policy

These maturity categories are derived from the existing frameworks described earlier; both Khoja [7] and Broens et al. [6] recognise technology and policy as determinants for the successful implementation of telemedicine. Technology is also a specific category in the NHS infrastructure maturity model [12], and is implied in the PACS maturity model [13]. Financing is recognised in both the layered implementation model [6] and the NHS infrastructure maturity model [12]. The users category is aligned with categories from other models, such as people and skills [12], learning readiness [7], and user acceptance. Finally, the procedures are related to organisation [6] and to principles, standards, procedures, and guidelines [12].

4.2. Second dimension: Maturity levels

This dimension addresses two design features set out earlier in this paper:

• it can be used as a means of internal and external benchmarking, self-assessment, change management, and organisational learning; and
• it aligns the strategic and tactical priorities of the organisation.

In a maturity model, the current maturity level is measured initially by how many other levels serve as a guide to system maturity. Most maturity models show five maturity levels [10]. These generic levels correspond well with the levels of the NIMM [12], and to a lesser extent with the PACS maturity model [11], and were adopted for the purposes of this study (see below). The NIMM level descriptors appear in brackets.

• Level 1: initial, ad hoc process (basic);
• Level 2: managed, stable process (controlled environment);
• Level 3: defined, standardised process (consistent execution);
• Level 4: measured process (quality and productivity); and
• Level 5: optimising (continuous improvement).

Table 1 shows the performance indicators for each intercept of the maturity category and maturity level dimensions.

 

 

4.3. Third dimension: Telemedicine process

This dimension addresses one of the key design features set out earlier in this paper: it is easy to maintain and extend according to changes in an organisation's needs and priorities.

Most maturity models focus on the maturity of processes within a certain organisation and enterprise. Telemedicine, by definition, crosses the boundaries of organisations, enterprises, and even jurisdictional borders and countries. In fact, each specific process frequently crosses a different boundary. For a successful telemedicine process to take place, irrespective of the context or required technology, each step in the telemedicine process needs to be successfully executed [16].

Figure 4 shows a typical teleradiology process. In this example, if a radiographer is not trained to use the picture archiving and communication system (PACS), there is a risk that the radiological image will not be successfully uploaded to the PACS. If this were so, the image could not be transmitted and retrieved by the specialist and no diagnosis could be made, packaged, transmitted, or retrieved. The telemedicine service would therefore be ineffective. And if the image is not packaged according to the digital imaging and communications in medicine (DICOM) standard, it may be impossible to retrieve the diagnosis. A telemedicine maturity model (TMMM) should thus facilitate the evaluation and optimisation of each step of this process. For this reason, the telemedicine process forms the third dimension (z-axis) of the maturity model (Figure 3).

 

 

The maturity assessment procedure involves gauging the maturity levels for each step of the telemedicine process. For example, Figure 5 shows the maturity assessment of the teleradiology process of Figure 4. This assessment was done by a representative group from a specific South African Provincial department of health where teleradiology is standard practice (Level 3). The results of this self-assessment indicate that training of users is a priority.

 

 

At the same Provincial department of health, a dermatologist regularly uses mobile phone technology to treat his existing patients (Figure 6). However, this is not standard practice, as is evident from the assessment of this process (Figure 7). The process may be effective, but the maturity level of all the elements of this teledermatology service is low. This service is therefore not likely to be sustainable.

 

 

 

 

5. VALIDATION AND CONCLUSION

Between June and December 2011, four telemedicine workshops were facilitated in collaboration with the Medical Research Council (MRC) of South Africa. Each workshop was held in a different region of South Africa, and involved a group of medical practitioners (doctors, nurses, radiologists etc.) along with people involved in the provision of IT services and infrastructure for a specific region.

The first day of these two-day workshops was used to educate delegates on the origin and current practices relating to telemedicine, including the articles discussed in this paper. On the second day, the workshop attendees were taken through a process in which they were asked to map out the telemedicine processes related to their environment, and then to use the maturity model proposed in this article to undergo a self-assessment.

Each workshop concluded with a focus group discussion, the results of which were used to further develop this maturity model. The focus group questions were aligned with the key design features, based on the literature presented earlier in this paper. A reflection on the extent to which the model adheres to the key design features was then used to confirm the validity of the TMMM proposed in this paper.

5.1 It aligns the strategic and tactical priorities of the organisation.

Although all the delegates (medical practitioners and IT managers) agreed that this maturity model contributes to tactical and strategic alignment, the IT managers valued this design feature far more than the medical practitioners. This may be because medical practitioners are involved with health services at an operational level, while IT managers are much more concerned with technical and strategic alignment. The medical practitioners did, however, appreciate the fact that through working with IT managers on the self-assessment exercise, they gained an insight into the alignment of their daily operations with the strategies and tactics of the organisation.

5.2 It is easy to maintain and extend according to changes in an organisation's needs and priorities.

At present, the implementation and maturation of teleradiology services is a priority in all Provinces. However, as was shown in the previous section, the TMMM can be effectively applied to teledermatology, and indeed to any other telemedicine specialisation.

5.3 It is relevant to organisational structure, culture, and working practices.

This model was presented at four workshops attended by delegates with different structures, culture, and working practices from four different Provincial departments of health. The applicability of the model in the private health sector has not yet been examined, and should therefore be considered in future research.

5.4 It considers both technical and non-technical IT capabilities.

The workshop delegates (medical practitioners and IT managers alike) agreed that all the capabilities had been addressed in the TMMM.

5.5 It is simple and intuitive to use, and is written in plain English without technical jargon.

Even though time was limited, each group had an opportunity to consider all the maturity indicators for their selected telemedicine process. The maturity model was declared to be easy to use, although concern was raised about the level of difficulty in its use if a skilled facilitator were not present, as was the case at the workshop. Developing an interface or scorecard to enhance the intuitive use of the model may be a consideration for the future.

5.6 It can be used as a means of internal and external benchmarking, self-assessment, change management, and organisational learning.

Change management was identified in the literature, and by the DoH representatives, as the key to the successful implementation of telemedicine. The TMMM could thus be instrumental in managing this change. Although it is simple and intuitive to use (one of the design features of a maturity model), it provides users with a tool to assess and manage the entire context of their telemedicine initiatives.

As with simulation modelling, the challenge here was to produce a model that is "transparent [and] that facilitate[s] stakeholders in validating and understanding key decision variables, their workings, and model output" [17]. Delegates agreed that the value of this workshop lay not only in the measureable outcomes, but in the fact that different role players, with diverse viewpoints, communicated their opinions based on a common holistic framework. Stakeholders contribute their domain knowledge to create solutions, and in doing so, greater user acceptance is achieved [18].

The purpose of this study was to develop a telemedicine maturity model (TMMM) and to validate this within the context of the South African public health sector. Existing eHealth readiness tools, maturity models, and telemedicine implementation frameworks were used to develop a maturity model that adheres to the required design features. It was developed and validated as an iterative process, and involved 51 people from South Africa's public health sector who can contribute to releasing the potential of telemedicine to accomplish the vision of quality healthcare for all citizens.

 

6. ACKNOWLEDGEMENTS

This research would not have been possible without the support of, and interaction with, Jill Fortuin, director of Telemedicine and mHealth at the Medical Research Council of South Africa. Furthermore, this material is based on work supported financially by the National Research Foundation.

 

REFERENCES

[1] Strehle, E.M. & Shabde, N. 2006. One hundred years of telemedicine: Does this new technology have a place in paediatrics?, Archives of Disease in Childhood, 91(12), 956-959.         [ Links ]

[2] Van Dyk, L. 2010. Telemedicine - Leverage competitive advantage through the use of ICT investment, in Annual Conference of the Southern African Institute for Industrial Engineering, Johannesburg.         [ Links ]

[3] Mars, M. 2009. Telemedicine in South Africa, in Telehealth in the Developing World, Ottawa, OECD, 2009, 222-231.         [ Links ]

[4] Yellowlees, P. 2005. Successfully developing a telemedicine system, Journal of Telemedicine and Telecare, 11(7), 331-336.         [ Links ]

[5] Bashshur, R., Shannon, G., & Sapci, H. 2005. Telemedicine evaluation, Telemedicine Journal & e-Health, 11(3), 296-316.         [ Links ]

[6] Broens, T., Huis in't Veld, R.M.H.A., Vollenbroek-Hutten, M.M.R., Hermens, H.J., Van Halteren, A.T. & Niewenhuis, L.J.M. 2007. Determinants of successful telemedicine implementations, Journal of Telemedicine and Telecare, 6(13), 303-309.         [ Links ]

[7] Khoja, S., Scott, R., Casebeer, A., Mohsin, M., Ishaq, A. & Gilani, S. 2007. e-Health readiness assessment tools for healthcare institutions in developing countries, Telemedicine and e-Health, 13(4), 425-432.         [ Links ]

[8] Whiten, P. 1997. The diffusion of telemedicine, Science Communication, 19(1), 21-40.         [ Links ]

[9] Wikipedia. 2011. Capability maturity model, en.wikipedia.org/wiki/Capability_ Maturity_Model. Accessed 7 December 2011.         [ Links ]

[10] Essman, H. 2009. Toward innovation capability maturity, PhD thesis, Stellenbosch University.         [ Links ]

[11] Kirrane, J. 2009. A maturity model for continuous quality improvement of a clinical information system used in critical care medicine, College of Business, Public Policy and Law, National University of Ireland, Galway, Ireland.         [ Links ]

[12] Savidas, A. 2009. Your guide to the NHS infrastructure maturity model, Informatics Directorate, Policy and Planning, Informatics Planning, United Kingdom.         [ Links ]

[13] Van de Wetering, R. & Batenburg, R. 2009. A PACS maturity model: A systematic meta-analytic review on maturation and evolvability of PACS in the hospital enterprise, International Journal of Medical Informatics, 78, 127-140.         [ Links ]

[14] Jennett, P., Gagnon, M. & Brandstadt, H. 2010. Readiness models for rural telehealth, Journal of Postgraduate Medicine, 51(4), 279-283.         [ Links ]

[15] Legare, E., Vincent, C., Lehoux, P., Anderson, D., Kairy, D. & Gagnon, M. 2010. Telehealth readiness assessment tools, Journal of Telemedicine and Telecare, 16(3), 107-115.         [ Links ]

[16] Van Zyl, A.J. & Van Dyk, L. 2011. A decision support tool for telemedicine project management, in Proceedings of the 2nd Southern African Telemedicine and eHealth Conference, Cape Town, 50.         [ Links ]

[17] Van der Zee, D. & van der Vorst, J.G.A.J. 2007. Guiding principles for conceptual model creation in manufacturing simulation, Proceedings of the 2007 Winter Simulation Conference, 776-784.         [ Links ]

[18] Van der Zee, D. 2009. Developing participative simulation models - Framing decomposition principles for joint understanding, Journal of Simulation, 1, 187-202.         [ Links ]

 

 

* Corresponding author.
# This article is an extended version of a paper presented at the 2011 ISEM conference.