SciELO - Scientific Electronic Library Online

 
vol.27Pilot Testing of an Information Extraction (IE) Prototype for Legal ResearchBook Review: International Telecommunications Law and Policy author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Article

Indicators

Related links

  • On index processCited by Google
  • On index processSimilars in Google

Share


The African Journal of Information and Communication

On-line version ISSN 2077-7213
Print version ISSN 2077-7205

AJIC vol.27  Johannesburg  2021

http://dx.doi.org/10.23962/10539/31368 

ARTICLES

 

Reviewing a Decade of Human-Computer Interaction for Development (HCI4D) Research, as One of Best's "Grand Challenges"

 

 

Judy van BiljonI; Karen RenaudII

IProfessor, and SARChI Chair in Information and Communication Technology for Development (ICT4D), School of Computing, College of Science, Engineering and Technology, University of South Africa (UNISA) Science Campus, Johannesburg https://orcid.org/0000-0002-4646-1641
IIChancellor's Fellow, University of Strathclyde, Glasgow; and Professor Extraordinarius, University of South Africa (UNISA), Johannesburg https://orcid.org/0000-0002-7187-6531

 

 


ABSTRACT

The human-computer interaction for development (HCI4D) field emerged at the intersection of the fields of information and communication technology for development (ICT4D) and human-computer interaction (HCI). In 2010, Michael Best nominated HCI4D as one of ICT4D's "grand challenges". This HCI4D field is now entering its second decade, and it is important to reflect on the research that has been conducted, and to consider how HCI4D researchers have addressed the challenge that constitutes the raison d'etre of HCI4D's existence. Best provided four guidelines to inform researchers embracing this challenge. This study commences by identifying the primary HCI4D-specific themes, and then carries out a systematic literature review of the HCI4D literature to build a corpus to support the analysis. The corpus is analysed to reflect on how well the field's practices align with Best's guidelines. The overall finding is that HCI4D researchers have largely been following Best's guidelines and that the HCI4D field is demonstrating encouraging signs of emerging maturity.

Keywords: Human-computer interaction for development (HCI4D), information and communication technology for development (ICT4D), human-computer interaction (HCI), guidelines, Michael Best


 

 

1. Introduction

In 2010, Michael Best, the internationally recognised computer scientist and scholar of information and communication technology for development (ICT4D), contemplated the necessities facing the ICT4D field. In a brief theoretical contribution published in the Information Technologies & International Development (ITID) journal, Best (2010) enumerated four "grand challenges" that the ICT4D field needs to address to prove itself a progressive intellectual enterprise (2010, p. 52): (1) sustain-ability, (2) post-conflict and disaster computing, (3) HCI4D, and (4) appliances.

Human-computer interaction for development (HCI4D) is a multi-faceted field that focuses on understanding and designing technologies for under-served, under-resourced, and under-represented populations in a variety of geographic regions (Dell & Kumar, 2016). The HCI4D field, like its parent fields-information and communication technology for development (ICT4D) and human-computer interaction (HCI)-has enjoyed much attention, as evidenced by the significant and burgeoning number of research publications that have appeared since Best (2010) posed the challenges.

Best argued that, in order to address the challenges, researchers ought to: (1) return to the field's "interdisciplinary and holistic roots", (2) avoid "fetishistic techno-uto-pianism", (3) focus on "fundamental innovation" through "multi-year initiatives", and (4) develop "fundamental shared problems" and tackle the problems with "mixed methods" and strong "evaluation and assessment" (2010, p. 51). A decade has passed since Best published his seminal article, and it seems an appropriate time to gauge how well researchers have responded to the imperatives he proposed. Accordingly, this study, a systematic literature review, probes this core question: To what extent have HCI4D researchers followed Best's four guidelines in carrying out the research in this ICT4D challenge area?

 

2. Research design

The systematic literature review (SLR) was carried out to construct a corpus of HCI4D research published from 2009 to 2019. SLRs are a recognised method of supporting analysis of the literature in a research domain (Paré et al., 2015). The aim thereof is to go beyond merely aggregating existing evidence and to construct evidence-based lessons from the accumulated research (Kitchenham et al., 2009; Grant & Booth, 2009). The processes of data collection, data condensation, data display, drawing conclusions, and verifying conclusions are applied, either allowing the publications to suggest themes (open coding), or using pre-existing themes to constrain and inform the analysis (directional coding) (Miles et al., 2019). Over the past 11 years (2009 to 2019), several researchers have published surveys of the HCI4D literature. We extracted the dimensions and overarching themes from this existing review literature (see Table 1), and used those themes to conduct directional coding analysis, as advised by Bramer et al. (2018).

Different acronyms have been used to refer to research focusing on human-centred design aiming to foster socio-economic development. We acknowledge the ambiguity and even controversy surrounding the term "development", and the fundamental duality between those studies that focus on understanding technology "for development" (where there is a commitment to human and socio-economic development) and those studies that focus on understanding technology in developing countries. Despite the academic discourse, and the critique that the terms "development" and "for development" introduce ambiguity (Toyama, 2010), the term "HCI4D" has survived, as evidenced by increasing research in this area that uses this term in the publication titles or as keywords. This publication-based measure admittedly does not reflect the adoption of the term by government, industry, and the broader community, but that is beyond the scope of this paper.

the same approach in reviewing the topic. We followed the systematic literature review based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) method (Moher et al., 2009), which resulted in the corpus of literature to support the analysis.

The Scopus database was selected as the search database since it includes many of the titles where the ICT4D and HCI4D conference proceedings are published. The review was conducted in the week of 18 February 2020, using the keyword "HCI4D" and the period 2009 to 2019. The SLR excluded all items that were not peer-reviewed, and we excluded patents and magazine articles. That produced 230 publications, which included items-proceedings of workshops and symposiums, abstracts from books, and bulletins published as conference proceedings-in which the format was condensed and hence not comparable to the other pieces of literature. Removing these items, 176 publications remained to support analysis: 171 conference publications and five journal articles.1 Using only the term "HCI4D" might be considered a limitation due to the diversity surrounding "4D" terminology. However, given that Best's (2010) grand challenge specifically refers to HCI4D, it was decided that only this single keyword should be used.

 

3. Findings

When: Annual publication levels

Grouping the HCI4D publications in terms of number per year, as depicted in Figure 1, reveals that the annual number has increased since 2009 (from 1 in 2009 to 41 in 2019) with a dramatic increase in 2013, a decline in 2014, and a less obvious decline in 2017, followed by a strong rise in both 2018 and 2019. The reasons for the strong increase in 2013 are debatable but might be attributable to the convergence of a number of biannual conferences as well as the first HCI4D event being organised at CHI2013 that year.

 

 

Where: Research and author locations

Figure 2 depicts the geographic locations of the research reported in the 176 items reviewed. It shows that the largest amount of HCI4D research covered in the publications has been done in India (65), with Pakistan (13) and Kenya (13) in second place, followed by Bangladesh (10) in third.

It should be noted that publications focused on developing-world research often have authors based in developed countries (see Bai, 2018; Van Biljon & Renaud, 2019). Figure 3 demonstrates that most (67.5%) of the lead authors in our corpus of literature are based in the US, Europe, Australia, and Canada, i.e., outside the developing world. (If an author worked on five separate publications, they would be counted five times-the graph represents the authorship of the corpus, not the authorship of the HCI4D field.)

 

 

Why: Research domains

As seen in Figure 4, the health domain attracted the most attention (28%) in the HCI4D literature in our corpus, followed by education (24%) and gender (18%). The aim was to categorise publications according to the dominant domains, but two domains were selected for some publications. For example, a publication on the use of an app to teach about a medical condition, where the research involved both health and education, was counted in both categories, such that the total number of domain selections is 349 while the total number of publications is only 176. This means that the percentages provided per domain include overlap. Furthermore, some of the domains that emerged, such as community, Data4Dev, interactive voice response (IVR) and access, might not be considered independent domains, and migration studies might be considered a discipline rather than a domain. This visualisation is provided as a snapshot provided primarily to demonstrate the diversity and reach of the research reported in the corpus.

How: Research designs and data collection methods

Figure 5 provides an overview of the methodological information reported in the 176 items of literature. Ethnography, experiment, action research, and case study were found to be the most common research designs, while interviews, observations, and surveys were the most common types of data collection strategies. Given the interdisciplinarity of HCI, there is little standardisation on methodological terminology. For example, some researchers refer to a survey as a research design while others consider it a data capturing method. To avoid misclassification, we do not separate research design and data capturing methods.

Alignment with Best's four guidelines

We now set out the findings from the analysis of the corpus in respect of alignment with each of Best's (2010) four guidelines. As depicted in Figure 6, we split the fourth guideline into two parts for the purposes of our analysis, with 4a relating to the set of fundamental problems and 4b detailing the research methods used for each.

 

 

1: Is the field returning to interdisciplinary and holistic roots?

Interdisciplinarity occurs when researchers from diverse disciplines work together in carrying out a piece of research. To assess whether the HCI4D field is engaging in this kind of research, the following actions were carried out:

ascertain the discipline of each publication's authors (based on their institutional home);

group semantically similar disciplines (for example, Computer Science, Computer Science and Engineering, Computer Engineering and Software Engineering were considered to be one meta-category); and

tally the number of publications written by authors coming from each discipline.

The final percentages are provided in Figure 7. It is clear that Computer Science and Computer Science-aligned disciplines such as Information Systems, Information Science and Management, Human-Computer Interaction (HCI), and Education Technologies dominate, which is to be expected because HCI4D is grounded in Computer Science. Yet there is still evidence of a great deal of interdisciplinary work. Of the 176 publications, 109 (62%) have authors from at least two different disciplines, while 44 (25%) have authors from three different disciplines. (The "Other" group includes all disciplines which, on their own, make up less than 1% of the total. These disciplines are wide-ranging, and include International Relations, Television and Features, and Organic Chemistry.)

2:Is the f ieldavoiding techno-utopianism?

In order to evaluate this aspect, we focused on the maturity of research in the field- on the assumption that a mature field would be one that has moved beyond any one narrow focus, e.g., a focus on the benefits of technology. We categorised each item in the corpus in terms of type of publication and the artefact(s) produced in the publication. Van Biljon and Renaud (2018) use three cross-cutting meta-themes to assess the maturity of the mobile communication technology for development (M4D) field based on an analysis of the M4D conference publications. Adopting the same three purpose-related meta-themes as the basis for the analysis, the publications were categorised according to their stated purpose, as follows:

Foundational publications: Publications that seek to capture and describe the status quo towards understanding the user's needs, context, or use of technology. The research design often involves self-reporting during interviews, surveys, or focus groups where the context, needs, expectations, and aspirations are captured. N = 51 (29% of the items in the corpus).

Design or intervention publications: Publications that describe the design, implementation, and evaluation of applications (benefiting from the findings of theme 1 publications, but also extending them). The research design encompasses requirements gathering, artefact development, and evaluation based on users' self-reporting, observations, usability testing, or ethnography. The artefacts include prototypes of ICT systems and technology probes. N = 116 (67%).

Abstraction or theorisation publications: Publications that apply existing knowledge to implement ICT technology, replicate that knowledge in a new context, or extend existing research. That is, these are items that analyse, synthesise, and refine existing research in order to generate abstract new knowledge. Meta-view analyses identified trends and explicitly built on the extant literature, e.g., systematic literature reviews, analysis of trends towards suggesting models and theories, and making recommendations about the way ahead. N = 9 (5%).

Figure 8 shows the number of publications and citations per publication type and the year of the first publication of each type. The first foundational publication is from 2009; the first design or intervention publication is from 2010; and the first abstraction or theorisation publication is from 2014. This progression is to be expected because abstraction or theorisation publications reflect increased maturity in the field, i.e., the field needs to be mature enough to have generated significant research in the foundational and design or intervention categories if there is to be sufficient material to inform abstraction or theorisation publications.

 

 

These results indicate that the largest percentage of publications (67%) are those describing a design or intervention. That resonates with the real-world focus of the area. To gain insight into how many of these design and intervention publications produced a new system, the publications were sub-categorised in terms of their artefact contribution. It was found that 83 (47%) do not present any type of system as contribution. New digital systems are presented in 91 (52%) of the publications and non-digital artefacts (e.g., curricula) in two (1%) of the cases. New digital application systems are informed by both target user requirements and academic literature.

In the 91 studies where new systems are developed or tested, the results show that both users and the literature are consulted in 77 (85%) of the cases and only users are consulted in 12 (13%) of the studies. Only 2% of the publications are silent on requirements gathering. This implies that the contextual requirements are considered a priority in much of the HCI4D research, and obtaining end-user input seems to be treated as more important than consulting the research literature. The importance of context in HCI4D is supported by Patterson et al.(2009) and Abdelnour-Nocera and Densmore (2017), but the finding of potential lack of emphasis on engagement with the relevant literature is somewhat novel.

Considering the distribution of the types of publications and the contextual em-beddedness of the new technologies developed, it seems as if contextual needs are guiding development, so that techno-utopianism does not appear to dominate the HCI4D field. Resource constraints, which often manifest in the HCI4D context (Toyama, 2010; Dell & Kumar, 2016) necessitate the consideration of cost and sus-tainability. These resource constraints and contextual requirements mitigate the drive for using new technology, which could explain why techno-utopianism in less pronounced in HCI4D research than in HCI.

3: Is the field engaging in fundamental innovation through multi-year initiatives?

Kondrat'ev (1984) argues that fundamental innovations drive economic growth. A fundamental innovation enables other incremental research to improve, derive from, or extend it. Hence, a way to detect whether a field is embracing this principle is to look for multiple successive publications addressing the same innovation. If a publication introduces an innovation which then leads to multiple other publications extending the original research, it could be argued that the original publication's reported innovation has become fundamental.

To reveal fundamental innovation, two steps were carried out. First, a search for the publication was conducted using Google Scholar. Second, the "cited by" link was used to obtain the list of publications that have cited the source publication (as an indication of the degree to which the research is being built on). When it was found that the citing publication was written by one or more of the same authors as the source publication, and deals with the same topic, that was taken as indicating that one or more of the authors is engaged in a multi-year research initiative.

Of the 176 items in our corpus, 81 have been cited by subsequent publications by at least one of the original authors. This can be considered as evidence of multi-year initiatives. We identified 39 different multi-year publication profiles. These are distinct multi-year innovation research profiles. Figure 9 shows the 39 multi-year publication profiles. Some appear repeatedly, mostly towards 2019. This might well be an indication that the field is demonstrating increasing maturity, showing that researchers are starting to embrace the need for multi-year initiatives and acknowledging that genuine innovation can occur only when researchers extend research rather than continually re-invent.

4a: Is the field developing a set of shared research problems?

As seen in Figure 4 above, central themes have emerged, such as health, education, and gender. This provides evidence of shared research problems while other publications lie at the intersection of domains, e.g., the Yadav et al. (2019) study on the potential of chatbots for breastfeeding education links to both health and education.

4b: Is the field characterised by mixed methods and strong evaluation and assessment?

The most often-cited data collection method, as seen in Figure 5, is interviews, followed by observations (almost 50% less), and then by surveys. However, as is evident from Figure 8, a range of research designs and data capturing methods is being used. With respect to the evaluation component in the published research, 81 of the 91 publications that present a new digital system report an evaluation procedure. Hence most new digital systems (89%) were subjected to formal evaluation. However, it must be acknowledged that the rigour of the claimed evaluation could not always be deduced from the publications.

 

4. Analysis in terms of Best's guidelines

Returning to interdisciplinary and holistic roots

As seen in the findings, there are encouraging signs that researchers from a variety of disciplines are indeed working together, if we use the disciplines of the publication authors as evidence of this. Furthermore, there are suggestions that researchers in all realms of development-related research should work together, as a community (Ho & Veeraraghavan, 2008), and in multidisciplinary, interdisciplinary, or transdis-ciplinary ways (Walsham, 2017).

Avoiding the pitfalls of techno-utopianism

The findings suggest there is a tendency for researchers to design or build new interventions. This is typical of the problem-solving first phase of a new field (Winters & Toyama, 2009), which places primary importance on an intervention showing measurable improvement, but there is evidence of the values-first approach focusing on the construction of a shared perspective between researchers and the communities they research. However, only nine publications were found to be building on the extant literature. While it can be expected for a new field such as HCI4D to spend its first decade on foundational publications and design or intervention publications (the first two types in Figure 8), we anticipate uncovering evidence of greater maturity over the next 10 years, as per the emergence, as seen in Figure 8, of abstraction or theorisation publications in 2014.

Engaging in fundamental innovation through multi-year initiatives

Figure 9 demonstrates a number of multi-year research profiles, which shows that authors are not merely parachuting in to do a single study and then moving on. This is evidence of researchers' commitment to their own innovations. We have to consider this against the dynamic nature of technology development and funding realities where multi-year initiatives are often not practical despite the best intentions of researchers.

Developing shared problems, using mixed methods, and conducting robust evaluation and assessment

Figure 4 shows that a number of shared problems (research domains) have started to emerge, as judged by the attention they have garnered. It can be observed that health and education are enjoying the most attention towards the end of the corpus time span. Many other topics cluster at the bottom of this graph, indicating that there is not, at this stage, implicit acknowledgement that these are compelling shared problems. Still, a large and diverse number of topics are enjoying attention.

Summary

The findings suggest that research projects reported in the field of HCI4D are mostly aligned to Best's guidelines. To engender continuation of these positive trends, it would be helpful if conference chairs could ensure that calls for publications specifically include topics that encourage research building on existing research, and other publications aimed at maturing the field. For instance, they could create a category for "systematisation of knowledge" publications (which would feed into abstraction or theorisation publications, the third and most mature type in Figure 9), or require those who develop technology to provide their code on GitHub so that other researchers can make use of it.

Study limitations

Despite a rigorous approach enforced by the strict application of the SLR as method, any literature survey can be faulted for having made contestable decisions (both intentionally and unintentionally) about which works to include and which elements to emphasise in the works identified. We acknowledge the ambiguity and limitations surrounding the term "development" (Sen, 2001; Toyama, 2010) and the duality in the ICT and development research agenda (Brown & Grant, 2010). However, this article does not attempt to engage with the ongoing discussion on how development should be defined and whether we should distinguish between research conducted in the developing world, with people at the margins, and research conducted specifically for development. We also recognise the potential limitations of using "HCI4D" as the sole search term-a choice made, as explained earlier, because that is the term used by Best in the seminal 2010 publication. In addition, only one database, Scopus, was used to identify publications to feed into the review and the publications retrieved were mostly conference publications. It might well have been possible to identify a larger and more diverse range of publications if other databases had also been consulted.

 

5. Conclusion

As the field of HCI4D enters its second decade, a veritable adolescence, we offer, through this study, an analysis of HCI4D publications published 2009 to 2019, in order to present a snapshot of the field. Using Best's guidelines to structure the analysis, this snapshot is provided as an overview of the field, in order to take cognisance of the sophisticated and important work that has been done in this space. In doing systematic literature reviews in fields such as this one, it is necessary to accept that the field is characterised by a wide diversity of terminology and, notwithstanding such constraints, to still push forward with the review exercise in order to generate the available insights.

 

References

Abdelnour-Nocera, J., & Densmore, M. (2017). A review of perspectives and challenges for international development in information and communication technologies. In Annals of the International Communication Association, 41(3-4), 250-257. https://doi.org/10.1080/23808985.2017.1392252        [ Links ]

Bai, Y. (2018). Has the Global South become a playground for Western scholars in information and communication technologies? Scientometrics, 116(3), 2139-2153. https://doi.org/10.14207/ejsd.2014.v3n1p13        [ Links ]

Best, M. L. M. (2010). Understanding our knowledge gaps: Or, do we have an ICT4D field? And do we want one? Information Technologies & International Development, 6, 49-52.         [ Links ]

Bramer, W. M., De Jonge, G. B., Rethlefsen, M. L., Mast, F., & Kleijnen, J. (2018). A systematic approach to searching: An efficient and complete method to develop literature searches. Journal of the Medical Library Association, 106(4), 531-541. https://doi.org/10.5195/jmla.2018.283        [ Links ]

Brown, A., & Grant, G. (2010). Highlighting the duality of the ICT and development research agenda. Information Technology for Development, 16(2), 96-111.         [ Links ]

Chetty, M., & Grinter, R. (2007). HCI4D: How do we design for the Global South? Paper presented to User Centered Design and International Development Workshop at CHI, 28 April-3 May, San Jose, CA.

Dell, N., & Kumar, N. (2016). The ins and outs of HCI for development. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems - CHI '16 (pp. 2220-2232). https://doi.org/10.1145/2858036.2858081

Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information and Libraries Journal, 26(2), 91-108. https://doi.org/10.1111/j.1471-1842.2009.00848.x        [ Links ]

Ho, M. R., Smyth, T. N., Kam, M., & Dearden, A. (2009). Human-computer interaction for development: The past, present, and future. Information Technologies & International Development (ITID), 5(4), 1-18.         [ Links ]

Ho, M. R., & Veeraraghavan, R. (2008). Bridging the divide between theory and practice in ICT for development. IDEALS Conference, University of Illinois.

Kitchenham, B., Brereton, O. P., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009). Systematic literature reviews in software engineering - a systematic literature review. Information and Software Technology, 51(1), 7-15. https://doi.org/10.1016/j.infsof.2008.09.009        [ Links ]

Kondrat'ev, N. D. (1984). The long wave cycle. Richardson & Snyder.

Miles, M. B., Huberman, M., & Saldana, J. (2019). Qualitative data analysis: A methods sourcebook (4th ed.). Sage.

Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Prisma Group (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Annals of Internal Medicine, 151, 264-269. https://doi.org/10.1016/j.ijsu.2010.02.007        [ Links ]

Okoli, C. (2015). A guide to conducting a standalone systematic literature review. Communications of the Association for Information Systems, 37(1), 879-910. https://doi.org/10.17705/1CAIS.03743        [ Links ]

Paré, G., Trudel, M. C., Jaana, M., & Kitsiou, S. (2015). Synthesizing information systems knowledge: A typology of literature reviews. Information and Management, 52(2), 183-199. https://doi.org/10.1016/j.im.2014.08.008        [ Links ]

Patterson, D. J., Sim, S. E., & Aiyelokun, T. (2009). Overcoming blind spots in interaction design: A case study in designing for African AIDS orphan care communities. Information Technologies & International Development, 5(4), 75-88.         [ Links ]

Sen, A. (2001). Development as freedom. Oxford Paperbacks.

Toyama, K. (2010). Human-computer interaction and global development. Foundations and Trends in Human-Computer Interaction, 4(1), 1-79. https://doi.org/10.1561/1100000021        [ Links ]

Van Biljon, J., & Renaud, K. (2018). Reflections on the maturity of the mobile communication technology for development (M4D) landscape: 2008-2016. In R. Baguma, & J. S. Pettersson (Eds.), Proceedings of the 6th International Conference on M4D Mobile Communication Technology for Development (pp. 7-18), Kampala.

Van Biljon, J., & Renaud, K. (2019). Human-computer interaction for development (HCI4D): The Southern African landscape. In IFIP 9.4 International Conference on Social Implications of Computers in Developing Countries (Vol. 1). Springer.https://doi.org/10.1007/978-3-030-19115-321

Van Biljon, J. (2020). Knowledge mobilization of human-computer interaction for development research: Core issues and domain questions. Information Technology for Development, 26(3), 551-576. https://doi.org/10.1080/02681102.2020.1767022        [ Links ]

Walsham, G. (2017). ICT4D research: Reflections on history and future agenda. Information Technology for Development, 23(1), 18-41. https://doi.org/10.1080/02681102.2016.1246406        [ Links ]

Winters, N., & Toyama, K. (2009). Human-computer interaction for development: Mapping the terrain. Information Technologies & International Development, 5(4), iii-viii.         [ Links ]

Yadav, D., Malik, P., Dabas, K., & Singh, P. (2019). Feedpal: Understanding opportunities for chatbots in breastfeeding education of women in India. In Proceedings of the ACM on Human-Computer Interaction, 3 (CSCW) (pp. 1-30).

 

 

1 The corpus of 176 items is available at https://tinyurl.com/HCI4D-Corpus

Creative Commons License All the contents of this journal, except where otherwise noted, is licensed under a Creative Commons Attribution License