GENERAL ARTICLES

 

Applying an integrated theoretical lens to evaluate the perceived effectiveness of a computer-based reading development course in higher education

 

 

B.Lenong^I; S. M. Holtzhausen^II

^IDepartment: Education and Professional Studies, Central University of Technology, Bloemfontein, South Africa. https://orcid.org/0000-0002-2709-7469
^IIDepartment: Curriculum Studies and Higher Education Studies, University of the Free State, Bloemfontein, South Africa. https://orcid.org/0000-0002-8241-0024

 

 

---

ABSTRACT

South African higher education institutions have implemented different means of developing first-year students' English language abilities. In 2007, a university of technology introduced a compulsory computer-based reading development course to help first-year students improve their reading ability. A decade after implementation, an investigation into the effectiveness of the course, from the students' perspectives, became imperative. This article reports on an evaluation of the perceived effectiveness of the course that was undertaken in 2018. A questionnaire survey of 269 Bachelor of Education students, followed by focus group interviews, were used to gather data. The design of the study was informed by a theoretical lens that highlights a set of directives that are underpinned by theory on student engagement and creating motivational conditions. This theory is integrated with theory emanating from studies on technology acceptance and use of the computer as medium. The findings indicate that the course was generally perceived as easy to use, useful and engaging, and that a good level of inclusion had been established. A few aspects needed attention, however, and pointed to the institution's obligation to ensure that all conditions are adhered to for the creation of a motivational environment for culturally diverse communities. The article not only touches upon the practical implementation of reading development in higher education, but also describes in detail an integrated theoretical lens that can be customised to evaluate the perceived effectiveness of many technologically enhanced teaching and learning applications in higher education.

Keywords: reading development, computer-based education; technology acceptance model (TAM), student engagement; motivational environment, integrated theoretical directives

---

 

 

INTRODUCTION

Students' English language proficiency is a matter of concern at South African higher education institutions. Research studies report that first-year students enter higher learning institutions with reading problems and poor writing skills (Pretorius 2002a; Bertram 2006; Ngwenya 2010; Bharuthram 2012; Livingstone et al. 2015). Many studies have explored students' reading abilities, in particular that of first-year students (Grabe and Stoller 2002; Nel, Dreyer and Klopper 2004; Bertram 2006; Pretorius 2002b; Klopper 2001; Ngwenya 2010; Bharuthram 2012; Livingstone et al. 2015). They determined, inter alia, that students' reading is far below the assumed reading level, which could lead to students' failure to cope with academic work (Klopper 2001; Nel et al. 2004; Bertram 2006; Ngwenya 2010; Bharuthram 2012). Similar alarming results were found by an analysis of the Placement Test in English for Educational Purposes (PTEEP), in which 79 per cent of the students failed to score above 50 per cent (Sebolai 2014).

Howell Major, Harris, and Zakrajsek (2015) point out that, when a student enters higher education, fluency in reading impacts the student's learning and success, regardless of the discipline or course; they emphasise that without suitable intervention, students who do not have the necessary reading skills will be at risk of failing. In South African higher education, different means of developing the English proficiency level of students have been implemented. A university of technology (referred to as "the UoT" in this article) addressed the problem by including a computer-based reading development (CRD) course in their academic literacy programme. ("Course" in the discussion, therefore, refers to the specific computer software application, as well as its use and organisation within the broader literacy development endeavour.)

Over the last decade technology-enhanced teaching and learning methods have become more common in higher education (Gachago et al. 2013; Waghid 2014; Govender 2015). The choice of a technology intervention was strengthened by research that showed that using technology can lead to positive experiences and outcomes in the higher education classroom and can help to engage students in active and meaningful learning environments (Lee, Tan, and Goh 2004; Gachago et al. 2013; Waghid 2014; Waghid and Waghid 2016). Aljaloud et al. (2015, 315) list advantages, such as the interactivity that promotes learner-centred teaching, better communication, new opportunities for support and feedback; and enhanced student performance and engagement "that brings fun in the learning environment, positive attitudes and a desire to learn". Similarly, Waghid and Waghid (2016) assert that higher education is challenged to use technology in educational settings to stimulate excitement, interaction, and sharing in students.

At the UoT, the CRD course uses the Readers are Leaders software, which its website describes as "a leading computer-mediated reading and language program in South Africa".¹Since its implementation in 2007, the course has been compulsory for all first-year students. Reading classes are structured within the academic timetable. Students who are enrolled for the course must spend 12 weeks (one term) in the reading laboratory. They are divided into groups according to their faculties, and they engage in reading activities to improve their reading speed, retention and comprehension. Students engage in the following exercises: timed passages; word definition exercises; comprehension exercises; word recognition; vocabulary exercises and grammar exercises. The students start work at Grade 12 level 13 and move on to upper (tertiary) levels once they have completed the lower levels and performed well (Lenong and Masoabi 2018). Students receive immediate feedback after completing a single exercise, and again after all exercises in a section have been completed. They can view their reports, which contain the results of all the activities completed. Students whose reading ability is still poor after completing the course are referred for remedial sessions.

A study by Livingstone et al. (2015) that evaluated an academic reading programme in South Africa with the aim of helping students who lack the prerequisite reading skills for the reading load at university level found that the programme improved the reading speed of participants, whilst maintaining comprehension. Another study, undertaken by Lenong and Masoabi (2018), explored the effects of the CRD course on first-year students' reading speed. The results provided evidence that the course improved the reading speed or fluency of all (100%) the participants, while 80 per cent of the sample obtained a final reading speed that was above their averages throughout the term.

However, since its implementation in 2007, no study has investigated students' views on the CRD course. Such a study has become imperative, not because the course is compulsory, but because it requires a great deal of time, effort, and resources. This study, thus, aimed to evaluate the perceived effectiveness of the implementation of the CRD course. The evaluation can be described as improvement-oriented evaluation, formative in nature, because, in the view of Martinez-Torres et al. (2008), using e-learning material can only be warranted by its evaluation and quality enhancement. The personal interest of the researcher (the first author) arose from her involvement in the course over several years, and consequent interest in the effectiveness of the course and the role the institution can play to enhance the quality of the students' experiences on the course.

The article commences with an overview of the theory that underpins the study and leads to the formulation of the integrated directive used in the evaluation. The theoretical directive integrates relevant educational theory, and theory related to technology use (computers in particular). In the discussion of the results that follows, the application of the directive is clearly illustrated. The article concludes with the researchers' views on the contribution of the study, the limitations of the study and suggestions regarding the refinement of the directive and possible further research.

 

A THEORETICAL LENS FOR THE STUDY

The theoretical lens comprises a set of directives that was developed for the evaluation of the computer-based reading course. The directives integrate relevant education theory related to student engagement, motivation and the use and acceptance of technology. Many of the concepts underlying the various theories/models overlap and strengthen one another. Integrating these concepts provides useful guidelines that underpin the effective use of computer-based and other technology in a higher education setting and, consequently, also for the evaluation of its perceived effectiveness.

Student engagement theory: Activity and motivation

Student engagement represents two important features of institutional quality: 1) The amount of time and effort students put into their studies and other educationally purposeful activities; and 2) How the institution deploys its resources and organises the curriculum and other learning opportunities to encourage students to participate in activities that decades of research studies show are linked to effective student learning (National Survey of Student Engagement 2013). There is, in fact, a sound body of knowledge on the positive influence of student engagement on academic achievement, satisfaction, and persistence (Trowler 2010; Kuh 2008). Again, teaching and learning create human engagement (Davids and Waghid 2018). Although students are usually the focus of student engagement discussions and research, there is no uncertainty about the obligation on institutions to create effective learning environments for their students (Kuh 2008).

Engagement involves, however, more than participation or involvement - "it requires feelings and sense making as well as activity" (Harper and Quaye 2009, 5). In this regard, Fredricks et al. (2005) distinguish three dimensions of student engagement, namely cognitive, behavioural, and emotional. In her conceptualisation, Barkley (2010, 8) describes student engagement as a process and a product that is experienced on a continuum and results from the synergistic interaction between motivation and active learning. Ginsberg and Wlodkowski (2009) point out that that there is extensive proof that motivation is consistently and positively related to academic achievement. The implication is, once again, that without motivation as essential ingredient, activity may not be enough to engage and bring about positive outcomes.

But how can the synergistic interaction between motivation and active learning and, thus, (positive) engagement be promoted?

Barkley (2010) identified three conditions for synergy between activity and motivation, which are all applicable to the use of technology, namely, helping students work at their optimal level of challenge; teaching so that students learn holistically; and creating a sense of classroom community. The latter condition is interwoven with the institution's role and the motivational aspect of student engagement. Do students feel they belong in the programme, that their diverse needs and circumstances are recognised and addressed, and that they learn something they value? All these and related aspects mentioned previously, put an obligation on the institution to create a motivational environment that encourages engagement and positive learning outcomes for a diverse community of students - who often originate from backgrounds that lack the necessary preparation for higher education learning.

Creating a motivational environment

Ginsberg and Wlodkowski's (2009, 24) motivational framework for culturally responsive teaching is useful for planning a learning experience that "1) respects diversity; 2) engages the motivation of a broad range of students; 3) creates a safe, inclusive, and respectful learning environment; 4) derives teaching practices from across disciplines and cultures; and 5) promotes equitable learning". The framework is believed to inherently represent the four motivational conditions of inclusion, attitude, meaning, and competence "that act individually and in concert to provide a pedagogical ecology that continuously enhances intrinsic motivation to learn" (Ginsberg and Wlodkowski 2009, 24). The four conditions for a motivational environment that flows from the framework include the following aspects:

1. Inclusion (feeling of connectedness, trust, respect);

2. A positive attitude (favourable disposition toward learning, created through personal/academic relevance);

3. Meaning (created through engaging and challenging learning experiences); and

4. Competence (effectively learning something valued and perceived as authentic to the real [personal or academic]).

The above framework provides the ideal basis or point of departure for the study of any educational innovation, with the main question being, do learners experience the innovation according to the four motivational conditions? Or, do they feel included, show positive attitudes, see meaning in and experience benefits from participation? In the case of technology, however, the question that arises is how the framework must be customised to make provision for the features of the tool. Applicable theory related to acceptance of technology and, more specifically, computer-mediated communication (CMC), can provide valuable guidelines in this regard.

Technology acceptance

The evolvement of computer-based education leads to questions about possible conditions or prerequisites for the effective use of technology/CMC in higher education, and in particular, how do students experience and perceive the effective use of technology?

The technology acceptance model (TAM), developed by Davis (1989) and extended by Venkatash and Davis (2000), presents an applicable theoretical point of departure for explaining the conditions for the successful use of various types of technology in an educational setting. The model suggests that an individual's behavioural intention to use a system is directed by two beliefs: perceived usefulness (seen as the extent to which users believe that using the tool or system will enhance their performance) and perceived ease of use (the extent to which users believe that using the system will not require effort). Venkatash and Davis (2000) developed and tested a theoretical extension of the TAM. They found that user acceptance intentions are significantly influenced by social and cognitive processes. Social influences refer to factors such as social acceptance of participation, identification with the system ("image") and voluntary participation. Cognitive instrumental processes include beliefs about the importance of the tool (related to goal setting), output quality (how well the system performs tasks) and perceived ease of use. In the case of the CRD, participation is not voluntary; this could mean that factors such as social acceptance and beliefs about the importance assigned to attaining goals should be strong enough to overcome the potential negative influence of compulsory participation.

In their research, Martinez-Torres et al. (2008) emphasise the role of factors related to human and social change processes in advancing technology acceptance and adoption in e-learning environments. The results of the study strongly support the extended TAM, as adapted for an e-learning context. Corresponding support for the extended TAM was found by Merhi (2015), in his study of factors that influence higher education students to adopted listening podcast. In his podcast acceptance model, perceived ease of use, and perceived usefulness had significant relationships with variables such as perceived enjoyment, perceived self-efficacy, relative advantage and image. Of importance is the motivational aspect of podcast use, and its perceived advantages, which are vital for the creation of positive attitudes about the use of the technology tool involved.

Computer-mediated Communication (CMC) Theory

To a greater or lesser extent, CMC theories are all relevant to the educational use of computers, also for the CRD course. The use of this tool exhibits many typical characteristics of CMC, such as being flexible, interactive, and instantaneous, and operating through a network system that makes it possible to reach many people at any time, and any place. The tool is asynchronous (not synchronous) and not face to face (as in classroom teaching). Two CMC theories illustrate relevance to - and even overlaps - educational theory and technology acceptance theory, namely the uses and gratifications theory (UGT) and the theory of CMC competence.

• The UGT links to the socio-psychological tradition that focuses on human behaviour in certain communication situations. The UGT focuses more on the human who uses a medium to address self-perceived needs, than on the message (Littlejohn and Foss 2005). Media can be used in many ways, and many factors can affect people's ideas, their ultimate choice of media, and what they want from it. These factors may include personal interest, social and education needs, a sense of connectivity with other users, the possibility of personalised communication, the ease of use of the technology, and its usefulness (Littlejohn and Foss 2005). The motivation to use media, therefore, relates to the gratifications it presents that are valued by its users. When applied to the reading development course, UGT points to the activity and interactivity, through the instant feedback provided, the goal-directedness, the importance of ease of use, and the fulfilment of perceived needs.

• The theory of CMC competence focuses on the motivation of the user and includes aspects such as context and the skills and knowledge gained (the outcomes). According to the theory, selection and effective use are influenced by culture, the environment, and social relationships, while knowledge as a skill in using the media also plays a role (Saritas 2006). The theory emphasises the importance of expectations, where positive expectations lead to positive outcomes, and vice versa. Positive encounters with the medium are related to better self-esteem, improved social relationships, greater cultural awareness, and engagement in learning (Coetzee, Wilkinson, and Krige 2016). The theory of CMC competence shows close links, not only with student engagement theory, but also with the framework for culturally diverse teaching and the creation of motivational conditions (Ginsberg and Wlodkowski 2009).

The research literature on the CMC and acceptance of technology not only embraces and underscores the research-based principles and models for effective teaching and learning in higher education (including student engagement theory and conditions for the creation of a motivational environment), but extends the theory, to allow for the use of computer-based and other technology in education. Integration of theory from the various spheres, therefore, provides a useful lens for the evaluation of the effectiveness of a technology application such as the CRD course.

Conceptualising the perceived effectiveness of a computer-based application

Based on acknowledged educational and CMC research, using technology has social, as well as cognitive advantages that can play an important role in creating a motivational environment, and can give rise to positive attitudes and feelings of inclusion, meaning, and competence. The latter aspects can be linked to the perceived ease of use and perceived usefulness of the technology involved; thus, acceptance of or a positive inclination towards the process and outcomes of using the tool involved - as also related to the UGT and the theory of CMC competence. In our conceptualisation, the interrelatedness of these conditions and feelings inform the perceptions regarding the effectiveness of a computer-based application (see Figure 1).

In our conceptualisation (Figure 1), student engagement is depicted as the foundation of the implementation - and thus perceived effectiveness - of the technology innovation. Student engagement embraces conditions for success from relevant CMC theory, such as optimal challenge, holistic learning, high expectations and classroom community. The engagement takes place in a computer-mediated environment, with applicable CMC theories pointing to the gratifications the medium offers, and the competencies attained. The three gears represent the interrelated "driving forces" of the use of technology in the educational setting: The creation of a motivational environment helps to form beliefs about the ease of use and usefulness of the tool involved, and the intention of use is enhanced. The model suggests that, in interacting, the driving forces "steer" a student to engage, act, interact, feel at ease, become intrinsically motivated, enjoy, identify with the programme, feel safe and connected, and develop positive attitudes. Positive experiences and feelings would not only enhance participants' perceptions of the effectiveness of the programme, but would promote effective learning (competence in reading ability, in this case), and have other gains, such as improved self-esteem, self-efficacy, and motivation to read. The obligation, however, lies on the institution to deploy resources and create a motivational environment in which the desired outcomes can be attained. The model can, consequently, also be regarded as a conceptualisation of good practice in the planning and implementation of technological innovations in the higher education environment (or as a computer-mediated learning model).

With the theoretical model in Figure 1 in mind, an integrated set of directives for the evaluation of the computer-based reading development course was formulated (see Figure 2).

The directives in Figure 2 reflect how the theory underlying our conceptualisation of perceived effectiveness (Figure 1) can be applied as a theoretical lens that focuses on and informs an evaluation of the CRD course. In the sections that follow, it will become clear how the directives were integrated into the research methodology employed.

 

RESEARCH METHODOLOGY

Overall, the current study shows the characteristics of an improvement-oriented programme evaluation. Such an evaluation is formative in nature and was executed with quality enhancement in mind; it strived to identify strengths and weaknesses of the course and to manage it more effectively in the future (Babbie and Mouton 2001, 338-339). The mixed methods approach that was employed resembles an integration of the explanatory and the triangulation types of mixed method design (Creswell 2009): Quantitative data was collected first, followed by the collection of qualitative data that would not only confirm findings (triangulation), but could shed led light on some of the quantitative findings.

The quantitative data was captured first using a questionnaire, with the purpose of eliciting information that is offered spontaneously - students recalled from memory their most important perceptions of their involvement with the CRD course. It was followed by the collection of qualitative data in focus group interviews.

The purposive, convenient, stratified random sample consisted of 269 first-year students at the UoT who were studying for various Bachelor of Education degrees (Babbie and Mouton 2001). The questionnaire was completed online near the end of the 2018 academic year; participation was voluntary, and participants were ensured of confidentiality and anonymity. The necessary institutional authorisation and ethical clearance were obtained.

The self-constructed questionnaire contained 32 statements that were guided by the five dimensions of the theoretical directives (see Table 2). Students' responses to the statements were rated according to a four-point scale, ranging from "Strongly agree" to "Strongly disagree". A couple of open-ended questions were included, to leave room for possible comments. The results were generated by QuestionPro, and included a summary of the responses, mean values, and the standard deviation for each statement, with confidence intervals (at 95%). These statistics were regarded as sufficient for an evaluation of a formative nature with the goal of diagnosis and improvement of key problem areas.

Participation in the five focus group discussions was voluntary and targeted different first-year B.Ed. specialisation groupings. There were 52 participants; 5 to 10 persons participated per discussion. The interview guide used was compiled, firstly, to triangulate the data gathered with the questionnaire survey. The main questions were, therefore, meant to elicit perceptions regarding what the participants liked and disliked about the computer-based reading development course; personal experiences; challenges; the value of the course; and suggestions for improving the course. Probing questions referred to, inter alia, the support they had received and how they were treated in the laboratory Questions were also asked to obtain a better understanding of some of the results of the questionnaire survey (see discussion under each category of findings).

A pilot run assured that the statements on the questionnaire were clearly understood by the participants. The validity and reliability were enhanced further by the mixed methods design followed, and the consequent triangulation of quantitative and qualitative data. The way the authors of this article openly share information on the theory that guided the research, and the detail of its application in the data collection, is likely to benefit repetition or customisation by other researchers.

The thematic analysis of the qualitative data was mainly deductive in nature, as predetermined by categories of the set of directives that signify the integrated lens used in the research.

 

FINDINGS AND DISCUSSION

Following the presentation of demographic information of the sample in Table 2, a synopsis of findings that emerged from the questionnaire survey and the focus group discussions will be presented.

Demographic information

The questionnaire survey was completed by 269 first-year Bachelor of Education students at the UoT. This sample represented 46 per cent of the total of 578 students enrolled for this qualification. Of these participants, 60 per cent indicated they were female and 39 per cent that they were male; a few (1%) did not specify. The respondents represented all the specialisation areas, namely for teaching languages (27%), economics and management sciences (20%), natural sciences (17%), mathematics (14%), technology (13%), and computer science (9%). The distribution of home languages of the participants, presented in Table 1, gives a clear indication of the multicultural diversity of the sample - with about 11 per cent representing the white and coloured student communities, and at least 85 per cent representing the Black South African population.

 

 

In this evaluation study, the results were not interpreted and compared according to the differences between the participants (although this analysis could be useful in future studies of a more advanced statistical nature).

Perceived effectiveness of the CRD course

In this very brief overview, the results of the questionnaire survey will be merged with the opinions expressed during the focus group interviews. In Table 2, the results of the survey are presented according to the categories to which the various statements most obviously relate (even though we acknowledge that all aspects covered in the questionnaire are interrelated and that most of the statements can, consequently, represent more than one category of perceived effectiveness). A couple of the most revealing results are, therefore, repeated under more than one suitable category - and are clearly indicated (bold, italics and in brackets). For ease of interpretation, we divided the results in the Engagement category into two sections: perceptions on student (personal) engagement, and perceptions related to institutional engagement. This division linked in a more meaningful way with theory on student engagement - as will be illustrated.

Inclusion

The first category in Table 2 gives an indication of the students' perceptions on the category of Inclusion or belonging in the course (indications of feeling connected, being trusted, respected - also about diversity in and provision for background, talents/abilities, personal circumstances - thus, equitable learning).

Except for statement 1.7 in Table 2, more than 90 per cent of the respondents either strongly agreed or agreed with the first six statements, thereby demonstrating a considerable level of trust and respect towards the CRD course. These sentiments are clearly reflected in the low mean values. The vast majority indicated that they felt comfortable in a laboratory, which they perceived as quiet, where they were treated equally by assistants, who were regarded as helpful and patient.

In an improvement-oriented evaluation, it is, however, necessary to also take notice of the perceptions of the minority (even though, in this case, they represent fewer than 10%). The open-ended questions and focus group interviews brought diverse perspectives to the fore. Contrary to the responses of the majority on the survey, some respondents indicated that they sometimes found the laboratory noisy, with disturbances coming from "restless learners" (only one respondent describing the environment as "too silent"). Several students mentioned "overcrowding" of the lab, that there was "not enough space" and that the number of people using the lab at a time should be "minimised". Several mentioned the time factor - that they needed more time to complete the course or certain exercises. The implication is that the infrastructure and the organisational arrangements of the CRD course require attention.

There were also complaints that assistants "were not assisting enough", and respondents requested that "lecturers should have patience with us". Others mentioned, "There were a lot of people and some of them did not respect the teacher". The most alarming finding in the Inclusion category was the perception of the majority of the participants (60%, mean 2.06) that some lecturers had a negative attitude about reading skills development (and, thus, the CRD course). Figure 3 displays the responses to this statement. It is clear that 60 per cent of the respondents strongly agreed/agreed on the reality of negative attitudes among their lecturers. Such a situation can lead to a lack of support, which limits adaptation of technologies in teaching and learning (Bozalek, Ng'ambi, and Gachango 2013).

According to responses to the open-ended questions and comments made during the discussions, some lecturers seemingly regarded reading classes as a "waste of time" or are of the opinion that "periods take their time for theory". Participants perceived that such lecturers believed that "students know how to read they don't have to be taught how to read at tertiary level"; and "Reading courses should be done in high school". This type of attitude among lecturers not only has a bearing on the inclusion established in the course, but also affects the image of the programme. (This aspect is also discussed under other applicable categories.)

Ease of use

Ease of Use (also related to user-friendliness) is regarded as a key driver influencing user acceptance and usage behaviour (or intention to use) information technologies (Venkatesh 2000, 342; Wu and Wang 2005). Venkatesh (2000, 342) warns that the emotional aspect (conceptualised as computer anxiety) could determine early perceptions of ease of use of a new system, which could inhibit the experience. The aspect of computer anxiety is a factor that should be taken into consideration in a South African context, where many entry-level students have never had access or limited access to a computer before - as several respondents mentioned during the focus group interviews. The results of the questionnaire survey (section on Table 2) showed, however, that the vast majority of the respondents strongly agreed/agreed on the functional value of the CRD course, namely that it was easy to learn to use, the instructions were clear, and that all reading material was easy to understand. This positive inclination to use computers suggests the development of computer self-efficacy and internal motivation by respondents, which is consistent with the work of Venkatesh and Davis (2000).

Contrary to the aspects mentioned above, it seems that a relatively large percentage of users (44%) were frustrated by technical problems, as depicted in Figure 4. The qualitative responses confirmed this finding and provided possible explanations for the negative perceptions. References were made to the network that "would usually get lost while busy", "slow" computers, computers that needed maintenance and "not enough" computers to accommodate all students. Such negative experiences could inhibit students' experiences and give rise to computer anxiety and frustration.

The various findings related to Ease of use must be interpreted cautiously. The survey was conducted at the end of the academic year, long after most of the respondents had completed the 12-week course, and probably after they had forgotten about any computer anxiety they had experienced. References to initial computer anxiety, in the interviews, should caution facilitators (as representatives of the institution) to be sensitive and helpful when introducing computer programs to a diverse group of entry-level students. The frustration experienced because of technical problems also points to institutional responsibility (see discussion on Institutional engagement).

Attitude

One of the conditions for the creation of a motivational environment for the use (or even future continuation of the use) of technology is determined by a positive attitude (Merhi 2015). Responses to the first six statements in the Attitude category (Table 2) reflect a strong positive attitude towards the course and its relevant features, which were perceived as enjoyable and meaningful, and involving interesting stories. There was also a positive inclination toward active, engaged learning, which is created through the immediate feedback function. The results also suggest acknowledgment of the positive attitudes of peers towards CRD (which include memory exercises, which are generally perceived as academically valuable for comprehension and spelling purposes). Similar positive attitudes were detected in the focus group interviews. The element of "image" or identification with the program, can be recognised in the findings in general. However, as in the Inclusion category, the negative attitudes of some lectures (see 4.2.1), may cast a shadow over the "image" of the course. The frustration caused by technical problems (see Ease of use category) and challenging grammar exercises (see Student engagement category), may also give rise to negative perceptions in some students, and subsequently act as possible threats to the creation of a motivational environment.

Usefulness

The Usefulness category (see Table 2) provides evidence of a very high level of agreement about perceived outcomes and the personal value of the CRD course. More than 90 per cent of the respondents agreed/strongly agreed that the course improved their reading speed, and that it improved their concentration, comprehension, vocabulary, and overall performance. Similar beliefs were expressed during the focus group discussions. In addition to all the advantages experienced, several participants mentioned that they became interested in reading for the first time and had started to enjoy reading books and articles. One mentioned that she was reading differently than before, with better concentration and understanding.

Caution should, however, be exercised regarding the perception of improvement in particular constructs, such as comprehension and concentration. A quantitative study by Livingstone et al. (2015, 6), which also investigated the effect of the Readers are Leaders software, found no change in the mean values for comprehension.

Students reported that "people who help us, ... didn't give up on helping us" and that "the course should continue", especially since enhanced reading speed, concentration, vocabulary, and comprehension resulted in greater confidence and goal-directed effectiveness while studying, and also improved performance (see Table 2). However, some participants identified threats, namely that the duration of the CRD course was too long and too intensive. Also, it is evident that some students did not finish the CRD course in the specified time since they missed some of the classes and some did not achieve the required reading speed. These findings indicate that the students' reading ability was below maturity level, which suggests a negative attitude towards the course. Ribbens (2008) reports that students who reread texts experience more difficulties in understanding texts.

Student (personal) engagement

Student engagement has been found to be a foundational construct in our conceptualisation of the perceived effectiveness of a technological tool (such as in computer-based reading development). Statements in the Engagement category (Table 2) are meant to reflect the "synergetic interaction between activity and motivation" (Barkley 2010) and the time and effort exerted to complete a task (National Survey of Student Engagement 2013; Davids and Waghid 2018).

The responses reported in Table 2 suggest a high level of engagement in the CRD course. More than 90 per cent of the respondents believed that they put in maximum effort to complete the course, and that they were kept engaged. Similar high percentages agreed/strongly agreed that they had managed to complete all activities in the time provided and that they had attended all classes. Almost all the respondents (98%) clearly liked the immediate feedback and found all the activities meaningful. The focus group interviews endorsed the high level of engagement detected by the survey.

It is clear, however, that most respondents (93%) found the grammar exercises challenging. This finding emerged in the focus group discussions, through comments such as, "I disliked the grammar exercises. It was very challenging to me" and "Take the grammar out and then they could just give us a choice to do grammar or not". This perceived challenging experience could, however, have contributed to students' engagement, as helping students work at their optimal level of challenge is regarded as a precondition for creating synergy between activity and motivation (Barkley 2010; Davids and Waghid 2018). When taking both the qualitative and quantitative data into consideration, there appears to be enhanced personal engagement with learning "to improve own studies", as well as commitment to continue with "the repetition of something until being successful".

Institutional engagement

The mean values of the statements categorised under Institutional engagement are represented graphically in Figure 5. The lower mean values indicate that most of the respondents agreed/strongly agreed about aspects such as enough lecturers in the laboratory, equal treatment of all by the lecturers, and that the time allocated for completion was sufficient.

Perceptions about the negative attitudes of some lecturers towards reading skills development were discussed under Inclusion and Attitude, and the technical problems with computers in the Ease-of-use category. Other points emphasised in the focus group interviews were a "lack of enough computers" "slow computers" and an occasional "overcrowded" laboratory, and that the computers were slow. All these points of criticism are examples of the inter-relatedness of the theoretical directives and how one negative factor can influence the perceived (and real) effectiveness of any teaching and learning endeavour in higher education. (Fortunately, the same can be said about the effect of positive aspects.)

Two largely contradictory findings were, however, detected: Almost all the respondents (98% mean 1.16, see Figure 5) expressed that the UoT did well to provide the reading development course. The perception, "The UoT should use another program for reading development", is, therefore, not easy to interpret (see Figure 4, mean 2.88). Although a minority (36%) agreed/strongly agreed with the statement, it is just too large a group to ignore. Even probing during the focus group discussions could not explain, to a satisfactory extent, the negative perception of almost 100 of the 269 participants in the survey. A couple of participants mentioned that some stories were not interesting; that they had to reread stories to understand them, which wasted much time; that they were not motivated, especially in the beginning; that personal problems prevented them from completing the course in the allocated time, leading to the need to repeat it the next term; and that the grammar exercises should be removed from the course. Other comments cast doubt on whether the course sufficiently made provision for the full diversity of students and offered equitable learning opportunities: "There should be various options for reading to accommodate all students"; "I would prefer it if the course was done manually - I am kind of old school". One participant recommended using textbooks and computers; one suggested the addition of audio clips, while another one mentioned that the use of the computer affected their sight. None of the suggestions provided sufficient motivation to change the course. One might suggest that the compulsory attendance of reading development classes by all, might be an underlying problem - among students who might believe (as some lecturers at the UoT seemingly do), that reading development should take place in high school. We may ask whether it is necessary to channel resources towards further reading development of all students, even those who test high or relatively high on entry-level language testing.

 

CONCLUSION

English language proficiency provides a foundation for success in South African higher education. This research investigated, through an integrated theoretical lens, an evaluation of the perceived effectiveness of a CRD course intended to benefit first-year B.Ed. students' reading ability at the UoT case study.

Computer-based theories of technology acceptance and computer-mediated communication, underpinned by theory on student engagement and the creation of motivational conditions, offered an academic clarification for the effective application, using technology, of the CRD course in a higher education context. The questionnaire survey and focus group interview findings indicate that the course was trusted and respected by most of the sample of 269 B.Ed. students. Through the theoretical lens applied, the evaluation also suggests that the CRD course was perceived as clear, easy to use and understand, and useful, and that students believed the course was inclusive. This course, with its interwoven interactive learning opportunities via motivational conditions, such as support and instant feedback, had seemingly improved the first-year student participants' reading ability and academic performance. Cognitive engagement with purposeful reading activities had stimulated a perceived safe, active, and meaningful motivation for learning, and created positive attitudes, and stimulated acknowledgement of fun behaviour. These outcomes appear to have had a positive impact on students' goal-directness and the perceived need to enhance reading speed, retention, and comprehension.

A few aspects need attention, however. Aspects such as the seemingly negative attitudes of some lecturers at the institution, technical problems students experienced, and challenges related to the software program used, point to an institutional obligation to ensure that all infrastructural, organisational, and time allocation conditions for the creation of a motivational environment are adhered to, to deliver equitable learning opportunities.

This article attempted to contribute to the practical organisation of reading development using educational technologies in higher education in South African context and global South, and to demonstrate how a theoretical lens - an integration of directives derived from applicable theory - can inform an evaluation of the perceived effectiveness of a specific technology tool (computer-based reading technology, in this case). We believe that this lens can be customised and applied to devise instruments to evaluate the perceived effectiveness of any technologically enhanced teaching and learning innovation in higher education. One exciting possibility is the application of the theoretical lens to customise data collection methods for evaluating the perceived effectiveness of online and other creative teaching and learning innovations that evolved during the Covid-19 pandemic. Computer anxiety is another factor that should be considered in a South African context, where many entry-level students have never used or only had limited access to a computer. Higher education institutions in the global South should be prepared to use technology to improve first-year reading.

Limitations of the study are acknowledged, considering the formative nature of the evaluation. Due to the limited sample and population (from one faculty only), further research on utilising the useful (and very flexible) theoretical lens with its integrated directives, is needed. Several statements on the questionnaire could be rephrased to become more critical, thereby making provision for alternative views expressed during the focus group interviews, for example. Eliciting of a wider variety of perceptions in the Inclusion category may provide a richer array of perceptions from a diverse student community. The fact that the research was undertaken at the end of an academic year may have diminished its formative and, thus, improvement-oriented value. Academically it may be more beneficial to undertake an evaluation of this nature at the end of each 12-week period, targeting the group that had just completed the course. Completion of the questionnaire may even be built into the course as a final obligation. Doing so will potentially not only maximally improve the response rate, but also the diagnostic value of the evaluation.

 

NOTE

1 http://readersareleaders.co.za/what-makes-readers-are-leaders-so-great/.

 

REFERENCES

Aljaloud, Abdulaziz, Nicolas Gromik, Willaim Billingsley, and Paul Wing Hing Kwan. 2015. "Research Trends in Student Response Systems: A Literature Review." International Journal of Learning Technology 10(4): 313-325. https://doi.org./:10.1504/IJLT.2015.074073.         [ Links ]

Babbie, Earl R. and Johann Mouton. 2001. The Practice of Social Research. Oxford: Oxford University Press.         [ Links ]

Barkley, E. 2010. Student Engagement Techniques: A Handbook for College Faculty. San Francisco: Jossey-Bass.         [ Links ]

Bertram, Carol. 2006. "Exploring Teachers' Reading Competences: A South African Case Study." Open Learning 21(1): 5-18. https://doi.org/10.1080/02680510500467874.         [ Links ]

Bharuthram, Sharita. 2012. "Making a Case for the Teaching of Reading Across the Curriculum in Higher Education." South African Journal of Education 32(2): 205-14. https://doi.org/10.15700/saje.v32n2a557.         [ Links ]

Bozalek, Vivienne, Dick Ng'ambi, and Daniella Gachago. 2013. "Transforming teaching with emerging technologies: Implications for higher education institutions." South African Journal of Higher Education 27(2): 419-436.         [ Links ]

Coetzee, Mercia, Annette Wilkinson, and Daleen Krige. 2016. "Mapping the Social Media Landscape: A Profile of Tools, Applications and Key Features." Communitas 21: 178-19. https://doi.org/10.18820/24150525/Comm.v21.12.         [ Links ]

Creswell, J. W. 2009. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Sage Publications.         [ Links ]

Davids, Nuraan and Yusef Waghid. 2018. "Indigeneity and African Education: Cultivating Decolonized University Teaching and Learning." In Indigenous Philosophies of Education Around the World, 61-81. Routledge.         [ Links ]

Davis, Fred D. 1989. "Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology." MIS Quarterly 13(3): 319-340. https://doi.org/10.2307/249008.         [ Links ]

Fredricks, J. A., P. Blumenfeld, J. Friedel, and A. Paris. 2005. "School Engagement." In What do Children Need to flourish? Conceptualizing and Measuring Indicators of Positive Development, ed. K. A. Moore and L. H. Lippman, 305-321. New York: Springer.         [ Links ]

Gachago, Daniela, Eunice Ivala, Judy Backhouse, Jan Petrus Bosman, Vivienne Bozalek, and Dick Ng'ambi. 2013. "Towards a shared understanding of emerging technologies: Experiences in a collaborative research project in South Africa." The African Journal of Information Systems 5(3): Article 4.         [ Links ]

Ginsberg, M. B. and Raymond J. Wlodkowski. 2009. "Professional Learning to Promote Motivation and Academic Performance among Diverse Adults." In CAEL FORUM and NEWS: Learning Never Ends, ed. Diana Bamford-Reese, Beth Doyle, Becky Klein-Collins and Judith Wertheim, 23-32. November.         [ Links ]

Govender, Samantha. 2015. "Students' perceptions of teaching methods used at South African Higher Education Institutions: Part 1." South African Journal of Higher Education 29(3): 23-41.         [ Links ]

Grabe, William and Fredericka L. Stoller. 2002. Teaching and Researching Reading. London: Pearson Education Longman.         [ Links ]

Harper, S. R. and S. J. Quaye. 2009. "Beyond Sameness, With Engagement and Outcomes for All. Student Engagement in Higher Education." In Student Engagement in Higher Education, ed. S. R. Harper and S. J. Quaye, 1-15. London: Routledge.         [ Links ]

Howell Major, Claire, Micael S. Harris, and Todd D. Zakrajsek. 2015. Teaching for Learning: 101 Intentionally Designed Educational Activities to put Students on the Path to Success. Routledge.         [ Links ]

Klopper, Mariana. 2001. "Leesverryking en Leerontwikkeling as Metodes van Leerderondersteuning vir die Verbetering van Akademiese Prestasie van Eerstejaar Risikostudente" [Reading Enrichment and Learning Development as Methods to Improve learner Performance of At Risk First-year Students]. Master's thesis, Potchefstroom University.         [ Links ]

Kuh, George D. 2008. "Excerpt from High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter." Association of American Colleges and Universities 14(3): 28-29.         [ Links ]

Lee, Chye Seng, Daniel TionHok Tan, and Wee Sen Goh. 2004. The Next Generation of e-Learning: Strategies for Media Rich Online Teaching and Engaged Learning. International Journal of Distance Education Technologies (IJDET) 2(4): 1-17. https://doi.org./:10.4018/jdet.2004100101.         [ Links ]

Lenong, S. B. and C. S. Masoabi. 2018. "Advancing First-Year Students' Reading Skills Using Computer Reading Programme to Enhance Learning." In 1st International Conference on Religion, Social Sciences and Technological Education, ed. M. A. Ibrahim, 238-249. INSIGHT.         [ Links ]

Littlejohn, S. W. and K. A. Foss. 2005. Theories of Human Communication. 8^th Edition. Belmont, CA: Thomson Wadsworth.         [ Links ]

Livingstone, Candice, Betsie Klopper, Sanet Cox, and Corrie Uys. 2015. "The Impact of an Academic Reading Programme in the Bachelor of Education (Intermediate and Senior Phase) Degree." Reading & Writing: Journal of the Reading Association of South Africa 6(1): Article 66. https://doi.org./10.4102/rw.v6i1.66.         [ Links ]

Martinez-Torres, M. R., S. L. Toral Marin, F. Barrero Garcia, S. Gallardo Vázquez, M. Arias Oliva and T. Torres. 2008. "A Technological Acceptance of e-learning Tools Used in Practical and Laboratory Teaching, according to the European Higher Education Area 1." Behaviour & Information Technology 27(6): 495-505. https://doi.org./:10.1080/01449290600958965.         [ Links ]

Merhi, Mohammad I. 2015. "Factors Influencing Higher Education Students to Adopt Podcast: An Empirical Study." Computers & Education 83: 32-43. https://doi.org/10.1016/j.compedu.2014.12.014.         [ Links ]

National Survey of Student Engagement. 2013. National Survey of Student Engagement: Origins and Potential. http://nsse.iub.edu/html/origins.cfm.         [ Links ]

Nel, Charl, Carisma Dreyer, and Mariaan Klopper. 2004. "An Analysis of Reading Profiles of First-year Students at Potchefstroom University: A Cross-sectional Study and a Case Study." South African Journal of Education 24(1): 95-103.         [ Links ]

Ngwenya, Themba. 2010. "Correlating First-year Law Students' Profile with the Language Demands of their content Subjects." Per Linguam 26(1): 74-99. https://doi.org/10.5785/26-1-15.         [ Links ]

Pretorius, Elizabeth J. 2002a. "Reading ability and academic performance in South Africa: Are we fiddling while Rome is burning?" Language Matters: Studies in the Languages of Southern Africa 33:169-196.         [ Links ]

Pretorius, Elizabeth J. 2002b. "Reading and Applied Linguistics - A Deafening Silence?" South African and Applied Studies 20(1-2): 91-103. https://doi.org/10.2989/16073610209486300.         [ Links ]

Ribbens, R. 2008. "Hypnotised by Guntenberg? A report on the reading habits of some learners in academia." Journal for Language Teaching 42(2): 105-122.         [ Links ]

Saritas, Mustfa T. 2006. "Computer-mediated Communication in Higher Education: An Exploration of Knowledge Construction." PhD diss., Iowa State University.         [ Links ]

Sebolai, Kabelo. 2014. "Evaluating Academic Literacy Teaching at a South African University: A Case Study of an Academic Literacy Programme." Journal for Language Teaching = Ijenali Yekufundzisa Lulwimi = Tydskrif vir Taalonderrig 48(1): 51-69. https://doi.org/10.4314/jlt.v48i1.3.         [ Links ]

Trowler, Vicki. 2010. Student Engagement Literature Review. November. The Higher Education Academy.         [ Links ]

Venkatesh, Viswanath and Fred D. Davis. 2000. "A Theoretical Extension of Technology Acceptance Model: Four Longitudinal Studies." Management Science 46(2): 186-204. https://doi.org./10.1287/mnsc.46.2.186.11926.         [ Links ]

Venkatesh, Viswanath. 2000. "Determinants of perceived ease of use: Integrating control, intrinsic motivation, and emotion into the technology acceptance model." Information Systems Research 11(4): 342-365. https://doi.org./10.1287/isre.11.4.342.11872.         [ Links ]

Waghid, Yusef. 2014. "Islamic education and cosmopolitanism: A philosophical interlude." Studies in Philosophy and Education 33(3): 329-342.         [ Links ]

Waghid, Zaid and Faiq Waghid. 2016. "Examining digital technology for (higher) education through action research and critical discourse analysis." South African Journal of Higher Education 30(1): 265-284.         [ Links ]

Wu, Jen-Her and Shu-Chin Wang. 2005. "What drives mobile commerce?: An empirical evaluation of the revised technology acceptance model." Information & Management 42(5): 719-729.         [ Links ]