versión On-line ISSN 2310-3833
versión impresa ISSN 0038-2337
S. Afr. j. occup. ther. vol.44 no.1 Pretoria ene. 2014
Paula Barnard AshtonI; Lindsay KochII; Alan RothbergIII
IBSc OT (Wits), MSc OT (Wits); Lecturer and Manager of elearing project, Departement of Occupational Therapy, School of Therapetiuc Sciences, University of the Witwatersrand
IIBSc (OT) UOFS; MSc (OT) Wits; Lecturer; Department of Occupational Therapy; School of Therapeutic Science; University of the Witwatersrand
IIIMBBCh (Wits), FC Paed (Wits), PhD (Wits); School of Therapeutic Sciences, Faculty of Health Sciences, University of the Witwatersrand
Change is scary, especially when the world of technology, lecturers (digital immigrants) and students (digital natives) come together with learning in mind. Developing blended learning by integrating e-learning into an existing undergraduate Problem Based Learning (PBL) curriculum requires adaptable lecturers and the time for students to become habitual users of the Virtual Learning Environment (VLE). The occupational therapy curriculum at the University of the Witwatersrand has traditionally been delivered via PBL, but the increasing need to improve throughput rates and meet the diversity of learning needs of the students has driven the strategy towards blended learning. This study investigates the effect of habituation (student experience in using e-learning automatically) on student performance in one PBL module.
A retrospective two-cohort design was used to review the students' access to the VLE and their performance on the summative assessments of the PBL module of two concurrent academic cohorts. Data were analysed descriptively and statistically for significance (Mann-Whitney U) and effect size (Cohen's d and Hedge's g).
There was a significant difference between the two cohort's access to the VLE (p<0.002) indicating higher habituation to blended learning in the second cohort, who had more exposure to e-learning due to their second year of using VLE. There was a small but relevant effect size (average d=0.3l) in all three measures of student performance when comparing the two cohorts. The average of the student marks on each measure shifted from a failing to a passing average. This study shows that the habituation of blended learning into an existing curriculum results in improved academic performance.
Keywords: curriculum design, blended learning, connectivism, e-learning, developmental assessment
The context of this study is within an existing Problem Based Learning (PBL) occupational therapy undergraduate curriculum and considers the effect of change management of embedding blended learning within one PBL module by comparing two concurrent cohorts of students in the third year of study. This learning module relates to the skills and knowledge required to assess and treat children with learning challenges. Integrating e-learning into the existing PBL curriculum (creating a blended learning environment) has been a strategic focus within the department over the past four years.
Problem Based Learning (PBL) is recognised as a successful pedagogical strategy in the training of undergraduate occupational therapy students. It is thought to improve the clinical reasoning13 and problem solving abilities3,4 of the students and the ability to transfer skills and knowledge to new contexts3,4. This method of instruction is challenging to students as it requires collaborative group work and independent navigation of the literature, which results in a heavier workload than more traditional forms of instruction5. This notion has been challenged by Vardi and Ciccarelli6, who found that their strategies to focus the students' search for information, and directing them towards prior preparation for PBL sessions, reduced overall workload to within the 10 hours per week allocated to each PBL unit.
Problem Based Learning has been the primary strategy of curriculum delivery for the occupational therapy subjects in this faculty for over a decade. As a mode of curriculum delivery it is soundly based on the constructivist theory of learning design68. The students engage in building their knowledge through investigating (via accessing the literature) contextually meaningful scenarios that mimic real life7. An advantage of PBL is that the learning scenario creates the framework for learning the content and applying the knowledge, while giving the learner some freedom as to how to access the knowledge base6,7. The knowledge creation is therefore always current and adapts to changes in the body of knowledge3. Enhancing productivity and time management in a curriculum is a common goal in health science education, specifically in the context of PBL which is seen as having a high workload for students6.
Connectivism is a learning theory that has emerged within the last decade through the work of George Siemens9. He responded to the digital era and recognised that historical learning theories (such as cognitivism, behaviourism and constructivism) needed to be reconsidered in the face of the opportunities created by technology and the internet9. Connectivism builds on social constructivism by engaging students in collaborative learning activities, but with the view that it is just as valuable to know where to find knowledge as it is to build it10. Merrolee Penman in the 2007 Frances Rutherford Lecture11 identified connectivism as a mechanism of learning and continued professional development for occupational therapists as it creates opportunities for conversations that create learning. She highlighted that Siemens' portrayal of connectivism is that we should not strive to learn more facts, but rather know where to find them and how to apply them appropriately11. Problem Based Learning is well suited for the integration of connectivism as networks created by online environments extend the learning opportunities to the worldwide body of knowledge8. This includes access to international specialists as human resources, peer reviewed scientific content, patient blogs, international and local peers as well as the PBL group and facilitator12. Student's use of technology in PBL is thus an enabler to the learning process13.
Blended learning is viewed differently in different contexts. In many traditionally face-to-face institutions the definition provided by Zhu, Valcke and Schellens of "combining face-to-face settings with e-learning solutions reflecting social-constructivist conceptions"14:164 holds true. However, many traditional distance education environments have become flexible in their content delivery and students can select either e-learning or face-to-face learning at a module or degree level, regarding this as blended learning15. However, blended learning in the context of this study is the integration of e-learning within a traditional face-to-face or classroom based process, and is thus the instructional design used to implement connectivism within PBL. The use of Virtual Learning Environment (VLE) software creates a common web portal for lecturers, facilitators and students as the mode of delivery of blended learning for a particular course8. The VLE is a web-based access-controlled environment that facilitates the distribution of learning materials, administration notifications and assignments8. More importantly, it provides a closed space for students to participate in learning exercises, knowledge quizzes, synchronous chat rooms and asynchronous discussion boards, and knowledge collaboration activities such as wikis and blogs8,16. Lecturers are required to craft (blend) the combinations of VLE based activities (e-learning) and face-to-face experiences to ensure that learners cover the learning objectives in a manner that is meaningful and applicable to the profession8,12.
Investigations into the success of blended learning are fraught with the "no significant difference" phenomenon12,17. Sims12 supports Oblinger and Hawkins'13 position that the need for a significant difference in researching the impact of technology on learning is a myth. They propose that learning is an active process that requires motivation and social engagement, and that technology is an enabler of these learning opportunities13. All too often studies attempt to prove that either face-to-face learning or e-learning generates better student performance, alternatively studies attempt to prove that there is no difference12,13,17. Oblinger and Hawkins challenge researchers to consider the question of "difference in what?"13. They conclude that "the answer depends on how the question is asked"13:15. Common confounding limitations to education research is that the population is typically limited to class sizes, and thus studies having a truly randomised control group and experimental group are unlikely to yield sufficient numbers to achieve significance.
Cook18 asserts that prior studies of the effect of e-learning in health education has done little to inform education practice. In a systematic review of 126 studies, which investigated whether e-learning was better than no intervention (or baseline assessment scores) on the factor of knowledge gain, Cook's outcome was that there was a pooled effect size of 1.0, which he considered to indicate approximately 12% change in marks18. On the other hand when attempting to compare traditional instruction to e-learning, Cook's analysis of 76 studies yielded much lower effects (averaging an effect of 0.1 for knowledge, skills and satisfaction) and none demonstrated a significant difference18. He asserts that this is an unrealistic research question as there is little homogeneity within the observed factors across the existing studies. The review is however limited by the high heterogeneity of the studies, and should be cautiously interpreted. Cook therefore supports the notion that e-learning research should rather focus on when and how e-learning is used most effectively18. He dismisses the need to establish transferability or generalisability of study results to the global practice of health education18. It is far more valuable to now focus studies on the "when" and "how" to use blended learning effectively within the studies' own educational environment17,18. Studies published in line with this focus can then be reviewed for their potential applicability within a particular educational scope, rather than assuming that all e-learning interventions that generate statistically significant difference can then be applied to all learning contexts18.
Sims12 and Cook18 both argue that the heterogeneous nature (in terms of context, participant variables, and content variables) of blended learning and e-learning implementations offer little promise of achieving significant difference in researching these educational strategies. Despite this they both advocate promoting implementation of e-learning specifically because of the heterogeneity in current higher education, as it is a strategy that crosses cultural, time, and space divides12,18 while allowing opportunity for individuality12,18 within a social constructivist paradigm12.
Blended learning takes time to integrate into a pre-existing PBL curriculum. Lecturers and students have varying degrees of computer literacy and confidence in their computer skills. There are early adopters who jump at the opportunity to try new technology and those who have more of a "wait and see" approach. The lecturers of this occupational therapy department committed to this process, attended training on the VLE software and actively engaged with the support provided by the e-learning team. Students have access to the VLE via computer laboratories across campus and in the residences. The department also has a computer in each PBL room as well as two 10 seater computer rooms. This ensured that all students had access to the VLE irrespective of owning personal use computers. This study focuses on the adaptation of the lecturers and students' behaviour from novice uses to habitual users of blended learning within the context of a single PBL module.
Background to Paediatric Learning Disabilities module
The Paediatric Learning Disabilities module runs in the second semester of the third year of the occupational therapy degree. The focus of the module is on teaching students about the assessment and treatment of children with a variety of learning based difficulties or disabilities. Students learn the occupational therapy process applied to this module during 45 scheduled teaching hours over a period of 10 weeks. During the same time period students complete fieldwork in a variety of practice settings that are not related to the module. The students' attention is thus divided between the stressors, demands and learning opportunities associated with field-work, and the educational activities associated with the Paediatric Learning Disabilities module.
The module is designed on PBL principles and consists of a variety of activities. Two paper-based client cases combined with client videos are used to stimulate student investigations. A number of workshops and skills laboratories are spread over the 10 weeks to assist students in gaining both theoretical knowledge about assessment and treatment, as well as practical skill in administering, scoring and interpreting standardised assessments (such as the Developmental Test of Visual Perception Second Edition - DTVP-2).
e-Learning Intervention Strategies (Blended Learning)
In 2009 and the first part of 2010 a number of online activities were developed to complement the already existing PBL process in this module. These activities and resources were loaded onto the course VLE and are detailed below.
1. Wiki: A Wiki is an online shared text document, which allows students to simultaneously work, comment and edit on the same document (Wikipedia is possibly the most well-known multi page Wiki as it allows multiple independent authors to simultaneously provide content)19. Wikis were created to allow students to collaborate and share information. Students were able to work on the wikis simultaneously and from any location (home, computer laboratories) and at any time. These documents focussed on theoretical aspects of assessment and treatment.
2. Pre-clinical formative quiz: This open-book test was a prerequisite to attend the skills laboratory on paediatric standardised assessments. Students were required to pre-read the assessment manuals and then attempt the quiz as many times as they needed to in order to obtain the required mark of 100%.
3. Interactive lesson: This online module took students step-by-step through the administration and scoring of the DTVP-2. It included videos and online scoring activities that students had to complete to progress to the next part of the module. The initial access to the lesson was in groups in their assigned PBL room, requiring face-to-face collaboration between group members during a timetabled workshop. After the workshop the lesson was opened up for access at any time for revision or study purposes.
4. Optional formative lessons: Lessons containing test administration videos and scoring practice activities of other paediatric assessments were offered to students as optional self-study and practice opportunities.
5. Content repository: Teaching materials (such as videos and documents) were posted for easy continued access.
The rationale for creating these e-learning activities was to assist students' ability to study and revise material anytime and anywhere. These e-learning activities were available to the students of the 2010 and 2011 academic years of the course.
The aim of the study was to investigate if an increase in uptake (habituation) of the blended learning approach, infusing e-learning within a PBL occupational therapy curriculum will have an impact on student performance. The following were investigated:
1. The change in student behaviour in terms of online access to the VLE during the single 3rd year PBL module over two concurrent academic student cohorts. Habituation of blended learning behaviour is seen in students who are more experienced and have greater uptake of e-learning.
2. Comparison of effect on student performance in the PBL module between the 2010 (novice cohort) and 2011 (more habituated cohort) third year students over the same two academic years in terms of:
a. Clinical competence in the skills acquired during the module.
b. Academic competence in the knowledge gained during the module.
c. Overall performance of the students.
Study Design and Sampling Method
A retrospective cohort study design was used. Ethical clearance was obtained as part of a larger study from the University of the Witwatersrand Human Research Ethics Committee (Medical). Students and lecturers gave informed consent at various times during the larger study. Records of student marks and student VLE access footprint were retrieved and reviewed in 2013 in order to fulfil the objectives of the study.
A cohort of all third year occupational therapy student records from 2010 and 2011 were initially reviewed for inclusion in the study. The 2010 cohort consisted of 32 student records and the 2011 cohort consisted of 43 student records. To ensure no prior exposure to the selected PBL module, only those students enrolled in the third year occupational therapy courses for the first time were included. Records of three students in 2010 and two students in 2011 were thus excluded from the study and a total of 70 records were included for analysis: 29 from 2010 and 41 from 2011.
The primary difference between the 2010 and the 2011 cohorts was experience. The VLE was introduced into the curriculum at the beginning of the 2010 academic year, thus students and lecturers were novice users of blended learning. The 2011 cohort had the benefit of prior knowledge of the VLE and the lecturers were more adept at blended learning.
In order to determine whether the two cohorts of student records were comparable, the final second year occupational therapy results of each cohort were compared using the Mann-Whitney U Test. If the two cohorts of students performed at a similar academic level in their second year of occupational therapy (which is a prerequisite and foundation to access the third year course) it could be assumed that differences in marks in the third year module were not due to differences in the academic potential of the two cohorts.
Student behaviour was measured through retrieval of the VLE access footprint (a) of each student per cohort. Student performance on the module was measured through retrieval of two types of records: practical competency test marks (b) and end-of-module knowledge test marks (c). The overall module performance (d) of the students was then calculated specifically for this study by combining (b) and (c) into one score.
a) The VLE access footprint is a record of each student's activity on the VLE as identified via unique personal login usernames. The footprint record indicates the time and date of access and the navigation of materials and activities on the VLE. Each footprint was analysed to determine the number of instances the VLE was accessed during the 10 weeks of the PBL module. A new access instance was determined by inactivity for more than one hour or accessing the VLE from a new Internet Protocol (IP) address.
b) The practical competency test marks reflect performance on a clinical summative assessment of the students' ability to practically administer the mechanics of the paediatric standardised tests, their ability to score the results and interpret those scores correctly according to the psychometric properties of the relevant test. This summative assessment is evaluated in a highly standardised manner that did not change between 2010 and 2011.
c) The end-of-module knowledge test marks are the results of a summative paper-based test that is written at the end of the Paediatric Learning Disabilities module. This test assesses students' ability to apply the knowledge gained during the module to a paper-based case. This evaluation of student performance is less rigorous in terms of standardisation between the cohorts as test questions change from year to year, and thus the difficulty index of the test has the potential to change. This test is however typical of the end-of-module tests for all modules of the course and has maintained the same overall format.
d) The overall module performance for each student was calculated for the purposes of this study. The overall score consisted of the sum of the competency test (pass of 75/100) and the end-of-module test (pass of 50/100) and thus consisted of a score out of 200 with a pass mark of 125. This overall score was calculated to determine overall competence of students in the module as some students might have greater practical skill while others have greater academic skill.
Data Collection Procedure
Student records (extracted with only student numbers) were accessed retrospectively from the electronic marks system at the university after permission was obtained from the head of department. Records of access to the VLE were drawn from the VLE server database (student numbers are the standard username per student). These records were drawn in 2013 upholding the retrospective nature of the study thus conforming to the ethical principles specified in the ethical clearance. An Excel spreadsheet was used by an independent research assistant, to code each student to ensure anonymity and blinding, and integrate the matching of the records of their second year mark, as well as their data on each of the above measures.
All data were tabulated and means and standard deviations calculated for each cohort for each variable. Two forms of analyses were undertaken, analysis to determine the size of differences between the groups and the second to determine the statistical significance or possible influence of chance on the results.
To determine the size of the differences between cohorts (i.e. to determine effect size), the Cohen d and Hedges g effect size calculators were used20. The data for each of the student performance measures (b, c, and d) were compared using the Cohen d effect size calculator as the standard deviations for all three of these variables were similar and thus fulfilled the criteria for this test21. The standard deviations for VLE access footprint of the two cohorts were dissimilar and thus, because the sample sizes between the two cohorts were also dissimilar, the Hedges g effect size calculator was used to analyse this variable20. Cohen's definition of small, medium and large effect sizes was used to determine practical significance of results and a minimum value of d/g = 0.2 was set for effect size22.
All variables were analysed using the Mann-Whitney U-test23 to determine the statistical significance of the differences between the two cohorts. The Mann-Whitney U-test was used because the sample size of the two groups was small and data did not fulfil the requirement of normal distribution for parametric statistical tests24. The significance level was set at 5% (a = 0.05). The statistics were analysed and interpreted by the authors.
Comparability of the Cohorts
The Mann-Whitney U Test showed no significant differences between the two cohorts (U = 60l.5; p=0.94), indicating that both performed at a similar academic level in the second year.
Analysis of the Behaviour and Performance of the Cohorts
Table I summarises the results of the statistical analyses performed on the data sets of both cohorts in this study. The means (m) and Standard Deviations (SD) of each cohort on each measure is indicated, as well as the effect size of the change (habituation) from the 20l0 cohort to the 20ll cohort and the statistical significance between each cohort. Table I will be further referred to while presenting the results of each variable.
Student Behaviour Related to Blended Learning
The number of instances of access to the VLE was used to measure the relative uptake of e-learning between the cohorts. The 20l0 cohort had an average of 48.8 (SD=24.5) instances, and the 2011 cohort averaged 69.4 (SD = 30.0) instances (Figure 1).
Statistical analysis (Table I) showed that the effect size for the behaviour difference between the two cohorts was g=0.73 (using the Hedges g effect size calculator) and that this difference also had a statistical significance (U = 334; p<0.002).
The practical competency test marks for the 20l0 cohort averaged 71.8% (SD=l5.6), with a range from 27.7% to 90.8%. This cohort average was below the 75% pass mark. Of the 29 students in this cohort, 13 (45%) failed the initial competency test. The 2011 cohort average was 76.2% (SD=16.1), with a range from 41.7% to 97.7%. This cohort average was above the pass mark of 75%. Of the 41 students in this cohort, 14 (34%) failed the initial competency test (see Figure 2).
The effect size (Table I) of positive change in the practical competency test marks from the 2010 to the 2011 cohort was calculated at d=0.28 (using the Cohen d effect size calculator19) with a non-overlap of cohort scores of approximately 21.3%. This effect size, however, did not reach statistical significance on the Mann-Whitney U-test (U=480.5; p<0.0I8).
The average end-of-module knowledge test marks for the 2010 cohort was 49.0% (SD=12.3), with a range from 33.33% to 73.33% (See Figure 2). This average is below the required 50% pass mark. Of the 29 students in the 2010 cohort, 14 (48%) failed this test. The average mark of the 2011 cohort was 52.9% (SD=12.3), with a range from 23.33% to 83.33%. This average was above the required 50% pass mark. Of the 41 students, 16 (39%) students failed this test.
Effect size (Table I) for positive change in end-of-module knowledge test marks from the 2010 to the 2011 cohort was calculated at d=0.31 (using the Cohen d effect size calculator19) with a non-overlap of cohort scores of approximately 21.3%. Again, however, this effect size did not reach statistical significance when tested on the Mann-Whitney U-test (U=489.5; p<0.2I).
The overall module performance average for the 2010 cohort was 120.8 (SD=23.9), with a range from 62.7 to 157.5 (see Figure 2). This average is below the 125 "pass" mark. The average for the 2011 cohort was 129.1 (SD=23.6), with a range from 83.25 to 181.1. The average score of the 2011 cohort shifted above the 125 pass mark.
The change in overall module performance between the 2010 cohort and the 2011 cohort showed the largest effect size measured in this study with d=0.35 (Table I) and a non-overlap of sample scores between 21.3% and 27.4%. This result did not reach statistical significance when tested on the Mann-Whitney U-test (U=482; p<0.I8)
This study set out to investigate whether a change in student behaviour regarding the uptake of blended learning within a PBL module made a difference to their performance in academic and practical student assessments. The 2011 cohort results indicated a significantly greater voluntary uptake of access to the VLE. The results of this study show a promising trend that may have practical significance for educationalists attempting to blend e-learning into a PBL curriculum.
A total of 29 occupational therapy third year students (2010 cohort) entered the PBL module related to Paediatric Learning Disabilities for the first time in 2010, and 41 students (2011 cohort) entered the same module in 2011. There were thus inconsistent sample sizes between the cohorts, which is a typical challenge in educational research18.
The 2011 cohort had a significantly greater use of VLE over the 2010 cohort, as indicated by an increased average number of instances of access (Figure 1). This can be viewed as the students habituating to the blended learning process as they had more experience in the use of the VLE. The 2010 cohort was the first cohort of students to be exposed to the VLE. The lecturers within the department were also novice users and despite training workshops for both the lecturers and students, by the third semester this cohort could still be considered novice users. The 2011 cohort entered the course with prior knowledge of the VLE and the lecturers of all modules were more experienced at the implementation of blended learning. By the third semester these students accessed the VLE habitually. The lecturers were all actively contributing blended learning content and learning activities to all aspects of the occupational therapy curriculum via the VLE during 2011. The statistically significant difference (p<0.002) and large effect size (g=0.73) support the fact that student behaviour related to blended learning changed from the 2010 cohort to the 2011 cohort (Table I). Penman's11 2007 call to consider connectivism as an educational theory for occupational therapy curricula seems to have been achieved by this change in student learning behaviour.
Does the change in student behaviour influence their performance?
The "no significant difference" phenomenon12,13,18 is not uncommon in studies pertaining to the effect of e-learning on educational outcomes, and thus does not support dismissal of observed improvement in student performance as irrelevant. The small cohort sizes (nl=29, n2=4l) within this study further limited the possibility of achieving statistically significant results24 as the number of students entering this module in any given year is dependent on the university's capacity, student enrolment and academic success rates. The degree of change (effect size) may be considered to be more practically significant to educational practice within the domain of occupational therapy22,24.
The changes in student performance during the PBL module in the three independent measures did not achieve statistical significance (Table I). The small population of third year occupational therapy students at this faculty in 2010 and 2011 however preclude the opportunity for small and medium effect sizes to achieve statistical significance24.
In all three independent measures, there was positive change (Figure 2) from failing average marks for the 2010 cohort to passing average marks for the 2011 cohort, and a smaller percentage of students failed the summative student performance measures for the modules in the 20ll cohort. The failure rate dropped by an average of 9.97% for this PBL module, indicating that almost 10% more students using the VLE habitually (2011 cohort) experienced success. In education, the shift from failing averages to passing averages is of practical importance and should thus not be dismissed.
The effect sizes of these student performance changes (Table I) are classified as "small" according to Cohen's guarded classification22, however they still show positive change with the 2011 cohort's average marks placed at the 62nd percentile rank of the 2010 cohort's averages. The average effect change of d=0.3 is better than the d=0.l effect observed in the review of 76 randomised control studies of e-learning versus traditional learning18. It focusses on the single factor of habituation to blended learning specifically within a PBL occupational therapy curriculum, thus meeting Cookl8 and Sims12 call for focussed and contextual studies. Even if the effect change is deemed small, the consequences for students are relevant and can mean the difference between repeating a year of study or progressing to the next year of study.
Limitations of the study
This study context is confined to a single course and the cohort numbers are confined to the number of students enrolled at the university, for that course in the particular cohort years. A power analysis was done to determine the sample size required to detect statistical significance24. The lower the anticipated effect size the greater the required cohort numbers. A small effect size (d=0.3) would require more than 100 student records in each cohort24. The constraints of sample size are a limitation to this study.
Crafting e-learning into an existing PBL curriculum is supported by the premise that connectivism improves motivation and access to individualised learning7,11. It is however, a process that takes time to transition the students and lecturers from novice to habitual VLE users. This study shows that when e-learning activities are well crafted into a PBL module, using a variety or resources and VLE tasks, the students who are habitual users of the VLE perform better in the summative assessments, than students who are novice users.
Thus it can be concluded that integrating e-learning into the larger process of PBL assists students in obtaining and retaining the knowledge and skills they require in their progress to becoming occupational therapists. Blended learning can improve student throughput rates. The study further demonstrates that collaborative integration of e-learning in the context of a South African university can be achieved, despite apparent hesitance in terms of infrastructure, computer literacy of students and lecturers, and diversity of the participants.
1. McCarron KA, D'Amico FF The impact of problem-based learning on clinical reasoning in occupational therapy education. Occupational Therapy in Health Care, 2002; 16 (1): 1-13. [ Links ]
2. McCannon R, Robertson D, Caldwell J, Juwah C, Elfessi A. Comparison of clinical reasoning skills in occupational therapy students in the USA and Scotland. Occupational Therapy International, 2004; 11 (3): 160-76. [ Links ]
3. Barrows HS. A taxonomy of problem-based learning methods. Medical Education, 1986; 20 (6): 481-6. [ Links ]
4. Liotta-Kleinfeld L, McPhee S. Comparison of final exam test scores of neuroscience students who experienced traditional methodologies versus problem-based learning methodologies. Occupational Therapy in Health Care,2001; 14 (3/4): 35-53. [ Links ]
5. Peacock D, Walton G, Booth A. The role of LIS in supporting learning. Managing Knowledge in Health Services London: Facet Publishing, 2004: 99-112. [ Links ]
6. Vardi I, Ciccarelli M. Overcoming problems in problem-based learning: a trial of strategies in an undergraduate unit. Innovations in Education & Teaching International, 2008; 45 (4): 345-54. [ Links ]
7. Savery JR, Duffy TM. Problem based learning: An instructional model and its constructivist framework. Educational Technology-Saddle Brook Nj-, 1995; 35: 31. [ Links ]
8. Ellaway R, Masters K. AMEE Guide 32: e-Learning in medical education Part 1: Learning, teaching and assessment. Medical Teacher, 2008; 30: 455-73. [ Links ]
9. Siemens G. Connectivism: A learning theory for the digital age. International Journal of Instructional Technology and Distance Learning, 2005; 2 (1): 3-10. [ Links ]
10. Kop R, Hill A. Connectivism: Learning Theory of the Future or Vestige of the Past? International Review of Research in Open and Distance Learning, 2008; 9 (3): 1-13. [ Links ]
11. Penman M. Frances Rutherford Lecture Transforming views of learning: Connecting professional conversations. New Zealand Journal of Occupational Therapy, 2007; 54 (2): 4-15. [ Links ]
12. Sims R. Rethinking (e)learning: a manifesto for connected generations. Distance Education,2008; 29: 153-64. [ Links ]
13. Oblinger DG, Hawkins BL. The Myth about No Significant Difference. EDUCAUSE Review, 2006; 41 (6): 14-5. [ Links ]
14. Zhu C, Valcke M, Schellens T. Cultural differences in the perception of a social-constructivist e-learning environment. British Journal of Educational Technology, 2009; 40 (1): 164-8. [ Links ]
15. Sethy SS. Distance Education in the Age of Globalization: An Overwhelming Desire towards Blended Learning. Turkish Online Journal of Distance Education (TOJDE), 2008; 9 (3): 29-44. [ Links ]
16. Ward R, Moule P, Lockyer L. Adoption of Web 2.0 Technologies in Education for Health Professionals in the UK: Where Are We and Why? Electronic Journal of e-Learning, 2009; 7 (2): 165-72. [ Links ]
17. Masters K, Ellaway R. e-Learning in medical education Guide 32 Part 2: Technology, management and design. Medical Teacher, 2008; 30: 474-89. [ Links ]
18. Cook DA. The failure of e-learning research to inform educational practice, and what we can do about it. 2009; 31 (2): 158-62. [ Links ]
19. Wikipedia contributors. "Wiki". Wikipedia, The Free Encyclopedia. 2014. <http://en.wikipedia.org/w/index.php?title=Wiki&olsid+593465119> (18 February 2014). [ Links ]
20. Ellis PD. "Effect size calculators". 2009. <http://www.polyu.edu.hk/mm/effectsizefaqs/calculator/calculator.html> (02 May 2013). [ Links ]
21. Ellis Pd. The essential guide to effect sizes: Statistical power, meta-analysis, and the interpretation of research results: Cambridge University Press, 2010. [ Links ]
22. Cohen J. A power primer. Psychological Bulletin, 1992; 112 (1): 155-9. [ Links ]
24. Tomita M. Statistical Analysis. In: Kielhofner G, editor. Research in Occupational Therapy: Methods of Inquiry for Enhancing Practice. Philidephia: F.A. Davis Company; 2006: 213-325. [ Links ]
Paula Barnard Ashton