On-line version ISSN 2076-3433
Print version ISSN 0256-0100
S. Afr. j. educ. vol.29 n.2 Pretoria May. 2009
Utilising learning environment assessments to improve teaching practices among in-service teachers undertaking a distance-education programme
Jill AldridgeI; Barry FraserII; Sipho NtuliIII
IJill Aldridge is Senior Lecturer at the Science and Mathematics Education Centre at Curtin University of Technology, Australia. Her research is in the field of teaching and learning, especially the use of teacher professional development and the use of action research as a meaningful tool for improvement. She is co-author of the book Outcomes-Focused Learning Environments: Determinants and Effects. E-mail: firstname.lastname@example.org
IIBarry Fraser is Professor and Director of the Science and Mathematics Education Centre at Curtin University of Technology, Australia. He is editor of Springer's Learning Environments Research: An International Journal and co-editor of Sense Publishers' Advances in Learning Environments Research book series and Springer's forthcoming Second International Handbook of Science Education.
IIISipho Ntuli was a doctoral student, at the Science and Mathematics Education Centre at the Curtin University of Technology in Perth, Western Australia, at the time of this study. He was also involved in teaching in an in-service teacher education course at the University of South Africa, Pretoria.
We examined the viability of using feedback from a learning environment instrument to guide improvements in the teaching practices of in-service teachers undertaking a distance-education programme. The 31 teachers involved administered a primary school version of the What Is Happening In this Class? (WIHICPrimary) questionnaire to their 1,077 learners in order to determine preferred and actual classroom environments. Feedback about discrepancies between learners' actual preferred learning environments were used to formulate teaching strategies to reduce discrepancies over a 12-week intervention period. In-service teachers' reports, contact sessions, interviews between teachers and researchers, and three case studies based on classroom visits (one of which is reported here) provided thick descriptions of teachers' reactions to utilising the learning environment instrument. Our research provided the first learning environment study at the primary school level in South Africa, cross-validated an IsiZulu version of the WIHIC when used for the first time in South Africa, and supported the success of teachers' use of a learning environment questionnaire in guiding improvements in their teaching.
The sociopolitical transformation in South Africa has been paralleled by major educational changes, including the Policy Document for Education and Training, which gives a prominent place to open learning and distance education (SAIDE, 1994). A major problem confronting educators and teacher educators in South Africa is the lack of qualified teachers, particularly in rural areas where many teachers are underqualified. To overcome this, professional development programmes have been developed to enable teachers to be engaged in in-service teacher training at a distance. In this study we examined the effectiveness of using feedback from a learning environment instrument to guide improvements in these teachers' teaching practice in their primary schools.
Background and theoretical framework
Several studies in distance education pre-service teacher education reveal that the organisation of practice teaching for teacher trainees presents both logistical and educational difficulties. Despite this, practice teaching is regarded as a pillar of teacher education as it provides opportunities for evaluating both pre-service and in-service teachers in authentic environments (Department of Education, 1996:127). In-service teachers enrolled in distance- education programmes are usually already involved in teaching in their own classrooms. Logistical problems for institutions involved in distance teacher education arise out of a need to observe in-service teachers' work in schools that are at a considerable distance from each other and from the teacher educators' institutions (Perraton, 1993; 2000). Educational difficulties arise when teachers attempt to integrate theory with their actual practice. Teachers also find this integration difficult from a practical point of view (Duschl & Waxman, 1991).
Where observation of teaching practice has been abandoned because of organisational difficulties, various alternatives have been sought (e.g. Admiraal, Lockhorst, Wubbels, Korthagen & Veen, 1998; Fong & Woodruff, 2003; Frey, 2008; Holmes, Karmacharya & Mayo, 1993; Oliveira & Orivel, 1993; Perraton, 1993). In our study, we hypothesised that involving teachers undertaking distance in-service courses in assessing their school classroom environments may add to these alternatives for providing feedback and therefore assist teachers to change their teaching practice.
Research on learning environments has provided a number of ideas and techniques that could potentially be valuable for inclusion in teacher education programs. Whilst research at the university and teacher education levels has been limited, there have been some important studies that have contributed to this area (Duschl & Waxman, 1991; Fisher & Fraser, 1991; Nix, Fraser & Ledbetter, 2005; Yarrow, Millwater & Fraser, 1997).
The research described in this article drew upon and contributed to the field of classroom learning environments (Fisher & Khine, 2006; Fraser, 2007; Khine & Fisher, 2003). 'Learning environment' refers to the tone, ambience or atmosphere created by a teacher through the relationships developed within the classroom and the way in which instruction is delivered. Research in the field of learning environments over the past few decades has often involved associations between students' cognitive and affective learning outcomes and their perceptions of psychosocial characteristics of their classroom environments (Fraser & Fisher, 1982; Haertel, Walberg & Haertel, 1981; McRobbie & Fraser, 1993). Another application of learning questionnaires in past research has been as a source of process criteria of effectiveness in the evaluation of educational innovations (Khoo & Fraser, 2008; Maor & Fraser, 1996; Martin-Dunlop & Fraser, 2008; Teh & Fraser, 1994; Wolf & Fraser, 2008).
We examined the feasibility of using learning environment ideas to guide the improvement of the teaching practice of in-service primary school mathematics teachers undertaking a distance-education course. Past studies in western countries have successfully employed feedback information based on students' perceptions of the actual and preferred learning environment in improving the learning environments created by teachers (Fraser & Fisher, 1986; Sinclair & Fraser, 2002; Thorp, Burden & Fraser, 1994; Yarrow et al., 1997). In many cases, these action research studies have made use of a five-step procedure that involves:
1. Assessment of students' perceptions of their learning environment;
2. providing feedback to the teacher based on students' responses to the questionnaire;
3. reflection and discussion based on feedback;
4. the teacher introducing an intervention over a period of time; and
5. re-administration of the questionnaire to students at the end of the intervention period to determine whether they perceive their learning environment differently from before.
This five-step procedure has been used successfully at a range of educational levels, including the primary school level (Fraser & Deer, 1983; Fraser, Docker & Fisher, 1988), but there have been no previous studies that have used this approach with distance education in-service primary school teachers.
Research aims of the study
1. To modify and validate the What Is Happening In this Class? (WIHIC Primary) questionnaire for assessing primary school learners' perceptions of their classroom environments in South Africa.
2. To examine learners' perceptions of actual and preferred environment in the school classes of in-service teachers attending a distance-education course.
3. To examine the extent to which feedback, based on primary school learners' perceptions on the WIHICPrimary, can guide teachers' improvement of their classroom learning environments.
Research design and methodology
Our study involved primary school teachers enrolled in a two-year mathematics education course, at a teacher education college in the KwaZulu-Natal province of South Africa, which is taught as a distance-education programme. The course offers online sessions as well as a minimum of two full-day face-to-face (contact) sessions each year.
The study incorporated a mixed-methods approach that included both qualitative and quantitative research methods as recommended by Tashakkori and Teddlie (2003) and Tobin and Fraser (1998). Our mixed-methods approach involved collecting and analysing both quantitative and qualitative data. It was recognised that all methods have limitations and that using a range of research methods allows for triangulation of data sources. In this way, the study used observations and interviews to help illuminate the data collected using surveys.
The study involved the interface between the in-service teachers and their primary school learners. During the course of study, teachers were required to assess their learners' perceptions of their actual learning environment and the one that they would prefer. Teachers were then involved in action research aimed at reducing discrepancies between the actual and preferred learning environment as perceived by students.
Few studies in the field of learning environments have been conducted in South Africa. A review of literature reveals that, whilst there have been some studies carried out at high school level in South Africa (Aldridge, Fraser & Sebela, 2004; Aldridge, Laugksch, & Fraser, 2006; Aldridge, Laugksch, Seopa & Fraser, 2005) and one at tertiary level (Adams, 1997), none has been conducted at the primary school level. Therefore, as there was no learning environment questionnaire available that had been field tested and validated for use at the primary school level in South Africa, it was necessary to identify and modify an existing questionnaire for use in the present study.
When a review of literature was carried out to identify questionnaires that could be suitable in the South African context, the What Is Happening In this Class? (WIHIC) questionnaire was selected. The WIHIC (Fraser, McRobbie & Fisher, 1996) brings parsimony to the field of learning environment research by combining modified versions of the most salient scales from a wide range of existing questionnaires with additional scales that accommodate contemporary educational concerns (such as equity and constructivism). The seven original scales were Student Cohesiveness (the extent to which students know, help and are supportive of one another), Teacher Support (the extent to which the teacher helps, befriends, trusts and is interested in students), Task Orientation (the extent to which it is important to complete activities planned and to stay on the subject matter), Involvement (the extent to which students have attentive interest, participate in discussions, do additional work and enjoy the class), Investigation (the extent to which emphasis is placed on the skills and processes of inquiry and their use in problem solving and investigation), Co-operation (the extent to which students co-operate rather than compete with one another on learning tasks) and Equity (the extent to which students are treated equally by the teacher).
The WIHIC has been used in a range of western and non-western countries including Brunei (Riah & Fraser, 1998), Canada (Raaflaub & Fraser, 2002; Zandvliet & Fraser, 2004, 2005), Korea (Kim, Fisher & Fraser, 2000), India (Koul & Fisher, 2005), Singapore (Chionh & Fraser, in press; Khoo & Fraser, 2008), USA (Allen & Fraser, 2007; Martin-Dunlop & Fraser, 2008; Ogbuehi & Fraser, 2007; Wolf & Fraser, 2008), Taiwan (Aldridge & Fraser, 2000) and Indonesia (Margianti, Aldridge & Fraser, 2004). The instrument has also been used across a range of subjects, including mathematics (Ogbuehi & Fraser, 2007) and geography (Chionh & Fraser, in press), science (Aldridge & Fraser, 2000; Wolf & Fraser, 2008) and computing (Zandvliet & Fraser, 2004; 2005). In each of these cases, the WIHIC has been found to be a robust and reliable instrument. The WIHIC has also been used successfully across a range of different educational levels, including higher education (Margianti, Aldridge & Fraser, 2004), high school (Aldridge & Fraser, 2000; Kim, Fisher & Fraser, 2000; Riah & Fraser, 1998; Fraser, McRobbie & Fisher, 1996), middle school (Ogbuehi & Fraser, 2007) and primary school (Allen & Fraser, 2007). In addition, Dorman (2003) used confirmatory factor analysis to support the seven-scale a priori factor structure and international applicability of the WIHIC among a sample of 3,980 students from Australia, Britain and Canada.
Although the robust and reliable nature of the WIHIC in a range of settings and across a number of different cultural contexts made it a sensible choice for the present study, it required a number of modifications to make it suitable for use at the primary school level in South Africa. Initially assistance was sought from teachers attending the in-service course at the college involved in this study.
To facilitate this questionnaire modification process, the teachers were sensitised to the subtle but important aspects of classroom environments by responding to actual and preferred versions of the WIHIC in relation to their current college classrooms. Teachers then commented on the suitability of the WIHIC for their own primary-school learners in South Africa. Teachers had an opportunity to comment on, evaluate, modify or eliminate items of the instrument. Because teachers considered the Investigation scale inappropriate for the primary school level, it was omitted from the questionnaire. The word 'learner' was substituted for 'student' throughout the instrument, as this term had been recommended by the Department of Education.
In addition to making the questionnaire suitable for use with students in South Africa, consideration was also given to making the questionnaire suitable for primary-school-age learners. Whilst it was recognised that reducing the number of items in a scale could threaten scale reliability, it was important to do so to suit the concentration span of primary school-age learners and to ensure that they could cope with the instrument. Therefore, the number of items was reduced from 56 items (seven scales with eight items in each) in the WIHIC's original form to 36 items (six scales with six items in each). Table 1 provides a sample item for each scale for the original and modified versions of the WIHIC.
The five-point frequency response scale used in the original version of WIHIC was also modified to reduce confusion amongst primary-level learners. A three-point frequency scale of Almost Never, Sometimes, and Almost Always was used. As English is the language of instruction for students in Grade 4 and above in South Africa, the teachers felt that it was unnecessary to translate the questionnaire into the native language for learners in Grade 6 and above. Furthermore, in a small pilot study involving administration of the questionnaire to one class followed by individual student interviews, we confirmed Grade 4 students' ability to comprehend the English version of the questionnaire. However, for learners in Grades 4 and 5 who had just started to learn English, the instrument was translated into IsiZulu using a process of translation and back translation (as recommended by Brislin, 1970). This process involved the first author translating the instrument into IsiZulu and then a person conversant in both languages, but not familiar with the questionnaire, translating items back into English. The back translations could then be checked by researchers to ensure that the IsiZulu version maintained the meanings and concepts in the original English version.
Past learning environment studies that have involved attempts at improving the learning environment have included students' responses to actual and preferred versions of a questionnaire. The actual form of a questionnaire assesses students' perceptions of the learning environment that a teacher has created. The preferred version assesses the learning environment that students would prefer to be present. For example, an item in the Teacher Support scale of the actual version reads "The teacher helps me with my work", whereas the parallel preferred version reads "The teacher would help me with my work". We developed a parallel preferred form of the WIHICPrimary.
To ensure its suitability for use with South African primary school learners, the WIHICPrimary was administered to the learners in one Grade 7 class (approximately 40 students). Five learners from the class were selected on the basis of their responses for interviews aimed at determining the comprehensibility and readability of individual items in the questionnaire, and whether items had been interpreted in the way in which the researchers had intended. The results of the pilot study led to some item rewording to ensure the instrument's suitability.
The large sample for the main study involved 1,077 primary-school mathematics learners in the classes of the 31 in-service teachers attending the distance-education course. These classes were all located in rural and semi-rural areas of South Africa and are representative of schools in these areas. All 31 teachers enrolled in the distance-education course were involved in attempts to use the feedback provided from students' responses to the WIHICPrimary in improving the learning environments of their primary school classrooms.
Two major ethical issues were addressed prior to the commencement of the study. The first was related to the teacher participation and the second was related to the student participation. In the first case, because the third author also was one of the lecturers at the university, care was taken to ensure that teachers were aware that their students' responses would not influence their grades. In the second case, it was considered important that students who responded to the questionnaire did not feel that their responses would compromise their situation in the classroom. In both cases, all of the participants were provided with detailed information related to the study. Both the in-service teachers and their students were made aware of their role in the study and their expectations in terms of data collection. It was made clear to both the student teachers and their school students that participation was on a voluntary basis and that they were entitled to withdraw from the study at any time without penalty. One of the key elements in the study was the confidential nature of all responses (to both the questionnaire and interviews).
Collecting and analysing data
The primary school version of the WIHIC questionnaire was administered to a sample of 1,077 Grade 4, 5, 6, and 7 learners in mathematics classes attending 31 schools spread across the province of KwaZulu-Natal in South Africa. The quantitative data were analysed to explore the factor structure, internal consistency reliability and ability to differentiate between the perceptions of students in different classes. The honesty of student responses was an important consideration. To enhance the likelihood of student honesty, students were not required to put their names on their questionnaire, and they were assured that their responses to the questions would be confidential. In addition, the questionnaire was administered by someone other than the teacher.
Descriptive analysis, based on the learners' responses to the WIHIC, was used to describe the classroom environments of mathematics classes. Graphical profiles, for the whole group and for individual class groups for both actual and preferred responses to the WIHIC, were used to identify scales for which the actual-preferred discrepancies were appreciable, and to explore strategies that might be employed by teachers in an attempt to reduce these discrepancies during the intervention stage.
Case studies of three teachers (one of which is reported here) were used to gauge the effectiveness of using profiles to help teachers change the learning environments in their classrooms. The in-service teachers were selected from different schools based on their written reports and participation during face-to-face sessions at the university (as it was important to ensure that selected case-study teachers were prepared to share their experiences). Qualitative data were gathered using observations, interviews and narratives (discussed below) to gain deeper insights into classroom environments and the learners' attitudes to their mathematics classes.
Observations were carried out in each of the classes of the three case-study teachers. Observations were conducted once every two weeks. In most cases, the researcher was a non-participant observer, in which observations were conducted from the back of the classroom with limited interaction with the participants. On one occasion, the third author provided a demonstration and became a participant observer. Observations were conducted to provide information about the strategies being implemented and the students' reactions to these strategies. Information was recorded in the form of extensive field notes. To provide a representation of the classrooms of the case-study teachers, methods were drawn from narrative inquiry (Polkinghorne, 1995). Representation of the voices of the interviewees was of major concern to the authors (Geertz, 1988; Grumet, 1991). In all cases, we ensured that the subjects were represented in a way that was respectful and positive. Finally, we took Brunner's (1994) advice and attempted to use narrative forms that would engage our readers aesthetically as well as critically.
Interviews were held with each of the case-study teachers at the beginning and end of each classroom visit. These interviews helped to clarify the teacher's aims and the observations made during the class visit. These interviews ranged from informal to formal, with the corresponding range of unstructured to structured questions in the interview schedule.
Five primary school learners from the classes of each of the three case-study teachers were also selected for interviews at the beginning, middle and end of the intervention period. The selection of these students was based primarily on their willingness to be involved. Care was also taken to try and provide a sample that was representative of the range of student academic abilities in the classroom. The interviews were semi-structured to allow some flexibility, whilst maintaining a framework that would provide a degree of consistency of response types across the three teachers. Interviews were conducted in the students' mother tongue by the third author who was also the lecturer/facilitator for the distance-education course.
The distance-education course includes four sessions that the teachers are required to attend at the university (known as 'contact' sessions). These sessions were used to introduce and explain the research and to generate discussions with and between teachers that were recorded and later analysed to help to determine whether teachers were able to use feedback effectively to change their learning environments. In addition, reports written by teachers regarding their views about the effectiveness of the exercise helped to provide further insights into the viability of teachers' using feedback from a learning environment instrument to improve their teaching practices.
In this study, qualitative data analysis began at the time of commencement of qualitative data collection and was carried out on an ongoing basis until the end of the study. Analyses occurred as the researcher was synthesising, sifting and selecting relevant data from the field notes. This process evoked new assertions and questions in the form of a recursive review as recommended by Erickson (1998:1162). Throughout the observation and interview process, constructions from both the teachers and one of the researchers were negotiated and refined to establish major themes. Throughout the analyses of the data, the researchers were cautious to observe Guba and Lincoln's (1989) condition of 'willingness to share power' with the teachers. To ensure the trustworthiness of the process, teachers checked the narratives that were produced to ensure the authenticity of the accounts. All of the teachers were asked to comment on the major themes that were established during the contact session to verify their accuracy.
Intervention and action research
The 31 distance in-service teachers, who were attending a distance-education course, each administered the actual and preferred versions of the WIHICPrimary to the learners in one of his/her classes. As part of the in-service course, these teachers received clear guidelines on how to administer the questionnaires. The responses of the learners were analysed to provide each of the in-service teachers with a graphical profile of the learners' scores on the actual and preferred forms of the WIHICPrimary.
During a lecture that students had to attend at the university during the distance-education course, the teachers agreed that they would all like to focus on improving the Involvement scale (based on the results of data collected). Together, the teachers then brain-stormed a range of strategies that they felt could be implemented to improve learners' involvement in the class. The teachers then selected from this list ideas that they would implement in their classes over the following 12-week intervention period.
During this intervention period, each of the teachers was required to attend contact sessions and to submit reports related to their attempts to change the learning environment. Discussions held during the contact sessions were tape-recorded and later transcribed, providing a rich source of data regarding the way in which teachers were coping during the intervention period and how they felt about the exercise as a whole. In addition, teachers were required to submit two reports, one in the middle of the intervention period (which outlined the strategies they had selected) and one at the end of the intervention period (which provided an account of the degree of success of the strategies they had selected to implement and whether they felt that the exercise had been worthwhile). This second report was written in the form of a letter to a friend.
At the end of the intervention period, the actual form of the WIHICPrimary was readministered to the learners (usually by one of the researchers) in the classes of the 31 in-service teachers. Graphical profiles, representing the learners' scores on the pretest actual, posttest actual and preferred forms of the WIHIC, were provided to each of the teachers. These profiles, along with the reports, were used to investigate the viability of using feedback generated from students' responses to the WIHICPrimary to improve teaching practices. Based on considerable past research involving the use of classroom environment instruments in studies with a pretest-posttest design (Fraser, 2007), we were confident that students' posttest responses would be influenced very little by the previous experience of responding during the pretest administration.
Case studies and narratives
Throughout this 12-week intervention period, a case study was constructed for each of three teachers (with one of these case studies reported here) who were visited one morning a week by one of the researchers. During these visits, the researcher observed the teachers and made extensive field notes related to the types of strategies being used by the teacher and the students' reactions to these strategies. Semi-structured, in-depth interviews with the teachers were tape recorded and later transcribed for analysis. These classroom observations and interviews ensured thick descriptions of a qualitative nature. Five primary school learners from each of the case-study classes were also selected for interviews. These interviews were structured and based on items of the WIHIC. Learners' interviews helped us to interpret their responses to the questionnaires and provided richer insights into learners' perceptions of the strategies implemented to improve the learning environment in their class. At the end of the intervention period, the WIHICprimary was readministered by someone other than the classroom teacher (wherever possible this was one of the researchers) to the primary school learners of the 31 teachers to determine whether discrepancies between the actual and preferred learning environments had been reduced. A report, encompassing learners' profiles of actual classroom environments before and after the interventions and learners' preferred learning environments, was obtained from each in-service teacher at the end of the intervention period.
A narrative, based on classroom observations and interviews, was written for each case-study teacher to provide the reader with insights into these classrooms. According to Lincoln and Denzin (1994), 'bricolage', or the piecing together of information from a range of sources, is one way of acquiring understanding in qualitative research. Stake (2002) refers to many scholars within education who have used stories and story telling as a central element in their research. Carter (1993) further asserts that the attractiveness of story in contemporary research on teaching and teacher education is grounded in the notion that story represents a way of knowing and thinking that is particularly suited for explicating the issues with which we deal. Clandinin and Connelly (1996:16) define narrative as "the making of meaning through personal experience by way of a process of reflection in which story telling is a key element and in which metaphors and folk knowledge take their place". We assumed that using narratives was a suitable method by which we could understand what was happening in the classes of these teachers. A commentary, following each of the three narratives, was used to interpret the narratives and to make sense of the responses and actions, as recommended by Polkinghorne (1995). These commentaries provided a second layer of representation that added meaning and insight into what was happening in the classes of these three teachers.
Analyses and results
Validity and reliability of the WIHICPrimary
Our first aim in the present study was to modify and validate the What Is Happening In this Class? (WIHIC) questionnaire to assess primary school learners' perceptions of the learning environment in South Africa. Data collected from the 1,077 learners in 31 classes were analysed to investigate the reliability and validity of the primary-school version of the WIHIC. Principal components factor analysis followed by varimax rotation confirmed a refined structure for the instrument comprising 19 items in four scales (see Table 2). With the exception of Item 19, all items had a loading of at least 0.30 on their a priori scale and no other scale. The percentage of variance and eigenvalues are reported at the bottom of Table 2. For the four WIHICPrimary scales, the percentage of variance ranged between 6.14% and 9.41%, with the total being 32.36%, and eigenvalues ranged between 1.17 and 1.79 for the four WIHICPrimary scales.
To examine whether the items in a scale assessed the same construct, the internal consistency reliability was calculated (see Table 3). For the actual form of the WIHICPrimary, internal consistency (Cronbach alpha reliability) estimates for different scales ranged from 0.68 to 0.72 using the individual as the unit of analysis and from 0.85 and 0.94 using the class mean as the unit of analysis. For the preferred form of the WIHICPrimary, internal consistency reliability estimates ranged from 0.52 to 0.57 with the individual as the unit of analysis and from 0.86 to 0.88 with the class mean as the unit of analysis.
To determine whether the actual form of each WIHICPrimary scale could differentiate between the perceptions of learners in different classes, an analysis of variance (ANOVA) was calculated for each scale, involving class membership as the independent variable and the individual learner as the unit of analysis. The results in Table 3 suggest that all scales differentiated significantly between primary school mathematics classes (p < 0.01). Thus, students within the same class perceived the classroom learning environment in a relatively similar manner, while within-class mean perceptions of the students varied between classes. The eta2 statistic (calculated to provide an estimate of the strength of association between class membership and the dependent variable) ranged from 0.41 to 0.49 for different WIHICPrimary scales (see Table 3).
Differences between learners' perceptions of actual and preferred learning environment
Primary learners were asked to indicate not only the extent to which a practice takes place in their classrooms but also the extent to which they would prefer it to take place. Multivariate analysis of variance (MANOVA) with repeated measures (using the class mean as the unit of analysis) was used to investigate whether differences between actual and preferred scale scores were statistically significant. When the multivariate test (Wilks' lambda) revealed significant actual-preferred differences overall, the ANOVA with repeated measures was interpreted for each individual WIHICPrimary scale (see Table 4). For all scales, learners preferred a more favourable level of each WIHICPrimary scale than was currently perceived to be present. Furthermore, actual-preferred differences were statistically significant (p < 0.01) for three of the four WIHICprimary scales, namely, Involvement, Task Orientation and Equity. These results replicate numerous studies worldwide, which have reported that learners would prefer a learning environment more favourable than the one perceived as being present (Fraser, 1998; 2002).
To examine the magnitudes of these actual-preferred differences, as well as their statistical significance (as recommended by Thompson, 1998; 2001), effect sizes were calculated in terms of the differences in means divided by the pooled standard deviation. The effect sizes, for those scales with statistically significant differences, ranged between approximately three quarters of a standard deviation (0.78) and two-and-a-half standard deviations (2.43). These results suggest educationally important differences between learners' perceptions of the actual classroom learning environment and that which they would prefer.
Action research: closing the gap between actual and preferred learning environments
The third aim of the present study was to examine the extent to which feedback, based on primary school learners' perceptions on the WIHICPrimary, can guide teachers' attempts to improve their classroom learning environments. To provide a basis for meaningful discussions, the researchers provided feedback to the 31 in-service teachers in the form of a graphical profile depicting the overall mean scores for the students of the combined group of in-service teachers (N = 1,077 students). In-service teachers were also provided with graphical profiles for their individual classes that could be used as the basis for reflection and the selection of strategies. During the contact session, teachers were provided with information about how to interpret their profiles and small-group discussions allowed teachers to discuss their profiles collaboratively in a non-threatening environment.
The results for the total sample (discussed in the previous section), showing learners' scores on the actual and preferred forms of the WIHICPrimary, were shared with the 31 in-service teachers during a contact session at the beginning of the academic year. The teachers were then given time to discuss the combined results, as well as the profile for their own classroom. The teachers all decided that they would like to close the gap between what students perceive and prefer, and they all felt that they would like to implement strategies that focused specifically on improving students' perceptions on the Involvement scale. During the contact session, teachers were encouraged to brain-storm a range of strategies and ideas that they could use in their own mathematics classes.
During the 12-week intervention period that followed this contact session, three of the 31 teachers were selected for case studies (one of which is reported here), with the remainder of the teachers having contact only through telephone or individual visits by the teachers to the college where the distance-education course was being offered. The classes of the three case-study teachers were visited on a weekly basis, when data were collected through classroom observations and interviews with teachers and students. To record the attempts made by the teachers to improve the level of Involvement in their classrooms, a narrative, based on interviews and observations of classroom life during the intervention period, was written for each case- study teacher. However, for economy, we report the case study for only one of these three teachers. After we provide the narrative of this teacher (see Figure 1), we next provide a commentary about the situations described in the narrative, thus providing a second layer of representation (Polkinghorne, 1995). This section also examines data collected from interviews with the other teachers who attended the in-service course and attempted to make changes to the level of Involvement in their classrooms.
Commentary Teacher A
Teacher A is an experienced teacher who is confident and enthusiastic about implementing new ideas. She was excited about the mathematics content and new concepts that she was learning in her teacher-upgrading course. Likewise, she appears to have engaged into this action research project with similar enthusiasm and confidence.
Before the commencement of the intervention period, this teacher had formulated a number of strategies that she intended to implement to involve her learners more. Her major goal was to emphasise practical work more in her lessons. As she also taught science to the same learners, she felt that this was feasible. The story in Figure 1 describes the observation of a lesson on polyhedra during which she constructed three-dimensional shapes with her learners. In the past, she had done such constructions herself, but she felt that the learners' familiarity with such hands-on experiences in science lessons helped. Throughout the intervention phase, Teacher A's initiative and use of hands-on activities were impressive.
Teacher A also tried hard to encourage the learners to help each other in their groups and to explain their findings to one another. During the observations, Teacher A repeatedly encouraged learners to do this. She explained to the learners the importance of involving themselves in their learning and preceded each mathematics lessons with games that were aimed at encouraging learners to talk to one another. In addition, she made a point of encouraging learners to share their findings. "Come on, show us how you got the number of edges in your box ...tell your classmates. Can you see how he avoids repeating the counting of the edge that has already been counted?" On each occasion, she would praise the learners for their efforts before asking them to sit down.
During a conversation after one of my classroom observations, the teacher told one researcher that she had tried to encourage her learners to express themselves and talk to one another, but she felt that the IsiZulu culture could be inhibiting the learners. She felt that, because of respect in the IsiZulu culture, children could be reluctant to talk freely to their elders. The teacher felt that the learners' reluctance to speak out in class was likely to be an assertion of IsiZulu culture. Such instances provided insight into life inside the classrooms of teachers who, whilst understanding the benefits of involving learners in teaching and learning, sometimes come up against cultural artefacts and historical background that impinge upon their ability to change the learning environment.
Although she felt that she was not experiencing the success that she would have liked, she still felt that the strategies that she had used had contributed to building confidence in her learners and helping them to realise the importance of discussing things with each other in class. She stated that "the kids are beginning to realise the importance of talking to one another and with me about what they are learning". Despite the cultural difficulties encountered by Teacher A, she was able to implement strategies that overcame these.
The graphical profile for Teacher A indicated that pretest scores for actual and preferred learning environment reflected some discrepancies (the largest being for the Involvement scale). According to the posttest scores (shown in Figure 2), learners perceived considerably more Involvement at the end of the intervention period than at the beginning. In addition, it would appear that the strategies implemented by Teacher A also influenced the levels of Teacher Support, Task Orientation, and Equity in the classroom.
Teacher A expressed her satisfaction with the way in which her learners responded to the questionnaires and how this made her change her teaching. She stated that "there were many things that I tried to change in my teaching of mathematics". She also felt that the project enabled her to integrate mathematics better with other subjects (particularly science), a point that had been recommended in Outcomes-Based Education workshops that she had attended. This teacher was so impressed by the success of using feedback based on the questionnaire as a means of improving her teaching that she also administered it in her science classes. She allowed the learners to analyse each item in the Involvement scale in relation to what they did in science.
Pre-post changes in classroom environment for other teachers
The case-study teacher described above was drawn from a class of 31 in-service teachers attending a distance-education course. The teachers in the larger group administered the actual and preferred form of the WIHICPrimary to their primary school children and attempted to improve the emphasis on Involvement in their learning environments. After the 12-week intervention period, the actual form of the WIHICPrimary was readministered to determine whether learners' perceptions of the degree of Involvement in the learning environment had improved. The overall posttest results for the group of 31 in-service teachers, in addition to their comments, indicated that the information provided through learners' perceptions of the actual and preferred learning environment generally was used successfully in improving the learning environment. Most of the 31 teachers appeared to have similar reactions, with more than two thirds of them confessing to being negative about the project and all that it entailed at the beginning. Interviews with the teachers indicated that this initial reaction could have arisen because neither the teachers nor the students had experienced any form of involvement in the classroom and, therefore, were concerned about a range of management and behavioural problems.
As the project drew to a close, however, more than half of the teachers felt that their involvement in it had improved their teaching. One teacher stated: "I found that the project was related to my classroom practice and assisted me to improve my teaching ...". Another teacher stated that other teachers at the school in which he was teaching had noted a change in his teaching practice: "Other educators in our school appreciated our course ... because it enabled us to reflect on our teaching practice ... and to improve our teaching of mathematics so that our learners could learn better". Another admitted: "In my school, my colleagues were very interested. One teacher even took copies of the questionnaire and said that she was going to use them in her class". One teacher summed this up when he said: "At first, I really thought that it was a waste of time. But, as I continued to work with the learners, I could see that the project was capable of showing me where I was lacking".
The teachers, all of whom concentrated on improving the level of classroom Involvement, appeared to recognise the value of giving students the opportunity to express themselves (something that many of them had not previously experienced). One teacher stated: "I noticed that the learners were surprised to get the chance to express their views in the classroom with the permission of a teacher". Another commented that "my learners enjoyed bringing up their own ideas, helping one another and getting involved in their learning".
With reference to administering the questionnaire to students for feedback on the learning environment, many of the teachers felt that this was a valuable opportunity for self-assessment and reflection. One teacher said: "The project gave me a chance to assess myself. I was able to reflect on my teaching practice in mathematics". Another commented: "I ... had an opportunity to diagnose myself. I realized that previously I did not give students a chance to express their ideas or ask questions". Another teacher stated: "What impressed me about the project was the fact that I, as a teacher, got to know how I taught through learners' responses". Although most of the teachers were able to use student responses (around 75%) to the WIHICPrimary to close the gap between students' perceptions of the actual and preferred environment, there were some teachers for whom this was not the case. It would appear that, in many cases, these teachers did not understand the objectives of the study and, in some cases, they did not share a similar understanding of what it means for students to be involved in their class.
Discussion and conclusion
A major contribution of the present study is the modification and validation of a questionnaire in the IsiZulu language for assessing and improving primary school students' perceptions of their actual and preferred classroom learning environments in South Africa. A limited number of studies of the learning environment have been conducted at the primary-school level around the world and this was the first at this level in South Africa. The primary- school version of the What Is Happening In this Class? (WIHICPrimary) questionnaire measures four dimensions that are important in classrooms that are outcomes-based, namely, Teacher Support, Involvement, Task Orientation and Equity. The questionnaire has 19 items, a three-point frequency scale, and parallel actual and preferred forms. It takes only around 20 minutes to administer each form.
The WIHICPrimary displayed satisfactory factorial validity. At both the individual and class mean levels of analysis, the internal consistency reliability was satisfactory for both the actual and preferred versions. Further analyses supported the ability of the actual form of each scale to differentiate between classrooms.
MANOVA with repeated measures revealed large differences between students' perceptions of the actual and preferred learning environment. For three of the four scales (Involvement, Task Orientation and Equity), students preferred a more favourable learning environment from that which they perceived to be actually present.
We also explored whether teachers enrolled in a distance-education in-service course could use information collected by means of the WIHIC Primary to reduce the gap between their students' perceptions of actual and preferred learning environment. A group of 31 teachers was involved in trying to improve scores on the Involvement scale. Three of these teachers were selected as the basis of case studies, one of which was reported here.
Different teachers were able to use feedback gained from the WIHIC Primary with varying degrees of success. Interestingly, the amount of resources available in the classroom did not appear to be a contributing factor to the degree of success that teachers experienced. Despite having few resources, some teachers were able to involve their students successfully in the mathematics lessons by providing them with opportunities to work in small groups, discuss their ideas and understandings with each other, and solve problems on their own.
Those teachers who did not experience success appear either not to have understood the objectives of the project or not to share a similar view about what student involvement might entail. Future use of this questionnaire in action research in a distance-education course, therefore, would need to ensure that all teachers shared a common understanding.
Our study in South Africa could be of practical significance to educators in other countries around the world. Teacher education at a distance is fraught with problems associated with evaluating the practical aspects of teaching. The use of use of students' perceptions on a learning environment instrument could be used to help in addressing the difficulties associated with the evaluation of the practical skills and of attempts to implement new teaching strategies in the classroom.
The results of the present study offer promise for the use of students' perceptions of the learning environment as guide for teacher improvement. However, as this study involved only one class of in-service distance education teachers, the generalisability of the findings could be limited. It is recommended, therefore, that further similar research be carried out.
Further, this study is relevant to other teacher education situations. Whilst research has been carried out in the western world with pre-service teachers, this type of research has not been carried out in developing countries. Therefore, it is recommended that this study be extended to include pre-service teachers to help to sensitise them to the needs and views of their students.
The reflective nature of this study, that involved encouraging teachers to examine their teaching practices through the eyes of their students, offers promise. The results of our research suggest that generally teachers are able to use feedback based on students' perceptions of the learning environment in a meaningful and constructive way that can enhance their teaching practice.
Adams WE 1997. Science laboratory environment in a South African college of education: The effect of class membership. South African Journal of Education, 17:49-52. [ Links ]
Admiraal WF, Lockhorst D, Wubbels T, Korthagen FAJ & Veen W 1998. Computer-mediated communication environments in teacher education: Computer conferencing and supervision of in-service teachers. Learning Environments Research, 1:59-74. [ Links ]
Aldridge JM & Fraser BJ 2000. A cross-cultural study of classroom learning environments in Australia and Taiwan. Learning Environments Research, 3:101-134. [ Links ]
Aldridge JM, Fraser BJ & Sebela MP 2004. Using teacher action research to promote constructivist learning environments in South Africa. South African Journal of Education, 24:245-253. [ Links ]
Aldridge JM, Laugksch RC & Fraser BJ 2006. School-level environment and outcomes-based education in South Africa. Learning Environments Research, 9:123-147. [ Links ]
Aldridge JM, Laugksch RC, Seopa MA & Fraser BJ 2005. Development and validation of an instrument to monitor the implementation of outcomes-based learning environments in science classrooms in South Africa. International Journal of Science Education, 9:123-147. [ Links ]
Allen D & Fraser BJ 2007. Parent and student perceptions of the classroom learning environment and its association with student outcomes. Learning Environments Research, 10:67-82. [ Links ]
Brislin R 1970. Back translation for cross-cultural research. Journal of Cross-Cultural Psychology, 1:185-216. [ Links ]
Brunner D 1994. Inquiry and reflection: Framing narrative practice in education. Albany, NY: State University of New York Press. [ Links ]
Carter K 1993. The place of story in the study of teaching and teacher education. Educational Researcher, 22:5-12. [ Links ]
Chionh YH & Fraser BJ in press. Classroom environment, attitudes, achievement, and self esteem in geography and mathematics in Singapore. International Research in Geographical and Environmental Education. [ Links ]
Clandinin DJ & Connelly FM 1996. Teachers' professional knowledge landscapes: Teacher stories stories of teachers school stories stories of schools. Educational Researcher, 25:24-30. [ Links ]
Department of Education 1996. Norms and standards for teacher educators: Technical committee on the revision of norms and standards for educators. Pretoria: Government Printer. [ Links ]
Duschl RA & Waxman HC 1991. Influencing the learning environment of student teaching. In: Fraser BJ & Walberg HJ (eds). Educational environments: Evaluation, antecedents and consequences (pp. 255-270). London: Pergamon. [ Links ]
Dorman JP 2003. Cross-national validation of the What Is Happening In this Class? (WIHIC) questionnaire using confirmatory factor analysis. Learning Environments Research, 6:231-245. [ Links ]
Erickson F 1998. Qualitative research methods for science education. In: Fraser BJ & Tobin KG (eds). The international handbook of science education (pp. 1155-1173). Dordrecht, The Netherlands: Kluwer Academic Publishers. [ Links ]
Fisher DL & Fraser BJ April 1991. Incorporating learning environment ideas into teacher education: An Australian perspective. Paper presented at the annual meeting of the American Educational Research Association, Chicago. [ Links ]
Fisher DL & Khine MS (eds) 2006. Contemporary approaches to research on learning environments: World views. Singapore: World Scientific. [ Links ]
Fong C & Woodruff E 2003. Web-based video and frame theory in the professional development of teachers: Some implications for distance education. Distance Education, 24:195-211. [ Links ]
Fraser BJ 1998. Science learning environments: Assessment, effects and determinants. In: Fraser BJ & Tobin KG (eds). The international handbook of science education (pp. 527-564). Dordrecht, The Netherlands: Kluwer Academic Publishers. [ Links ]
Fraser BJ 2002. Learning environments research: Yesterday, today and tomorrow. In: Goh SC & Khine MS (eds). Studies in educational learning environments: An international perspective (pp. 1-26). Singapore: World Scientific. [ Links ]
Fraser BJ 2007. Classroom learning environments. In: Abell SK & Lederman NG (eds). Handbook of research on science education (pp. 103-124). Mahwah, NJ: Lawrence Erlbaum. [ Links ]
Fraser BJ & Deer CE 1983. Improving classrooms through use of information about learning environment. Curriculum Perspectives, 3:41-46. [ Links ]
Fraser BJ, Docker JG & Fisher DL 1988. Assessing and improving school climate. Evaluation and Research in Education, 2:109-122. [ Links ]
Fraser BJ & Fisher DL 1982. Predicting student outcomes from their perceptions of classroom psychosocial environment. American Educational Research Journal, 19:498-518. [ Links ]
Fraser BJ & Fisher DL 1986. Using short forms of classroom climate instruments to assess and improve classroom psychosocial environment. Journal of Research in Science Teaching, 5:387-413. [ Links ]
Fraser BJ, McRobbie CJ & Fisher DL April, 1996. Development, validation and use of personal and class forms of a new classroom environment instrument. Paper presented at the annual meeting of the American Educational Research Association, New York. [ Links ]
Frey T 2008. Determining the impact of online practicum facilitation for in-service teachers. Journal of Technology and Teacher Education, 16:181-210. [ Links ]
Geertz C 1988. Works and lives: The anthropologist as author. Stanford, CA: Stanford University Press. [ Links ]
Grumet M 1991. The politics of personal knowledge. In: Witherell C & Noddings N (eds). Stories lives tell: Narrative and dialogue in education (pp. 67-77). New York: Teachers College Press. [ Links ]
Guba EG & Lincoln YS 1989. Fourth generation evaluation. Newbury Park, CA: Sage. [ Links ]
Haertel GD Walberg HJ & Haertel EH 1981. Socio-psychological environments and learning: A quantitative synthesis. British Educational Research Journal, 7:2736. [ Links ]
Holmes DR, Karmacharya DM & Mayo JK 1993. Radio education in Nepal. In: Perraton H (ed.). Distance education for teacher training (pp.136-195). New York: Routledge. [ Links ]
Khoo HS & Fraser BJ 2008. Using classroom psychosocial environment in the evaluation of adult computer application courses in Singapore. Technology, Pedagogy and Education, 17:53-67. [ Links ]
Kim HK, Fisher DL & Fraser BJ 2000. Classroom environment and teacher interpersonal behaviour in secondary science classes in Korea. Evaluation and Research in Education, 14:3-22. [ Links ]
Khine MS & Fisher D (eds) 2003. Technology-rich learning environments: A future perspective. Singapore: World Scientific. [ Links ]
Koul RB & Fisher DL 2005. Cultural background and students' perceptions of science classroom learning environment and teacher interpersonal behaviour in Jammu, India. Learning Environments Research, 8:195-211. [ Links ]
Lincoln YS & Denzin NK 1994. The fifth moment. In: Denzin NK & Lincoln YS (eds). Handbook of qualitative research (pp. 575-586). Thousand Oaks, CA: Sage. [ Links ]
Maor D & Fraser BJ 1996. Use of classroom environment perceptions in evaluating inquiry-based computer assisted learning. International Journal of Science Education, 18:401-421. [ Links ]
Margianti ES, Aldridge JM & Fraser BJ 2004. Learning environment perceptions, attitudes and achievement among private Indonesian university students. International Journal of Private Higher Education. Available at: www.xaiu.com/xaiujournal. [ Links ]
Martin-Dunlop C & Fraser BJ 2008. Learning environment and attitudes associated with an innovative science course designed for prospective elementary teachers. International Journal of Science and Mathematics Education, 6:163-190. [ Links ]
McRobbie CJ & Fraser BJ 1993. Associations between student outcomes and psychosocial science environment. Journal of Educational Research, 87:78-85. [ Links ]
Nix RK, Fraser BJ & Ledbetter CE 2005. Evaluating an integrated science learning environment using the Constructivist Learning Environment Survey. Learning Environments Research, 8:109-133. [ Links ]
Ogbuehi PI & Fraser BJ 2007. Learning environment, attitudes and conceptual development associated with innovative strategies in middle-school mathematics. Learning Environments Research, 10:101-114. [ Links ]
Oliveira J & Orivel F 1993. Logos II in Brazil. In: Perraton H (ed.). Distance education for teacher training (pp. 69-94). New York: Routledge. [ Links ]
Perraton H 1993. The effects. In: Perraton H (ed.). Distance education for teacher training (pp. 391-401). New York: Routledge. [ Links ]
Perraton H 2000. Open and distance learning in the developing world. London: Routledge. [ Links ]
Polkinghorne D 1995. Narrative configuration in qualitative analysis. In: Hatch J & Wisniewski R (eds). Life history and narrative (pp. 5-23). Bristol, PA: Falmer Press. [ Links ]
Raaflaub CA & Fraser BJ April, 2002. Investigating the learning environment in Canadian mathematics and science classes in which laptop computers are used. Paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA. [ Links ]
Riah H & Fraser BJ April, 1998. The learning environment of high school chemistry classes. Paper presented at the annual meeting of the National Association for Research in Science Teaching, San Diego, CA. [ Links ]
South African Institute for Distance Education (SAIDE) 1994. Open learning and distance education in South Africa. Report on an International Commission. SAIDE South African Institute for Distance Education on behalf of the African National Congress. Manzini, Swaziland: Macmillan Boleswa. [ Links ]
Sinclair BB & Fraser BJ 2002. Changing classroom environments in urban middle schools. Learning Environments Research, 5:301-328. [ Links ]
Stake RE 2002. Case-studies. In: Denzin NK & Lincoln YS (eds). Handbook of qualitative research, 2nd edn. (pp. 435-454). Thousand Oaks, CA: Sage. [ Links ]
Tashakkori A & Teddlie C Eds. 2003. Mixed methodology: Combining qualitative and quantitative approaches. Thousand Oaks, CA: Sage. [ Links ]
Teh G & Fraser BJ 1994. An evaluation of computer-assisted learning in terms of achievement, attitudes and classroom environment. Evaluation and Research in Education, 8:147-161. [ Links ]
Thompson B 1998. Review of 'what if there were no significance tests?'. Educational and Psychological Measurement, 58:334-346. [ Links ]
Thompson B 2001. Significance, effect sizes, stepwise methods and other issues: Strong arguments move the field. Journal of Experimental Education, 7:80-93. [ Links ]
Thorp H, Burden RL & Fraser BJ 1994. Assessing and improving classroom environment. School Science Review, 75:107-113. [ Links ]
Tobin K & Fraser B (eds) 1998. Qualitative and quantitative landscapes of classroom learning environments. In: Fraser BJ & Tobin KG (eds). The international handbook of science education (pp. 623-640). Dordrecht, The Netherlands: Kluwer Academic Publishers. [ Links ]
Wolf SJ & Fraser BJ 2008. Learning environment, attitudes and achievement among middle-school science students using inquiry-based laboratory activities. Research in Science Education, 38:321-341. [ Links ]
Yarrow A, Millwater J & Fraser BJ 1997. Improving university and primary school classroom environments through pre-service teachers' action research. International Journal of Practical Experiences in Professional Education, 1:68-93. [ Links ]
Zandvliet DB & Fraser BJ 2004. Learning environments in information and communications technology classrooms. Technology, Pedagogy and Education, 13:97-123. [ Links ]
Zandvliet DB & Fraser BJ 2005. Physical and psychosocial environments associated with networked classrooms. Learning Environments Research, 8:1-17. [ Links ]