SciELO - Scientific Electronic Library Online

 
vol.13 issue1Exploring barriers that nurses experience to enrolment for a postgraduate nursing qualification at a higher education institution in South Africa author indexsubject indexarticles search
Home Pagealphabetic serial listing  

Services on Demand

Article

Indicators

Related links

  • On index processCited by Google
  • On index processSimilars in Google

Share


African Journal of Health Professions Education

On-line version ISSN 2078-5127

Afr. J. Health Prof. Educ. (Online) vol.13 n.1 Pretoria Apr. 2021

http://dx.doi.org/10.7196/AJHPE.2021.v13i1.1273 

RESEARCH

 

Validation of a questionnaire evaluating the effect of a preparatory year on qualifying students for studying at health professions education faculties

 

 

Mohammed Al-RabiaI; Lana Al ShawwaII; Enas GoudaIII; Ahmed AldarmahiIV; Hani AsfourV; Hani AtwaIII, VI

IMD, PhD; Faculty of Medicine, King Abdulaziz University, Jeddah, Saudi Arabia
IIPhD; Faculty of Medicine, King Abdulaziz University, Jeddah, Saudi Arabia
IIIMD, PhD; Faculty of Medicine, Suez Canal University, Ismailia, Egypt
IVPhD; College of Medicine, King Saud Bin Abdulaziz University for Health Sciences (KSAU-HS), Ministry of National Guard-Health Affairs (MNG-HA), Jeddah, Saudi Arabia
VPhD; Faculty of Medicine, King Abdulaziz University, Jeddah, Saudi Arabia
VIMD, PhD; College of Medicine and Medical Sciences, Arabian Gulf University, Manama, Bahrain

Correspondence

 

 


ABSTRACT

BACKGROUND. The preparatory year or first-year experience in higher education aims to consistently meet the needs of students, institutions and broader society. It aims to help students transition from high school to university, acquaint them with the various academic disciplines at university and introduce them to the university environment.
OBJECTIVES. To validate a newly developed instrument that evaluates the effect of the preparatory year on preparing students for studying in health professions education faculties.
METHODS. A descriptive cross-sectional study was conducted at King Abdulaziz University, Saudi Arabia, on a comprehensive sample of male and female second- and third-year students who completed their preparatory year and started studying at health professions education faculties. Data were collected through a questionnaire newly developed by the authors. Descriptive statistics were used and statistical significance was set at p<0.05.
RESULTS. The study showed high internal consistency of the questionnaire. Cronbach's alpha coefficient value for the total scale was 0.94. Among the 5 dimensions of the questionnaire, the students were highly satisfied with 'university conduct', but less so with the 'perception of teachers, teaching and assessment'.
CONCLUSION. There was good validity and reliability of the newly developed questionnaire. From a student's perspective, the preparatory-year programme needs to be revisited.


 

 

The preparatory year or first-year experience in higher education is dynamic and contextually specific, and consistently aims to meet the needs of students, institutions and the broader society of which they are part.[1] Preparatory programmes are considered by some educational systems globally as one of the best practices in higher education.[2]

The preparatory year aims to help the student to transition from the high school system of teaching and learning to that of the university, acquaint them with the various academic disciplines and introduce them to the university environment before they decide on their future fields of study.[3] The year also prepares them psychologically for their prospective fields of study in subsequent years.[4]

Furthermore, it offers intensive training courses to set students on the right track towards their professional careers and enrich their cultural background.[4] One established aim of the preparatory year is to enable new students to explore the academic disciplines at the university, and to familiarise them with the campus environment. As a result, students are well prepared to act as meaningful contributors in their personal and professional lives at university and beyond.[4]

Although considerable attention is given in the higher education literature to the preparatory year, most universities did not practise this trend seriously.[5] Therefore, the aim of the current study was to validate a newly developed instrument that evaluates the effect of the preparatory year on students for studying in health professions education faculties.

Findings of this study had a great impact on reforming preparatory-year programmes, not only for the health professions but also for the concept in any other similar programmes. This study will direct the attention of Saudi Arabian medical educators to reform initiatives and to consider implementing changes in the existing structure of preparatory years.

 

Methods

This was a cross-sectional descriptive study conducted at King Abdulaziz University (KAU), Saudi Arabia.

The sample comprised male and female second- and third-year students who completed their preparatory year and started studying at the Faculty of Medicine, Faculty of Dentistry, Faculty of Pharmacy, Faculty of Applied Medical Sciences and Faculty of Nursing. Sampling was comprehensive - all students were invited to participate in the study. Of the 717 students in the second and third year who completed their preparatory year, 633 responded and completed the questionnaire (88.3%).

Instrument

A questionnaire was developed to include items that addressed the perception and opinion of students regarding the current status of the preparatory year. The questionnaire was based on a 5-point Likert scale and included 32 items evaluating different aspects of the preparatory year from the viewpoint of students.

To establish validity and reliability, the questionnaire was piloted on a small sample of students (n=100) (who were not included in the study population) prior to using it on a wide scale. Based on piloting, some items were rephrased for clarification.

Exploratory factor analysis (EFA) was performed to identify the number of factors that could explain most of the common variance. Reliability of the questionnaire was tested for internal consistency, using Cronbach's alpha.

Data analysis

SPSS version 22 (IBM Corp., USA) was used. Descriptive statistics, validation and reliability studies were performed and statistical significance was set at p<0.05.

Internal consistency for each scale was analysed using Cronbach's alpha. Missing data were replaced with means of missing variables.

EFA was used to test the psychometric proprieties of the questionnaire. Principal component analysis with varimax rotation was performed to identify the different factors. The number of factors extracted and used was based on Kaiser's criterion, which considers factors with an eigenvalue >1.00 as common factors,[6] the scree test criterion to identify the point of inflexion displayed by the scree plot[7] and the cumulative percent of variance. In humanities research, the explained variance is commonly as low as 50 - 60%.[8]

Factor solutions retained according to the psychometric criteria were then subjected to analysis in line with the following interpretability criteria:[9

A given factor contains at least 3 variables with significant loadings, a loading of 0.30 being suggested as the cut-off point.

Variables loading on the same factor share the same conceptual meaning.

Variables loading on different factors appear to measure different constructs.

The rotated factor pattern demonstrates 'simple structure', i.e.:

Most variables load relatively high on only one factor and low on the other factors.

Most factors have relatively high factor loadings for some variables and low loadings for the remaining ones.

Ethical approval

This study was part of a research project that was approved by the Research Ethics Committee of the Faculty of Medicine, King Abdulaziz University (ref. no. 155-16).

 

Results

To determine the suitability of the questionnaire, validity and reliability studies were conducted.

Validity study

Validation of the questionnaire was done through the following methods.

Face validity

The questionnaire was given to a group of preparatory-year programme experts and also to the Medical Education Department at KAU, as well as to an external expert in medical education. They were asked to review the questionnaire from the structural aspect, the different dimensions, and the students' viewpoints.

Modifications of some items were done based on their recommendations. For example, some items, which were not set out clearly, could be misunderstood by students and were thus reformulated.

Construct validity: Exploratory factor analysis

Checking the suitability of data for factor analysis

Sample size: N=633 participants, which is adequate for factor analysis.

Factorability of the correlation matrix: the correlation matrix revealed statistically significant, moderate correlations among the observed variables used in the analysis. None of the correlation coefficients was large. Therefore, there was no need to eliminate any variables at that stage.

Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy and Bartlett's test of sphericity: these revealed that the KMO measure of sampling adequacy was 0.94 (excellent). This value indicates that there were sufficient items predicted by each factor. Furthermore, Bartlett's test of sphericity was statistically significant (p<0.001), which indicated that the variables were significantly correlated. Therefore, this output indicated the appropriateness of the data for factor analysis.

Extraction of factors

Principal component analysis with varimax rotation was performed to identify and interpret the number of factors that could explain most of the common variance and to delete non-reflective or redundant items. Results revealed that the 32 items of the questionnaire resulted in 6 factors with an eigenvalue >1.00. The 6 factors that emerged from factor analysis accounted for 62.6% of total variance. The number of factors was also confirmed with the visual inspection of the scree plot that indicated a sudden drop in the scree, beginning with the sixth factor.

Rotation of factors

From the initial 32 items, 2 were removed from the analysis. The rules used for deleting items were the following:

Number of items per factor: a factor with <3 items is generally weak and unstable; >4 items are desirable and indicate a solid factor.

Cross-loadings of items: items that load at >0.3 on >2 factors.

Factor loading <0.30: lower factor loadings demonstrate a lower degree of association between the factor and the item.

Items 22 and 23 were deleted, as these were only 2 items in 1 factor (weak factor).

Finally, the questionnaire was composed of 30 items distributed on 5 factors. The factors were named according to the heaviness of loading of the statements on each factor and also on the idea behind the statements.

Table 1 shows that the highest contribution was from the College of Medicine (25.4%) and the lowest from the College of Pharmacy (14.7%). The majority of respondents were females (n=207; 58.8%), whereas overall there were 41.2% respondents.

 

 

Appendix 1 (http://ajhpe.org.za/public/files/1273-a1.pdf) shows the following:

Factor 1 explained 13.696% of the variance in responses, with an eigenvalue of 4.383. Seven statements loaded on this factor, with values between 0.565 and 0.771. This factor has been renamed 'general perception after completing the preparatory year'.

Factor 2 explained 13.133% of the variance in responses, with an eigenvalue of 4.203. Eight statements loaded on this factor, with values between 0.457 and 0.799. This factor has been renamed 'perception of teachers, teaching, and assessment'.

Factor 3 explained 12.165% of the variance in responses, with an eigenvalue of 3.893. Six statements loaded on this factor, with values between 0.479 and 0.761. This factor was renamed 'university conduct'.

Factor 4 explained 9.625% of variance in responses, with an eigenvalue of 3.080. Six statements loaded on this factor, with values between 0.445 and 0.770. This factor was renamed 'administrative and regulatory matters'.

Factor 5 explained 7.722% of variance in responses, with an eigenvalue of 2.471. Three statements loaded on this factor, with values between 0.593 and 0.765. This factor was renamed 'facilities and services'.

Furthermore, the communalities of the 30 items are presented in Table 1. It revealed that the communalities ranged between 0.517 and 0.751, i.e. extracted factors explained most of the variance in the variables being analysed. Only one item (item 17) had low communalities (<0.50).

Correlations between variables (using product moment-to-moment Pearson correlation coefficient)

The Pearson correlation coefficient was calculated between the 5 factors (after factor analysis) and each other, and between each factor and the total score of the questionnaire. Appendix 2 (http://ajhpe.org.za/public/files/1273-a2.pdf) shows the results of correlation.

There is high statistical significance for all correlation values. There were moderate relationships between all the factors of the questionnaire, i.e. each factor is considered independent of the other factors and measures a different aspect of student perception. Hence, all the factors show that the tool is valid for measuring students' perceptions of the preparatory year.

Also, values for correlation between the first 4 factors and total questionnaire values were high and statistically significant (p<0.01). However, for the factor 'generally, after completing the preparatory year', it gave a moderate correlation with the total questionnaire value (0.613).

Reliability study

Reliability of the questionnaire was studied by performing Cronbach's alpha for the 5 factors and the total questionnaire. Table 2 shows the following results:

 

 

All Cronbach's alpha values for the factors and the entire questionnaire were high (between 0.744 and 0.942), which indicates the high internal consistency (reliability) of the questionnaire. Alpha levels did not increase when items were deleted.

Perception of students in health professions education faculties of the effect of the preparatory year in qualifying for studying in their faculties

Table 3 reveals that the students were highly satisfied with factor 3 (university conduct), with a mean of 3.16. However, they were less satisfied with factor 2 (perception of teachers, teaching and assessment), with a mean of 2.59.

 

 

Appendix 3 (http://ajhpe.org.za/public/files/1273-a3.pdf) shows frequencies and percentages of students' response for all factors.

Regarding factor 1 (generally, after completing the preparatory year), around half of the students agreed that after the preparatory year they had confidence in their abilities to research information (51.9%), their thinking skills improved (46.6%), their computer skills became more sophisticated (46.1%) and they could access and use learning resources (53.5%).

Regarding factor 2 (perception of teachers, teaching and assessment), nearly two-thirds (64.8%) of the students disagreed that the preparatory year provided activities that developed their various personal skills (e.g. social, physical, cultural, creative). More than half disagreed that teachers motivated them to develop their creative and innovative abilities (56.4%), 58.5% stated that the academic counsellor was helpful, and 57.7% described this interaction as adequate and fruitful. With regard to teaching, ~53.4% believed it was interactive rather than that they were spoon-fed. When measuring the ease of communication with their teachers, 40.6% of students stated that it was easy, while around one-quarter (24.5%) were neutral and one-third (34.9%) stated that communication was not easy. Means of responses were low, except for ease of communication with teachers, which was slightly higher.

Regarding factor 3 (university conduct), around half of the study sample agreed that the preparatory year made them self-confident and reinforced their discipline (49.2% and 46.9%, respectively). Less than two-thirds agreed that the preparatory year reinforced their sense of responsibility and helped them adapt to the university educational environment (60.6% and 64.3%, respectively).

Regarding the role of the preparatory year in helping students select their major subjects, more than half of the students (58.5%) did not believe that the preparatory year had an effect. Also, more than half (57.2%) did not agree that the preparatory year prepared them for their major subjects. This is evident from the low mean score for their responses (2.41).

For factor 4 (administrative and regulatory matters), 45.7% of the students agreed that the assessment tasks were appropriate, 45.7% agreed that adequate awareness sessions were planned for newcomer students, 55.8% believed that student guidebooks were informative, and 49.7% believed that teaching timetables were set out appropriately. When addressing the accessibility of the academic affairs service, 41.4% agreed that it was easily accessible. However, less than one-third (28.9%) agreed that student support services were adequate.

Finally, regarding factor 5 (facilities and services), about two-thirds (63.5%) of the students agreed on the appropriateness of the facilities, such as library, computers, data displays, smart boards. About half of them (45.6%) agreed on the appropriateness of areas designed for students to rest. More than half disagreed about the suitability of food-purchasing areas for students.

 

Discussion

This study discusses the validation of a newly developed programme evaluation instrument. The instrument focuses on evaluation of the preparatory year at KAU as a novel experience. It was imperative to conduct such a study to search for solutions for problems to effectively prepare students for their new learning experiences as university students - from school to university life.

This was a cross-sectional descriptive study. The study sample comprised male and female second- and third-year students who completed their preparatory year and started studying at the faculties of Medicine, Dentistry, Pharmacy, Applied Medical Sciences and Nursing. Samples were comprehensive.

A self-administered questionnaire to evaluate the effect of the preparatory year in preparing students for studying in health professions education faculties was used and the psychometric properties of the instrument were tested. To test the construct validity of the questionnaire, EFA was conducted using principal component analysis with varimax rotation. Data revealed that the questionnaire was categorised into 5 factors.

The current study revealed high internal consistency (reliability) of the questionnaire. Cronbach's alpha coefficient value for the total scale was 0.94. In addition, internal consistency reliability, on its own, shows evidence of construct validity of the questionnaire.

A unique addition to the existing literature was development of a new, validated, highly reliable tool for evaluating the effect of preparatory programmes for students for studying in health professions education faculties. Therefore, it was difficult to find previous studies for comparison.

The current study findings revealed that among the 5 dimensions of the questionnaire, the students were highly satisfied with 'university conduct'. They agreed that the preparatory year made them self-confident. They also agreed that the preparatory year reinforced their sense of responsibility and helped them adapt to the university educational environment.

From the abovementioned results, we can infer directly that the effect of the preparatory year on students in the different aspects of 'university conduct' is good. This is similar to the principle on which the preparatory year was originally planned, being the first contact with university life and the transitional stage between that and high school life. However, the previous finding was inconsistent with that in a study by AlKathiri,[10]who found dissimilar results in his evaluation of the preparatory year.

In the current study students were less satisfied with the 'perception of teachers, teaching and assessment', i.e. most of the students disagreed regarding the preparatory year providing activities that developed their various personal skills (e.g. social, physical, cultural, creative).

Furthermore, students disagreed that teachers motivated them to develop their creative and innovative abilities. Students agreed that interaction with the academic counsellor was inadequate. Also, they were not very satisfied with the teaching strategies.

Teaching (including teachers) and assessment are the cornerstone of the preparatory year, where most of the attention should be directed. However, it is clear from the students' responses that they negatively perceived teaching, teachers and assessment. As stated by AlKathiri,[10] it is a major challenge to attract and train qualified teachers for the success of the preparatory year. The current study agrees with the recommendation by Zeller[11] and Zlotkowski,[12] who argued that universities should strive to attract qualified teachers for the preparatory year, otherwise that year may be continuously questioned, reflecting on the impact on the quality of the outputs and thus possibly affecting reviewing its significance in the future.

 

Conclusion and recommendations

This study established the validity and reliability of the newly developed questionnaire (preparatory-year programme evaluation survey) after measuring different types of construct validity evidence through EFA and reliability analysis.

The study also concluded that from the student perspective, the preparatory year programme needs to be revisited, as students were satisfied with certain facets, such as 'university conduct', while less satisfied with other aspects, such as 'perception of teachers, teaching and assessment'.

Consequently, research is needed to explore further the reasons behind the ineffectiveness of certain aspects in the preparatory year in its current format.

Declaration. None.

Acknowledgements. We thank the students who participated in the study. We also appreciate the endless support of the administration of King Abdulaziz University.

Author contributions. MA-R: developed the idea and original research project, as well as the tool for data collection, wrote the methodology section and entered data. HA and LAS were supervisors. HA did the statistical analysis, wrote the discussion and edited the manuscript. EG was responsible for doing and interpreting the factor analysis and testing the tool for validity and reliability. AA helped to write the introduction and edited the manuscript. HA was mainly responsible for arranging for data collection and writing of the results. All authors critically reviewed and approved the final draft and were responsible for the content and similarity index of the manuscript.

Funding. None.

Conflicts of interest. None.

 

References

1. Barefoot B, Griffin B, Koch A. Enhancing Student Success and Retention throughout Undergraduate Education. A National Survey. North Carolina: John N Gardner Institute for Excellence in Undergraduate Education, 2012.         [ Links ]

2. Alexander JS, Gardner JN. Beyond retention: A comprehensive approach to the first college year. About Campus. 2009;14(2):18-26. https://doi.org/10.1002/abc.285        [ Links ]

3. Adelman C. The Toolbox Revisited: Paths to Degree Completion from High School through College. Washington, DC: US Department of Education, 2006.         [ Links ]

4. Barefoot BO. Current institutional practices in the first college year. In: Upcraft ML, Gardner JN, Barefoot BO, eds. Challenging and Supporting the First-year Student: A Handbook for Improving the First Year of College. San Francisco: Jossey-Bass, 2005.         [ Links ]

5. Gardner J. Seven Principles of Good Practice for Student Success Partnerships. North Carolina: John N Gardner Institute for Excellence in Undergraduate Education, 2013.         [ Links ]

6. Kaiser HF. The application of electronic computers to factor analysis. Educ Psychol Measure 1960;20(1):141-151. https://doi.org/10.1177%2F001316446002000116        [ Links ]

7. Cattell RB. The scree test for the number of factors. Multivariate Behav Res 1966;1(2):245-276. https://doi.org/10.1207/s15327906mbr0102_10        [ Links ]

8. Pett MA, Lackey NR, Sullivan JJ. Making Sense of Factor Analysis: The Use of Factor Analysis for Instrument Development in Health Care Research. Thousand Oaks, CA: SAGE, 2003.         [ Links ]

9. Lee N, Saunders J, Hooley G. The evolution of 'classical mythology' within marketing measure development. Eur J Marketing 2005;39(3/4):365-385. https://doi.org/10.1108/03090560510581827        [ Links ]

10. AlKathiri N. Preparatory year (first year experience). Saudi J Higher Educ 2014;11:65-70.         [ Links ]

11. Zeller W. First-year student living environments. In: Upcraft ML, Gardner JN, Barefoot BO, eds. Challenging and Supporting the First-year Student: A Handbook for Improving the First Year of College. San Francisco: Jossey-Bass, 2005.         [ Links ]

12. Zlotkowski E. Service learning and the first-year student. In: Upcraft ML, Gardner JN, Barefoot BO, eds. Challenging and Supporting the First-year Student: A Handbook for Improving the First Year of College. San Francisco: Jossey-Bass, 2005.         [ Links ]

 

 

Correspondence:
H Atwa
doctorhani2000@yahoo.com

Accepted 14 February 2020.

Creative Commons License All the contents of this journal, except where otherwise noted, is licensed under a Creative Commons Attribution License