versão On-line ISSN 1025-9848
Health SA Gesondheid (Online) vol.18 no.1 Cape Town Out. 2013
Louis Frederick Small; Louise Pretorius; Althea Walters; Maria Ackerman; Pandeni Tshifugula
Nursing, University of Namibia, Namibia
The nursing profession is characterised by the fact that a significant amount of time is spent on competency-related activities. The assessment of clinical competence is therefore an important issue in nursing education and the utilisation of objective structured clinical evaluation for that purpose was considered to be very important in this study. The objective of this research was to explore and describe the perceptions of first- and third-year student nurses with regard to the objective structured clinical evaluation assessment approach. A quantitative, cross-sectional, analytical research design was used. The instrument was a questionnaire (ordinal). This questionnaire focused on the perceptions of student nurses with regard to the objective structured clinical evaluation approach in particular, and with regard to aspects such as reality, execution, time allocation and student and assessor variables. The findings indicated that, as an assessment tool, the objective structured clinical evaluation approach was perceived as not being totally realistic, especially by the more senior nursing students (third-year) as compared with the first-year nursing students. Varying degrees of stress were experienced by the nursing students in their first and third years but, in general, the overall perception appeared to be that the approach was well organised and that the respective students would be able to perform equally well in the clinical field. It may be concluded that the majority of students appreciate the format of the objective structured clinical evaluation approach. However, the study further highlighted the fact that more extensive training of students on time management and the relief of emotional stress is necessary during the implementation of this approach.
Die verpleeg professie word gekenmerk dat 'n betekenisvolle typerk aan vaardigheids verwante aspekte bestee word. Dus is die evaluering van kliniese vaardighede 'n belangrike komponent in verpleegonderwys en die gebruik van objektiewe gestruktureerde kliniese evaluering was sodoende as noodsaaklik beskou in hierdie studie. Die doelwit van hierdie studie was om die persepsies van eerste en derde jaar verpleegstudente te ondersoek en te beskryf ten opsigte van die objektiewe gestruktureerde kliniese evaluerings benadering. 'n Kwantitatiewe analitiese deursnee studie was gebruik. Die instrument was 'n vraelys (ordinaal). Die focus van die vraelys was op die persepsies van die student verpleegkundiges met betrekking tot die objektiewe kliniese evaluering in besonder, asook op aspekte soos realiteit, uitvoering, tyd toewysing en student en evalueerder veranderlikes. Uit die bevindings blyk dit dat die objektief gestruktureerde kliniese evaluering benadering beskou word as nie volkome realisties nie, veral deur die senior (derde jaar) verpleegkundestudente in verhouding tot die eerstejaars. Wisselende vlakke van spanning is in die eerste en derde studiejare deur die studente ondervind, maar in die algemeen blyk die siening te wees dat die objektief gestruktureerde kliniese evaluering benadering goed georganiseer was en dat die studente in staat sou wees om ewe goed in die kliniese veld te vaar. Die meerderheid van die studente het waardering betyg vir die objektief gestruktureerde kliniese evaluering benadering. Nietemin is dit in die studie bevind dat die studente meer intensief voorberei moet word ten opsigte van tyd spandering, asook hoe om stress te verwerk tydens die evalauering.
The nursing profession is characterised by the fact that a significant amount of time is spent on competency-related activities. These competency-related activities are fundamental to the caring role of nurses and, as such, are central to the nursing curriculum. The assessment of clinical competence assessment is therefore an important issue in nursing education and the utilisation of objective structured clinical evaluation (OSCE) for that purpose is considered very important (Alsenany & Al Saif 2012:597). Knowledge, attitude and skills form part of this competency component. In addition, assessment is an important part of this process toward competency and it contributes to the ongoing improvement of students' capabilities (Gamboa-Salcedo et al. 2011:169). One of the ways in which to assess the mastery of skills in an undergraduate programme is by means of an objective examination as is formulated in the OSCE (El-Nemer & Kandeel 2009:2465).
In Namibia, the skills component in nursing-related subjects contributes to at least 50% of the content of these subjects, with this high proportion also requiring that any assessment should reflect this significant emphasis on the skills (psychomotor) component. The most appropriate form of assessment comprises an individual assessment of each student within the clinical setting. Nevertheless, there is no ideal assessment method since all methods have their own strengths and weaknesses (Gamboa-Salcedo et al. 2011:169). In addition, individual assessments are not always realistic as a result of the (usually) large numbers of students. These large numbers of students, in turn, may create both logistical and administrative difficulties. As regards assessment within the clinical setting, it is possible that, apart from the logistical and administrative difficulties mentioned above, ethical dilemmas may also arise (Bruce, Klopper & Mellish 2011:273). According to Alsenany and Al Saif (2012:598) no single method is appropriate for assessing clinical competence amongst healthworkers and thus a multi-method strategy is proposed.
Accordingly, in view of the vast number of skills that need to be assessed in each nursing-related subject, the OSCE offers an alternative approach. This approach makes use of simulations of the actual skills that need to be assessed, although it may happen that only certain components of the total skills are included in the assessment. One of the advantages of such an approach is the fact that a number of skills may be assessed at a single session without having to take into account the possible availability of a procedure for assessing a particular skill, or the discomfort or even possible harm to a patient.
As a result of the large number of students - about hundred per academic year on each of two campuses - that need to be assessed, the School of Nursing and Public Health at the University of Namibia often chooses to adopt the OSCE approach. During these assessments, specific skills are selected for inclusion in this approach.
However, there was a problem in that it was not known how realistic or effective these OSCE assessments were, especially as the clinical competence of the students was graded on the basis of the results of the assessments. It was also not clear what students perceive regarding the OSCE. This supports Fidment's (2012:1) comment that as educators it is important that we explore this assessment from the perspective of the student.
In this study, the student nurses were the main stakeholders. Data on students' perception about OSCE examination yield important information that may be helpful for the driving force supporting a change with regard to assessment of students in nursing practice (Alsenany & Al Saif 2012:601).
The purpose of this research was to explore and describe the perceptions of first- and third-year student nurses with regard to the OSCE assessment approach.
Contribution to the field
The benefit of this study will be on a practical level as the results will be used during curriculum planning meetings in order to motivate the continuation, discontinuation or adaptation of the current practice being followed at the School of Nursing and Public Health with regard to the utilisation of OSCEs.
Research method and design
A quantitative, cross-sectional, analytical research design was used. This design was deemed appropriate because quantification of information was needed and the data collection was performed at all together, therefore a cross-sectional approach seemed appropriate. In addition, this research study was also retrospective in the sense that the students were required to comment on OSCE assessments that had already been completed and to give their perceptions regarding these assessments. The analytical part of the study strived to determine for any association between variables.
Population and sampling
The population consisted of all those student nurses who had been assessed by means of an OSCE during 2010. During October of 2010, first- and third-year nursing students at the University of Namibia had been assessed in this manner. No sampling was done as the total population (403) was included in the study. As an accurate recall of perceptions was necessary, it was decided not to include second- and fourth-year nursing students, as it was felt that their perceptions may not be as accurate as those of the first- and third-year nursing students, as a result of the passage of time. Based on these considerations, the populations to be assessed comprised 403 nursing students.
The instrument (questionnaire) was distributed to all of these students, so no sampling was carried out per se. A total of 204 completed questionnaires was received, indicating a 51% response rate.
Data collection methods
A questionnaire (self-report) was devised. This questionnaire focused on the perceptions of student nurses with regard to OSCEs, in particular, with regard to aspects such as reality, execution, time allocation and student and assessor variables. An ordinal type of questionnaire was used for most of the items, but not all. In terms of the items on the questionnaire a scale ranging between 1 and 5 was used, with the ratings grouped together for the purposes of data analysis. This meant that three 'groups' were possible, with ratings of 1 and 2 being regarded as the opposite of ratings 4 and 5, whilst ratings of 3 were regarded as being neutral. The few remaining items were nominal in nature.
Validity and reliability
The testing of the instrument involved assessing the aspects of reliability and validity.
The validity of the OSCE as an examination process involved two types of validity, namely face validity and content validity. As regards face validity, the checklist should be capable of measuring what it is intended to measure, whilst the measurement should also make sense and be reasonable (Bell 2006:117). The instrument was submitted to three lecturers who were involved with OSCE examinations on a yearly basis. They all agreed that the instrument focused on applicable areas and considerations and that it was unambiguous. As regards content validity, confirmation was obtained from relevant literature as well as a consensus agreement from the abovementioned three lecturers.
The researchers in this study adhered to the description of reliability as referring to the accuracy or precision of an instrument, the degree of consistency of agreement between two independent evaluator scores, and the extent to which independent administrations of the same instrument would yield the same results under comparable conditions (Cohen, Marion & Morrison 2007:164). The instrument was itemised and two examiners were employed so as to ensure inter-rater reliability.
Data collection and analysis
The data were collected during October 2010. This data collection process took place two days after the completion of the OSCEs, with the assumption being that the students would still be able to recall details accurately regarding their perceptions. Descriptive statistics were used to depict mean values, and these were enhanced by means of pie-charts and bar-graphs. In addition, chi-square calculations were utilised in order to identify associations between selected variables, and to extrapolate the results to the broader population.
Context of the study
The study was conducted at two main campuses of the University of Namibia. These are the only campuses in Namibia where student nurses are being trained at university level.
Ethical clearance for the study was obtained from the Post Graduate Research Committee of the School of Nursing and Public Health of the University of Namibia. In addition, informed consent was obtained from the students and they were assured of both anonymity and confidentiality. The students also participated voluntarily after being briefed in full. No psychological harm was foreseen as it was not possible to link any response to a specific individual (lecturer or student).
Discussion of results
The first item dealt with the academic distribution of the student nurses. In total, 204 nursing students completed the questionnaire. Of this total, 67% (n = 137) were first-year nursing students and 33% (n = 67) were third-year nursing students (see Figure 1).
The second item required the student nurses to indicate their perceptions as regards the preparation they had received from their lecturers prior to the assessment. A large number of the students, namely 65% (n = 132), indicated that they felt that they had received sufficient information from their lecturers in preparation for their OSCE examinations, whilst 20% (n = 41) were of opinion that they had not received enough information. Fifteen percent were undecided (neutral) (see Figure 2).
In whatever way the OSCE is either planned or used, it is important that the students be clearly briefed and informed about the aims and objectives of the OSCE sessions, as well as whether the sessions will cover an area of content that they already know (Alinier 2003:12).
A statistically-significant association was found between the year of study of the student nurses and their perceptions about the preparation and information provided by their lecturers.
This study found that first-year nursing students were more positive than third-year nursing students about whether they had received thorough preparation and enough information from their lecturers prior to their assessments (χ2 = 25.581, p = 0.000).
In addition, it was deemed necessary that the student nurses indicate the level of their own preparation. The responses (Figure 3) indicated that 82% (n = 167) of the students were of the opinion that they had prepared well for their OSCEs. No statistically-significant differences were found between the first- and third-year nursing student nurses. El-Nemer and Kandeel (2009:2468) maintain that students should receive orientation with regard to the nature of the OSCE and the process of the examination approximately one week prior to the examination. This was supported by Fidment (2012:5) who indicated that even though a thorough preparation is done, educators should still be aware that students might not feel that they were prepared appropriately.
Item 4 focused on the perceived level of stress experienced by the student nurses. These results are depicted in Figure 4.
It emerged from the results that 36% (n = 73) of the respondents had experienced stress and/or had felt extremely anxious. Forty-two per cent (n = 86) did not acknowledge that they had experienced any stress, whilst 22% (n = 45) responded in a neutral way to this question. A similar study conducted in Egypt discovered that 52.7% of the respondents had found the OSCE examination to be more stressful than a written examination, although the respondents had been prepared adequately for the examination (Mahmoud & Mostafa 2011:736). Similar findings also emerged from a study conducted by El-Nemer and Kandeel (2009:2470), who reported that the OSCE was particularly stressful for first-year students. In a qualitative study conducted in the U.K., Fidment (2012:4) discovered that anxiety forms a key theme in a student's experience. Students were anxious either before or after the assessment. Their anxiety manifested in different ways, with one student comparing the feeling to that of a driving test (Fidment 2012:4).
In item five, the students were required to rate the 'realism' of the OSCE examination. As is evident from Figure 5, 25% (n = 51) of the respondents regarded the OSCE examination as being 'not very realistic', whilst 50% (n = 102) regarded it as being realistic. The responses of the remaining 25% (n = 51) were neutral. It was found that students may experience an 'uncomfortable' feeling in a simulated environment. Some of the interviewees in Fidment's study described feeling silly or stupid, as if there was a reality barrier through which they could not connect or associate with the manikin (Fidment 2012:6).
This aspect of realism is one of the specific logistical issues identified in the study, especially in terms of equipment and scenarios. It was, therefore, recommended that relevant clinical staff be involved in the OSCE in order to enhance both the realism and the relevance of the scenarios (Alison Evans Consultancy 2008:3).
Item 6 focused on the atmosphere encountered at the various OSCE assessment points, as shown in Figure 6.
It emerged that 66% (n = 135) of the student nurses had perceived the atmosphere to be relaxed and calm, whilst 19% (n =39) had experienced it as busy and tense. However, 15% (n = 30) expressed neutral perceptions about the atmosphere at the OSCE assessment points.
Item 7 requested the student nurses to indicate the extent and quality of the information they had received at the OSCE stations. The results indicate that 70% (n = 143) of the student nurses regarded the extent and quality of information received as being both sufficient and clear (see Figure 7). It is shown in the literature that the unknown 'OSCE' element certainly plays a part in how students perceive the procedure, and the information provided beforehand is regarded as being of utmost importance (Fidment 2012:5).
However, 19% (n = 39) of the respondents were of the opinion that insufficient information had been provided and/or that the information provided had been unclear, whilst 11% (n = 22) gave neutral responses.
The student nurses were also consulted about the time allocated for reading the instructions. As may be seen in Figure 8, 71% (n = 145) believed that enough time had been allocated, although 20% (n = 41) did not agree, and 9% (n = 18) were neutral.
Despite the fact that an OSCE is conducted in a formative way, students are allowed to ask questions of the examiner present if they are not confident about the task they are required to perform (Alinier 2003:10).
It was also deemed necessary to determine whether enough time had been allowed to complete the different procedures. The results of this item (Figure 9) indicate that 73% (n = 149) of the student nurses felt that that they had been given enough time to complete the procedures, 16% (n = 33) disagreed and 11 % (n = 22) were neutral.
One of the advantages of the OSCE is the flexibility it allows in the setting up of the different stations. However, it is essential that the stations be focused so that it is possible to complete specific tasks within a planned time frame (Alinier 2003:12).
The attitudes (appearances at the OSCE stations) of lecturers may have a negative influence on performances during an OSCE assessment. In this study, 88% of the student nurses (n = 180) indicated that they had found the lecturers to be friendly and relaxed, only 2% (n = 4) disagreed and 10% (n = 20) were neutral (see Figure 10).
A statistically-significant association was found between the year of study of the student nurses and their ratings on the friendliness and professional appearance of lecturers during the completion of an OSCE examination.
In this study, third-year nursing students tended to rate the lecturers as being more friendly and professional than did the first-year nursing students (χ2 = 21.215; p = 0.000).
Apart from the topic of professional behaviour on the part of staff being raised as part of this research study, it is recommended that students should also be given the opportunity to respond in writing about the conduct of the assessors (Meyer & Van Niekerk 2008:197). This includes the attitudes of lecturers at the OSCE stations.
An assumption inherent in the OSCE assessments is that successful performance of the skills assessed automatically implies that students will duplicate the performance of these skills as successfully when dealing with real patients. It emerged from the responses of the students that they appeared to be confident that they were, indeed, capable of performing at the same level of proficiency in the clinical situation. This positive perception is represented in Figure 11.
The item regarding the feedback provided on completion of the assessment revealed that 78% (n = 159) of the student nurses were of the opinion that feedback had not been provided (see Figure 12).
A statistically-significant association was found between the year of study and whether or not feedback had been provided. In this study, 31% (n = 43) of the first-year nursing students indicated that they had received feedback in comparison with only 3% (n = 2) of the third-year nursing students (χ2 = 20.547; p = 0.000).
The item in respect of the reactions of the students with regard to their OSCE results revealed that 16% (n = 30) had felt that their results had been better than expected whilst a mere 1% (n = 2) of the students had expected better results (Figure 13).
It is essential that educators guide and support students by providing them with feedback after a formative and summative assessment (Meyer & Van Niekerk 2008:198). In a previous study conducted by Wong et al. (1997:5), it emerged that students regarded feedback as being a valuable learning experience which was considered to be beneficial to their development in both the clinical field and the cognitive domain. Previous research by Bradley and Postletwaite (2003:7) indicated that it is essential to give feedback regarding evaluation of clinical skills as it plays an important role in ensuring the quality of a learning programme. An OSCE is therefore considered to be a powerful tool in evaluation and an effective facilitator with regard to learning in health education (Alsenany & Al Saif 2012:599).
At the time that this specific study was conducted, not all the results had been received by the students. Accordingly, this item was completed by only 185 of the students. As may be deduced from Figure 13, it is not possible to arrive at any definite conclusions from the information available. This aspect is further referred to under 'limitations of the study'.
From the item regarding the preferred setting for the carrying out of a clinical examination (Figure 14), it emerged that 44% (n = 90) of the students prefer the OSCE examination whilst 41 % (n = 84) of the students indicated that they would have preferred real patients within a clinical setting. The findings from some studies conducted have shown that the students were concerned that the evaluation outcomes might not necessarily reflect their competence (Wong et al. 1997:1).
The ideal location for a practical assessment is the clinical setting, as this would make the assessment less artificial and it would be possible to observe the students working in a real- life situation (Bruce et al. 2011:273).
A statistically-significant association was found between the year of study and the students' preferences as regards the format of the practical examinations. In this study, 51% (n = 34) of the third-year nursing students indicated they would prefer to be evaluated working with a real patient in the clinical setting as compared with only 35% (n = 49) of the first-year nursing students (see Figure 14).
These differences between the preferences of the first- and third-year nursing students may be attributed to the fact that the third-year nursing students had had more than 1500 hours of clinical exposure in the clinical setting whilst the first-year students had been exposed to only 300 hours in a clinical setting (School of Nursing and Public Health 2011).
An additional open-ended question dealt with suggestions put forward by the students. The following six patterns merged from their suggestions:
- Better instructions.
- More time.
- More space on trolleys.
- Separate procedure rooms.
- Friendlier lecturers.
- Sufficient equipment.
Despite the fact that these patterns emerged, three of them, namely the instructions, time and friendliness of lecturers, had already been included as items that formed part of the structured section of the questionnaire. The remaining three patterns were addressed as part of the recommendations.
Limitations of the study
One limitation of this study was the high percentage of neutral responses by the students. The responses of second-and fourth-year nursing students were not obtained in this study. More detailed data could be obtained if they were to be included in a follow-up study. Another limitation was the fact that the responses of the students could not be compared with their actual OSCE results, as not all results had been provided to them during the time of data collection.
Based on the aforementioned results, the following recommendations may be considered:
• OSCE is an effective assessment tool and should continue to be used in the preregistration nursing programme.
• The management processes involved during an OSCE should be enhanced. This would include aspects such as examiner preparation, student preparation and technical preparation. Each of these three points will now be discussed separately.
The curriculum review committee or a similar committee should compile an OSCE guideline. This guideline should include information on conducting the assessment itself and, in view of the fact that some students may misinterpret facial expressions and gestures, also some hints about non-verbal communication. It is also recommended that the lecturers acknowledge each student by name as well as make a short, directed sentence.
Pre-assessment leaflets should be distributed to the students before the assessment. These leaflets should contain relevant information on the time allocated for all possible procedures, as such information would guide the students in their preparation for the assessments.
Adequate time for practising should also be made available in line with the remarks made by students in this regard. The comments about time may have indicated that, in the case of certain students, total consolidation of a specific skill had not yet taken place, and that more practising time needs be made available in order to facilitate such consolidation.
With regard to the remarks about space on the trolley, this may also indicate that some of the students were not yet comfortable or at ease with the 'space' within which certain activities were to be completed. It is thus recommended that, during demonstration and practice sessions, the exact same format and approach be followed as that used during the assessment.
The fact that a room is shared during the assessment of two or more practical procedures may lead to students feeling that there has been a breach of confidentiality. This principle of sharing a room should be discouraged as it might lead to examination disturbances. It is, therefore, recommended that one assessment per room be allocated or, where a small hall is used, that the different assessment stations be separated by placing dividing screens between them.
Despite the fact that the OSCE is an assessment method that has been used for a number of years, regular 'assessment' of an 'assessment method' is still required as part of an ongoing curriculum review process. In this study, the students were the main stakeholders in the assessment of the assessment method. Their perceptions and experiences indicate that the OSCE continues to be both a valuable and an accepted method of assessment. Nevertheless, there is a need for the careful preparation and organisation of the OSCE.
It may be concluded that the majority of students appreciate the format of the OSCE. However, the study further highlighted the fact that more extensive training of students on time management and the relief of emotional stress is necessary during the implementation of the OSCE (Mahmoud & Mostafa 2011:737).
The authors acknowledge the contribution and assistance provided by the different peer reviewers as part of the publication process.
The researchers declare that they have no financial or personal relationship(s) which may have influenced them inappropriately in the writing of this paper.
All the researchers made an equally-valuable contribution to the writing of this article.
Alinier, G., 2003, 'Nursing students' and lecturers' perspectives of OSCE, incorporating simulation', Department of Nursing and Paramedic Sciences, Herefordshire [obtained directly from author]. [ Links ]
Alison Evans Consultancy, 2008, 'Competency standards in professional nursing: a summary of literature published since 2000', prepared on behalf of EdCan, viewed 12 September 2011, from http://www.edcan.org/pdf/EdCancompetenciesliterature.reviewFinal.pdf [ Links ]
Alsenany, S. & Al Saif, A., 2012, 'Developing skills in managing Objective Structured Clinical Examinations (OSCE)', Life Science Journal 9(3), 597-602. [ Links ]
Bell, J., 2006, Doing your research project, a guide for first time researchers in education health and social science, 4th edn., Open University Press, United Kingdom. [ Links ]
Bruce, J.C., Klopper, H.C. & Mellish, J.M., 2011, Teaching and learning the practice of nursing, 5th edn., Heinemann, Cape Town. [ Links ]
Cohen, L., Marion, L. & Morrison, K., 2007, Research methods in education, 6th edn., Routledge, London. [ Links ]
El-Nemer, A. & Kandeel, N., 2009, 'Using OSCE as an assessment tool for clinical skills: Nursing students' feedback', Australian Journal of Basic and Applied Sciences 3(3), 2465-2472. [ Links ]
Fidment, S., 2012, 'The Objective Structured Clinical Exam (OSCE): A qualitative study exploring the healthcare student's experience', Student Engagement and Experience Journal 1(1), 37-47. http://dx.doi.org/10.7190/seej.v1i1.37 [ Links ]
Gamboa-Salcedo, T., Martinez-Viniegra, N., Pena-Alonso, Y.R., Pacheco-Rios, A., Garcia-Duran, R. & Sanchez-Medina, J., 2011, 'Objective Structured Clinical Examination as an instrument for evaluation of clinical competence in pediatrics. A pilot study', Boletín Médico del Hospital Infantil de México 68(3), 169-176, viewed 18 July 2011, from http://www.nietoeditores.com.mx [ Links ]
Meyer, S. & Van Niekerk, S., 2008, Nurse educator in practice, Juta, Cape Town. [ Links ]
Mahmoud, G.A. & Mostafa, M.F., 2011, 'The Egyptian nursing student's perceptive view about an Objective Structured Clinical Examination (OSCE)', Journal of American Science 7(4), 730-738, viewed 10 July 2011, from http://www.jofamericanscience.org/journals/am-sci/am0704/102_5258am0704_730_738.pdf [ Links ]
School Of Nursing and Public Health, 2011, 'Nursing Curricula', Unpublished, University of Namibia. [ Links ]
Wong, T.K.S., Chow, F.L.W., Lau, A.K.L., Lau, T.P.L., Wong, M.W.L. & Wong, S.K.F., 1997, 'Fostering student's clinical competence through Objective Structured Clinical Assessment (OSCA)' viewed 18 July 2011, from http://celt.ust.hk/files/public/bhkp0311-15.pdf [ Links ]
PO Box 86002, Eros
Received: 12 Oct. 2011
Accepted: 21 Feb. 2013
Published: 18 Apr. 2013