CLINICAL REVIEW

 

Do precursor tests influence the performance of Oral Hygiene students in subsequent semester tests?

 

 

V Bookhan^I; V Warren^II; Hs schoeman^III

^IBDS, M Dent (Prosthodontics) . Department of Odontology, School Of Dentistry, University of Pretoria, South Africa
^IIBChD, PGDipDent (Endodontics), MSc (Odontology). Department of Odontology, School of Dentistry, University of Pretoria, South Africa
^IIIBSc, MSc, DSc). Clinstat CC, South Africa

Correspondence

 

 

---

ABSTRACT

INTRODUCTION: Precursor tests may be useful educational tools In dentistry to enhance learning among Oral Hygiene students, but their application and possible effect on the subsequent performance of these students have not been studied
AIM: To determine whether precursor tests, used as dental educational and formative assessment tools, influence the performance of undergraduate Oral Hygiene students in ensuing formal semester tests in the discipline of Basic Restorative Dentistry
METHODS: This descriptive cross sectional study involved seventeen consenting Oral Hygiene students. An electronic Odontology Theory Test (OTT) and an electronic Objective Structured Practical Test (OSPT), were prepared and introduced as precursor tests prior to scheduled semester tests in the Division of Restorative Dentistry (Department of Odontology). Eleven (65%) of the seventeen students completed the precursor OTT, as well as the precursor OSPT, fourteen days prior to their scheduled semester OTT and semester OSPT and sixty two days prior to their final examinations. The results of the precursor and the semester tests were entered into a Microsoft Excel^® database for comparative analysis, using a Student's t-test
RESULTS: For both OTT and OSPT tests there were significant differences between the means of the scores of the students for precursor and semester tests (OTT: p = 0.0009; OSPT: p = 0.0180)
DISCUSSION: The students performed significantly better in their precursor OTT, whilst their performance in the OSPT was significantly better in the semester test. : In the context of this investigation, the precursor OTT did not enhance the performance of the students in their semester OTT, whilst the precursor OSPT was associated with an enhanced performance of the students in their semester OSPT

---

 

 

ACRONYMS

CBT: Computer Based Testing
OSPT: Objective Structured Practical Test
OTT: Odontology Theory Test

 

INTRODUCTION

There is evidence in the literature supporting a theory that precursor tests, (pre-tests written by students designed to improve their post-test performance), enhance student learning in both educational and industrial contexts.^1-9 These tests have teaching functions with the potential to improve performance by motivating learning amongst students.^1,9 They are commonly used for behavioural research.³ Previous studies involving medical and nursing students, have used precursor tests for the calibration of supervisors as well as teaching tools, in the assessment of the performance of students.^4,5,6 Precursor tests have also been shown to turn remediation into an active learning experience.⁸

Formative tests are used to obtain information that can aid teachers in introducing methods which will improve student learning. Formative tests and assessments are used to improve the progress of learning amongst students, compared with summative tests and assessments used to evaluate the progress of learning amongst students.¹⁰ The formative nature of precursor testing provides students with feedback allowing them to recognise their academic strengths and weaknesses during remediation. The remediation process allows teachers to provide students with guidance to overcome their weaknesses and provide them with opportunities to achieve academic progress.

Precursor tests, (formative), may be useful educational tools in dentistry to enhance learning amongst students and improve their summative (semester) test performances,^1,5,8 but this has not been investigated for Oral Hygiene students.

 

AIM

To determine the effect of precursor tests, used as dental educational and formative assessment tools, on the semester test assessments of undergraduate Oral Hygiene students in the Division of Basic Restorative Dentistry in the Department of Odontology.

 

MATERIALS AND METHODS

Sample

This descriptive cross-sectional study involved seventeen consenting second year Oral Hygiene students, sixteen female and one male, ranging in age from 18 to 24 years. The sample was divided into two groups. Group A consisted of 11 students who would participate in the precursor testing process and Group B consisted of six students who preferred not to, but would allow their summative results to be analysed for the study. The relevant consent forms were signed by each Oral Hygiene student prior to the commencement of the study.

Pre-test-post-test design

An electronic Objective Structure Practical Test (OSPT) and an electronic Odontology Theory Test (OTT) were designed and introduced as suitably appropriate precursor tests,³ prior to scheduled final semester tests in the Division of Restorative Dentistry. This study used a non-randomised control group in a pre-test-post-test design, to enable the measurement of the changes (if any) that the precursor tests may effect. In the development and designing of the questions for the precursor tests, reference was made to the assessment criteria appearing in the Students' Study Guide. The precursor test was completed by second year Oral Hygiene students at a computer based testing (CBT) laboratory.

Study guide

The guide was developed according to the recommended university guidelines and approved by:

i) The Department of Education;
ii) The Education Innovation Committee at the School of Dentistry and
iii) The Oral Hygiene Task Group Committee at the School of Dentistr y.

Educational materials

The educational study materials comprised of:

a) Lectures, notes and electronic presentation links (Microsoft® Power Point);
b) Seminars;
c) Clinical and practical demonstrations, instructions and electronic links and
d) Prescribed textbook (specific references).

Test questions

Test and examination questions, for both the precursor- and the semester tests, were designed by a specialist, using guidelines prescribed by the Department of Education and with reference to Miller's Pyramid of Professional Competence,^11-15 using three levels of the hierarchy. These three levels are classified as;

a) Knows level of the pyramid (testing facts);
b) Knows how level of the pyramid (testing theoretical knowledge of clinical applications using pictures of clinical cases) and
c) Shows how level of the pyramid (OSPE to test clinical applications).

All questions beginning with a verb, presenting no ambiguity, were of a similar level of difficulty and cognition and were aligned with the student exit level outcomes listed in the notice released by the South African Health Department, stated as Regulations Defining the Scope of the Profession of Oral Hygiene (Government Notice: No R.800 17 October 2013). The final questions were edited and scrutinised by an education specialist with a PhD in education, as well as by the chairpersons of the Assessment Committee, Quality Assurance Committee and the Education Innovation Committee. Specific memoranda were prepared for each question, accommodating a variety of possible answers with a mark allocation awarding one point per fact, unless otherwise stated.

Computer Based Testing (CBT)

CBT is well documented in literature and is favoured by undergraduate students.^16-19 The questions were set up on personal computers in a CBT laboratory designed specifically for the purpose of evaluation of students. All answers to questions were entered by the students into examination books provided by the university.

Eleven (65%) students completed the precursor OSPT, as well as the precursor OTT fourteen days prior to their scheduled semester OSPT and semester OTT and sixty two days prior to their final examination. Answers to the precursor tests were then provided to each student in the form of a memorandum and feedback was achieved by a verbal group discussion between students and their specialist teacher. Seventeen (100%) students completed the semester OTT and OSPT. The precursor and semester tests were assessed by the same specialist. The results of the precursor and semester tests were entered into an electronic Microsoft Excel^® database and analysed using a Student's t-test.

 

RESULTS

The average precursor test scores achieved by the students in Group A were100% for the precursor OTT, 78% for the precursor OSPT and 85% for precursor year mark. The average semester test scores achieved by the students in Group A were 94% for the semester OTT, 88% for the semester OSPT and 89% for semester year mark. A comparison of the results for Group A is illustrated graphically in Figure 1.

 

 

a) The differences between the means of the precursor OTT scores and the means of the semester OTT scores were significant (p = 0.0009).
b) The differences between the means of the scores of the precursor OSPT and the means of the semester OSPT scores were significant (p = 0. 0180).
c) The differences between the means of the precursor year-mark and the means of the semester year mark were not statistically significant (p = 0. 8803).

The average semester test scores achieved by the students in Group B were 89 % for the semester OTT, 80% for the semester OSPT and 85% for semester year mark. The results for Group A vs Group B are illustrated graphically in Figure 2.

 

 

a) There were statistically significant differences between the means when the semester test scores of Group A were compared against those of Group B (p = 0.0300).
b) There were statistically significant differences between the means of the semester OSPT scores of Group A compared against those of Group B (p = 0.0175).
c) When the differences between the means of the semester year marks for Group A were compared with those of Group B, no significant differences could be shown (p=0.211).

 

DISCUSSION

It is well known that innovative teaching methods and assessment tools that enhance student learning and improve the quality of student assessments, are beneficial to dental education.^20-27 Teachers at academic institutions need to make use of methods which effectively compliment modern educational practices,^11,28 by evaluating the teaching and materials of the teacher and the learning strategies of students. Precursor testing is therefore ideal for academic institutions that follow an outcomes based curriculum, for it allows self and peer evaluation. It is essential in modern dental education to use relevant educational tools that enhance teaching and learning, encourage discussion and effectively enhance dialogue.^29-35 The formative nature of precursor testing facilitated and encouraged these objectives.

A comparative analysis of the results achieved by the eleven students in Group A indicated the following:

a) The students performed significantly better in their precursor OTT compared to their semester OTT;
b) The students performed significantly better in their semester OSPT compared to their precursor OSPT;
c) The average precursor test year-mark did not differ significantly from the average semester test year mark;
d) The students in Group A performed significantly better in the semester test and OSPT compared with the students in Group B. and
e) The average semester test year mark of the students in Group A and did not differ significantly from the average semester test year mark of the students in Group B.

The results of this study indicate that taking the precursor OSPT was beneficial to the students, whilst those sitting the precursor OTT did not appear to have gained any advantage (Figure 1). Overall, however, the Group A students who took the precursor tests performed better than those who did not (Group B), although the differences were not statistically significant (Figure 2). A possible explanation for these similar performances may be attributed to the remediation process which followed the precursor tests and involved the entire Oral Hygiene class.

This process allowed student feedback in the form of student discussion groups and stimulated student dialogue amongst the group, as well as with their teacher. Students were taught during the remediation process to familiarise themselves with the precursor test questions, to recognise their design and structure and to attempt to predict the type of questions which would be set in the semester tests, in which they were reasonably successful. Application of the principles of precursor testing has the potential to improve the quality of learning amongst students and the quality of teaching amongst teachers, factors essential for the production of capable and competent students.^36-38 Students in both Group A and Group B were in favour of precursor testing, as well as the remediation following the precursor and semester tests. Students in Group A considered the precursor tests a valuable practice tool to test their short- term and long- term cognitive memories. The students in Group B shared similar opinions but reserved their options, indicating that, nevertheless, the precursor tests required too much effort and were in fact a waste of valuable time. Further studies are required to determine the significance of the feedback received from the students related to their examination. The benefit of these precursor tests in dental education is that learning for students is turned into an active experience which enhances thinking and in turn, helps them prepare for questions in their tests and examinations.^8-10

 

CONCLUSION

In this study, precursor testing had a positive impact on the performance of the Oral Hygiene students and was an effective educational tool that enhanced student learning and encouraged student dialogue. Student feedback favoured the implementation of precursor testing, combined with remediation and also identified a need to determine the effect of precursor testing on a student's long- term and shortterm cognition.

Further studies and the compilation of relevant evidence are required to test the effect of precursor tests as a dental educational and formative assessment tool on the semester test assessments of undergraduate Oral Hygiene students in the other Divisions of Dentistry.

 

Acknowledgements

The authors would like to sincerely thank Prof. HS Schoeman for analysing the results and all the students in the Oral Hygiene Class for their voluntary participation in this study and their passionate enthusiasm for learning. results of this study.

Declaration: No conflict of interest.

 

References

1. Hartley J. The effect of pre-testing on post-test performance Instr Sci 1973; 2:193-214.         [ Links ]

2. Brogan DR, Kutner MH. Comparative analyses of pre-test-post-test research designs. Am Stat 1980; 34(4): 229-32.         [ Links ]

3. Dimitrov DM, Rumrill PD. Pre-test-post-test designs and measurement of change. IOS Press 2003; 159-65.         [ Links ]

4. Hinck Sm, Webb P, Sims-Giddens S, Helton C, Hope KL, Utley R, Savinske D, Fahey EM, Yarbrough S. Student Learning with concept mapping of care plans in community-based education. J Prof Nurs 2006; 22(1): 23-9.         [ Links ]

5. Jacks ME, Blue C and Murphy D. Short and long term effects of training on dental hygiene faculty members' capacity to write SOAP notes. J Dent Educ 2008; 72 (6): 719-724.         [ Links ]

6. Shen C-Y and Liu H-C. Metacognitive skills development: A web-based approach in higher education. Turkish Online J Educ Tech 2011; 10 (2): 140-50.         [ Links ]

7. Kaur M. School children knowledge regarding dental hygiene. J Nurs Health Sci 2013; 1 (3): 5-8.         [ Links ]

8. Poorman SG and Mastorovich ML. Using metacognitive strategies to help students learn in pre-test and post-test review. Nurse Educ 2008 33(4): 176-80.         [ Links ]

9. Shivaswamy KN, Shyamprasad AL, Sumathy TK, Ranganathan C, Kumar SP. Knowledge of acne among medical students: Pre-teset and post-test assessment. ISRN Derm 2014; 1: 1-3        [ Links ]

10. Hudson JN, Bristow DR. Formative assessment can be fun as well as educational. Adv Physiol Educ 2006; 30: 33-37.         [ Links ]

11. McMillan W. Making the most of teaching at the chairside. Eur j Dent Educ 2011; 15: 63-8.         [ Links ]

12. Pickworth GE, Snyman WD. Changing assessment practice through in situ faculty development. Eur J Dent Educ 2011; 15: 1-5        [ Links ]

13. Munck C. Recommended outcome (competencies) of undergraduate training. S Af Dent J 2005; 60: 28-9, 32.         [ Links ]

14. Andrade H, Ying D. Student responses to criteria-referenced self-assessment. Assessment and Evaluation in Higher Education 2007; 32: 159-181.         [ Links ]

15. Kramer GA, Albino JEN, Andrieu SC et al. Student assessment toolbox. J Dent Educ 2009; 73: 12-35.         [ Links ]

16. Maggio MP, Hariton GK, Gluch J. The use of independent, interactive media for education in dental morphology. J Dent Educ 2012; 11: 1497-511.         [ Links ]

17. El T, Saleh SM. Attitudes of Dental students towards using computers in education-a mixed design study. East Mediterr Health        [ Links ]

18. Rosenburg H, Grad HA, Matear DW. The effectiveness of computer-aided self instructional programs in dental education: A systematic review of the literature. J Dent Educ 2003; 67: 524-32.         [ Links ]

19. Fletcher-Finn CM, Gravatt B. The efficacy of computer assisted nstruction (CAI): A meta analysis. J Educ Comp Res 12: 219-41        [ Links ]

20. Tennant M, Scriva J. Clinical assessment in dental education: a new method. Aust Dent J 2000; 45: 125-30.         [ Links ]

21 Bookhan V, Becker LH, Oosthuizen MR Criteria referenced student self assessment in Restorative Dentistry. S Af Dent J 2005; 60: 161-6.         [ Links ]

22. Bookhan V, Becker LH, Oosthuizen MP. A comparison of continuous clinical assessment and summative clinical assessment in Restorative Dentistry. S Af Dent J 2007; 62: 258-62.         [ Links ]

23. Albino JE, Young SK, Neumann LM et al. Assessing Dental students' competence: Best practice recommendations in the performance assessment literature and investigation of current practices in pre-doctoral dental education. J Dent Educ 2008,72: 1405-35.         [ Links ]

24. Nulty DD, Short LM, Johnson NW. Improving assessment in dental education through a paradigm of comprehensive care: A case report. J Dent Educ 2010; 74: 1367-79.         [ Links ]

25. Schönwetter DJ, Law D, Mazurat R, Sileikyte R, Nazarko O. Assessing graduating dental students' competencies: the impact of classroom, clinic and externships learning experiences. Eur J Dent Educ 2011; 15: 1-1.         [ Links ]

26. Priya M, Muthu MS, Deepti Aand Eapen T. Continuous assessment of undergraduate students at a Dental College in India. J Dent Educ 2012; 76; 501-8.         [ Links ]

27. Johnsen DC, Lipp MJ, Finkelstein MW. Guiding dental student learning and assessing performance in critical thinking with analysis of emerging strategies. J Dent Educ 2013; 76: 1548-58.         [ Links ]

28. Snyman WD, Ligthelm AJ. The new Pretoria curriculum. S Af Dent J 2000; 55: 642- 48.         [ Links ]

29. Bangert-Drowns R, Kulik CLC, Kulik J, Morgan MT. The instructional effect of feedback in test-like events. Review of Educ Res 1991; 61: 213-38.         [ Links ]

30. Hattie J, Timperly H. The power of feedback. Review of Educ Res 2007; 77: 81-112.         [ Links ]

31. Kramer GA, Albino JEN, Andrieu SC et al. Dental student assessment toolbox. J Dent Educ 2009; 73: 12-35.         [ Links ]

32. Redwood C, Winnin T, Lekkas D, Townsend G. Improving clinical assessment: evaluating students' ability to identify and apply clinical criteria. Eur J Dent Educ 2010;14:136-44.         [ Links ]

33. Hanson K , Alexander S. The influence of technology on reflective learning in dental hygiene education. J Dent Educ 2010; 74: 644-53.         [ Links ]

34. Wetmore AO, Boyd LD, Bowen DM, Pattillo RE. Reflective blogs in clinical education to promote critical thinking in dental hygiene students. J Dent Educ 2010; 74: 1337-50.         [ Links ]

35. Henzi D, Davis E, Jasinevicius R, Hendricson W. In the students' own words: what are the strengths and weaknesses of the dental school curriculum? J Dent Educ 2007; 71: 632-45.         [ Links ]

36. Appendix. Competencies for the New General Dentist. J Dent Educ 2008; 72: 1432-35.         [ Links ]

37. Kilfoil WR. Investigation into a dual or multi-mode teaching mode at the University Of Pretoria. Position Paper: Department of Education Innovation 2011: 1-19.         [ Links ]

38. Kogan JR, Holmboe ES, Hauer KE. Tools for direct observation and assessment of clinical skills of medical trainees: a systematic review. J Am Med Assoc 2009; 302: 1316 -26.         [ Links ]

 

 

Correspondence:
V Bookhan
Department of Odontology
School of Dentistry, University of Pretoria
South Africa
Tel: +27 12 319 2277
Fax: +27 12 548 2864
E-mail: Vinesh.Bookhan@UP.ac.za