SciELO - Scientific Electronic Library Online

 
vol.14 número1Upcoming research articles in Health SA GesondheidUsing subjective judgement to determine the validity of a tutorial performance evaluation instrument índice de autoresíndice de materiabúsqueda de artículos
Home Pagelista alfabética de revistas  

Servicios Personalizados

Articulo

Indicadores

Links relacionados

  • En proceso de indezaciónCitado por Google
  • En proceso de indezaciónSimilares en Google

Compartir


Health SA Gesondheid (Online)

versión On-line ISSN 1025-9848

Health SA Gesondheid (Online) vol.14 no.1 Cape Town  2009

 

ORIGINAL RESEARCH

 

Weighting of items in a tutorial performance evaluation instrument: Statistical analysis and results

 

 

Melanie L. LackI; Judith C. BruceI; Piet J. BeckerII

IDepartment of Nursing Education, University of the Witwatersrand, South Africa
IIDepartment of Biostatistics, University of the Witwatersrand, South Africa

Correspondence

 

 


ABSTRACT

Weighting of items in an evaluation instrument contributes to more meaningful and valid interpretations of student performance in respect of each learning outcome or item being assessed. It follows that the validity of instruments is important for meaningful inferences about students' learning performance, including their performance in tutorial groups. The Delphi technique was used to elicit experts' subjective judgement of the content validity of items in the tutorial performance evaluation instrument in rounds one and two. A sample of eight experts (n = 8) was selected by purposive, maximum variation sampling.
In round three Delphi a weighted score was determined for each of the instrument items, subitems and Likert scale points through pairwise comparison by the experts. Mathematical modelling of experts' weighting comparisons, recorded on visual analogue scales, resulted in proportional weights for each item; these weights are expressed as a percentage.
The final instrument comprised weighted items measured on a rating scale with points that are not equidistant. A computerised tutorial performance evaluator (TPE) was developed for accurate, economical and efficient calculation of student scores. The purpose of this article is to report on the statistical analysis and results of the weighting of items in an instrument to assess and evaluate baccalaureate nursing students' performance in problem-based learning tutorials.

Keywords: weighting; tutorial performance; problem- based learning; evaluation instrument; statistical analysis; Delphi technique


OPSOMMING

Die waardebepaling van items in 'n evalueringsinstrument dra by tot 'n meer betekenisvolle en geldige interpretasie van studente se vermoë ten opsigte van elke leeruitkoms of item wat geassesseer word. Hieruit volg dit dat die geldigheid van instrumente belangrik is vir betekenisvolleafleidings betreffende studente se leervermoë, insluitend hulle prestasie in leergroepe. Die Delphi-tegniek is gebruik in rondtes een en twee om kundiges se subjektiewe oordeel oor die inhoudsgeldigheid van items in die leergroepgedragsevalueringsinstrument te bekom. 'n Steekproef van agt kundiges (n = 8) is deur doelgerigte, maksimale-variasiesteekproeftrekking gekies.
Die kundiges het in rondte drie van die Delphi-tegniektoepassing deur middel van gepaarde vergelyking 'n waarde bepaal vir elk van die instrumentitems, sub-items en Likertskaalpunte. Die wiskundige modellering van kundiges se waardebepalingsvergelykings, aangeteken op visuele analogiese skale, het proporsionele gewigte vir elke item tot gevolg gehad. Hierdie gewigte word persentasiegewys voorgestel. Die finale instrument het bestaan uit items wat gemeet word teen 'n graderingskaal met punte wat nie op 'n gelyke afstand van mekaar is nie. 'n Rekenaargebaseerde leergroepgedragsevalueringsinstrument is ontwikkel vir die akkurate, ekonomiese en doeltreffende berekening van studente se punte.
Die doel van die artikel is om verslag te lewer oor die statistiese analise en resultate van die waardebepaling van items in 'n instrument om baccalaureate-verpleegkundestudente se prestasie in probleemgebaseerde leergroepe te meet en te evalueer.


 

 

“Full text available only in PDF format”

 

 

REFERENCES

Beekman, T.J., Cook, D.A. & Mandrekar, J.N., 2005, 'What is the validity evidence for assessments of clinical teaching?', Journal of General Internal Medicine 20, 1159-1164.         [ Links ]

Cook, D.A. & Beckman, T.J., 2006, 'Current concepts in validity and reliability for psychometric instruments: Theory and application', The American Journal of Medicine 119(2), 166. e7-166.e16.         [ Links ]

Crawford, G. & Williams, C., 1985, The analysis of subjective judgment matrices. A Project Air Force report, Rand, Santa Monica.         [ Links ]

David, H.A., 1963, The method of paired comparison, Charles Griffin and Co. Ltd., London.         [ Links ]

Miranda, E., 2001, Improving subjective estimates using paired comparisons, IEEE Software, Ericsson Research Canada, Mississauga, Ontario.         [ Links ]

Niemenin, J., Saure, P. & Lonka, K., 2006, 'On the relationship between group functioning and study success in problem-based learning', Medical Education 40, 64-71.         [ Links ]

Patton, M.Q., 2002, Qualitative research and evaluation methods, Sage, Thousand Oaks.         [ Links ]

Rideout, E., 1999, Transforming nursing education through problem- based learning, Jones and Bartlett Publishers International, London.         [ Links ]

Savin-Baden, M., 2000, Problem-based learning in higher education: Untold stories, viewed 7 July 2006, from: http://www.mcgraw-hill.co.uk/openup/chapter.         [ Links ]

 

 

Correspondence:
Judith Bruce
Department of Nursing Education
Faculty of Health Sciences
7 York Road, Parktown
Johannesburg 2193
South Africa
E-mail: judith.bruce@wits.ac.za

Received: 15 Oct. 2007
Accepted: 27 Oct. 2008
Published: 21 May 2009

Creative Commons License All the contents of this journal, except where otherwise noted, is licensed under a Creative Commons Attribution License