SciELO - Scientific Electronic Library Online

 
vol.48 número1Occupational therapy students' experiences and perceptions of culture during fieldwork educationConsiderations when assessing urban South African children with the Developmental Test of Visual Perception 2nd edition (DTVP-2) índice de autoresíndice de materiabúsqueda de artículos
Home Pagelista alfabética de revistas  

Servicios Personalizados

Articulo

Indicadores

Links relacionados

  • En proceso de indezaciónCitado por Google
  • En proceso de indezaciónSimilares en Google

Compartir


South African Journal of Occupational Therapy

versión On-line ISSN 2310-3833
versión impresa ISSN 0038-2337

S. Afr. j. occup. ther. vol.48 no.1 Pretoria abr. 2018

http://dx.doi.org/10.17159/2310-3833/2017/vol48n1a8 

SCIENTIFIC ARTICLES

 

Psychometric evaluation of the Quick Screening Procedure for referral to Occupational Therapy (QSPOT) for five year olds with and without barriers to learning

 

 

Lauren Jeannie VialI; Denise FranzsenII

IBSC OT (Wits), MSc OT (Wits). Postgraduate Student, Department of Occupational Therapy, School of Therapeutic Sciences, Faculty of Health Sciences, University of Witwatersrand; Private practitioner, Kempton Park, Gauteng
IIBSC OT (Wits), MSc OT (Wits), PhD (Wits) DHT (Pret). Senior Lecturer, Department of Occupational Therapy, School of Therapeutic Sciences, Faculty of Health Sciences, University of Witwatersrand

Correspondence

 

 


ABSTRACT

INTRODUCTION: Adjustments made to the Quick Screening Procedure for Referral to Occupational Therapy (QSPOT) required further validation of the test. This test which was developed in South Africa, is used to screen motor, praxis and sensory-perceptual performance skills related to intrinsic barriers to learning in 4 to 6 year old children.
METHOD: The aim of this study was to determine the known group discriminant validity and internal consistency of the QSPOT in identifying intrinsic barriers to learning in 5 year olds. The concurrent criterion validity of the QSPOT compared to the Movement ABC - 2nd Edition (MABC-2), and the Developmental Test of Visual-Motor Integration - 6th Edition (DtVMI-6) (Visual Motor Integration and Visual Perception subtests) as well as the accuracy of these tests were established. Seventy seven learners in mainstream schools and Learners with Special Education Needs (LSEN) schools were assessed with all three tests.
RESULTS/FINDINGS: In terms of accuracy acceptable specificity was found for the QSPOT; however, sensitivity was not at an adequate level and lower than for the MABC-2 and the DTVMI-VMI. Adequate concurrent criterion validity was found between the QSPOT Total Score and the MABC-2 Total Score, as well as between the QSPOT and the DTVMI-VMI for Age-band 1 (5 years 0 months to 5 years 5 months), but not for Age-band 2 (5 years 6 months to 5 years 11 months.
CONCLUSION: The QSPOT accurately identifies learners without barriers to learning, but may under-identify those with barriers to learning. Concurrent criterion validity of the QSPOT to the DTVMI-VMI and MABC-2 indicate that similar motor, praxis and sensory-perceptual performance skills deficits are identified within Age-band 1 but not for Age-band 2. In light of these findings revision of scoring and cut off criteria should be reviewed for certain items.

Key words: Intrinsic barriers to learning, motor and praxis performance skills, sensory-perceptual performance skills, screening, screening procedures, sensitivity, specificity, concurrent criterion validity.


 

 

INTRODUCTION

In 2009, the West Rand Occupational Therapists in Private Practice in Gauteng, South Africa, undertook to develop a screen assessment of intrinsic barriers to learning, related to client factors, motor and praxis performance skills, and sensory-perceptual performance skills for Grade 0 learners that could be administered and applied in the South African context. The Quick Screening Procedure for Referral to Occupational Therapy1,2 (QSPOT) was developed to screen learners between the ages of 4 years 0 months and 5 years 11 months. The QSPOT has undergone several changes dividing each age into two six-month age-bands with altered or added scoring for various tasks3, thus requiring further investigation for standardisation purposes. This paper reports on investigation for standardisation purposes for children 5 years 0 months to 5 years 5 months and 5 years 6 months to 5 years 11 months.

 

LITERATURE REVIEW

Barriers to learning are any number of hindrances that prevent a child from performing effectively in the educational occupational performance area4. The barriers may be internal or external, often only becoming apparent when learners show an inability to cope with or derive benefit from the learning process, which ultimately leads to school failure4,5. It is recognised that early and accurate detection of varying areas and levels of dysfunction within the learner, that hinder educational performance, is required to address barriers to learning timeously4,6.

To prevent delay in the identification of intrinsic barriers to learning, the South African Department of Education (DoE) has placed emphasis on the accurate assessment of appropriate performance skills of all learners in Grade 0 and Grade 157. The National Strategy on Screening, Identification, Assessment and Support (SIAS)7published by the DoE in 2008, provides guidelines for the screening, identification, formal assessment, and referral for appropriate intervention of learners with barriers to learning. According to this document7, all learners should be screened by early childhood development practitioners and health services before starting school, with at risk learners requiring further screening during the admission stages of Grade 0 and Grade 1. The DoE has authorised the use of mainly teacher-produced and teacher-completed checklists7, while the role of health practitioners, including occupational therapists, in screening within the SIAS process is unclear.

Research supports the use of assessment tools which are compiled by a panel of experts in the area of child development, many of which are completed by other individuals such as teachers and parents8-10. It is recognised that healthcare providers such as occupational therapists should be involved in the screening for barriers to learning before school-going age7,11, if not also at the Grade 0 and Grade 1 admission stage. This is in line with the scope of occupational therapy in educational performance, which includes the assessment of pre-academic areas of functioning such as motor and praxis, and sensory-perceptual performance skills. These skills include those involved in positioning the body, arms and hands to manipulate pencils and scissors as well as the directional changes in movement needed to copy shapes or cut on a line12-14. Muscle tone, eye-hand coordination and spatial skills are examples of client factors12 that are observed by occupational therapists in these tasks. Several standardised assessments15-17 are used by occupational therapists for the early identification of these pre-academic barriers to learning, but short valid screen assessments are needed for the screening children before more expensive extensive testing is done to identify specific problems.

Screen assessments contain a useful combination of various skills within a few items18, which should be familiar and culturally sensitive to the learners6,19. The screening results should indicate the necessity and type of comprehensive evaluation that is warranted6,11,20, thus ensuring the appropriate referral of only those learners who do require more in-depth assessment5,7. The validity, reliability, sensitivity and specificity of screen assessments need to be ensured in order for them to be considered evidence-based tools6,11,19. The investigations are not only to investigate the instrument's effectiveness, but also ensure that the occupational therapists using the test utilise and interpret its results appropriately, and the results are valid and reliable in identifying barriers to learning21.

 

PURPOSE OF THE STUDY

As part of a larger study, the purpose of this study was to determine the psychometric properties of the QSPOT for children aged 5 years to 5 years 11 months. This article reports on the accuracy of the QSPOT, the Movement Assessment Battery for Children - 2ndEdition (MABC-2) and the Beery-Buktenica Developmental Test of Visual-Motor Integration-6th Edition (DTVMI), for the visual-motor (DTVMI-VMI) and visual perception (DTVMI-VP) subtests as well as the and concurrent criterion validity between the QSPOT and the other two tests. Known group or discriminant validity was established for the QSPOT as well as the reliability in terms of internal consistency.

 

METHOD

This study used a quantitative non-experimental22, cross-sectional23, correlational22 research design. The sample consisted of learners between the age of 5 years and 5 years 11 months with and without barriers to learning. Once ethical clearance had been obtained from the Human Research Ethics Committee at the University of the Witwatersrand (MI207II), and approval of the study had been obtained from the Gauteng Department of Education (GDE), mainstream schools and schools for Learners with Special Educational Needs (LSEN) were approached in the Johannesburg area of Gauteng, to recruit a sample of learners. Permission was obtained from seven mainstream schools and eight LSEN schools who indicated that they had learners who suited the inclusion criteria, and permission to conduct the assessments was granted by the school principals. The mainstream participants were taken from a variety of independent and public suburban schools whereas only middle to upper class suburban schools were used to obtain participants for the LSEN sample.

A ratio of ten participants per item on the QSPOT was used to determine a sample size of 50 participants without barriers to learning attending mainstream schools, and 50 learners with barriers to learning attending LSEN schools. Learners were divided into two six-month age-bands as required by the QSPOT2,3, namely 5 years 0 months to 5 years 5 months (Age-band I), and 5 years 6 months to 5 years II months (Age-band 2).

Mainstream learners with a reported history of previous investigations and interventions for intrinsic barriers to learning were excluded, while learners at LSEN schools were included if they had definite or provisional diagnoses which resulted in intrinsic barriers to learning. Stratified sampling was used to ensure an equal number of male and female learners24 in each group.

Parent information sheets, informed consent forms and background questionnaires were distributed by the teachers in the mainstream schools to the parents of 67 learners for recruitment into the study. Informed consent was received for 50 learners; however, two were excluded due to illness and injury occurring between the test administrations, so the sample size was 48.

To recruit the LSEN group, 50 parent information sheets, informed consent forms and background questionnaires were given by the teachers to the parents of the learners selected for the study. Informed consent was obtained for 43 learners. The learners were recruited from a combination of LSEN and remedial schools, and learners who were in remedial classes in mainstream schools, due to a shortage of learners who met the inclusion criteria for the LSEN sample in LSEN schools alone. Some learners with physical impairments and intellectual impairments were therefore also recruited as learners presented with diagnoses or provisional diagnoses of Autism Spectrum Disorders (ASD), possible intellectual disabilities, hearing and speech impairment, ADHD and remedial difficulties. Four of these learners were subsequently eliminated due to difficulties in completing all the test items and 10 learners were excluded due to being in the incorrect age-group or withdrawal and refusal to participate. The remaining 29 learners were assessed for the study so the total sample consisted of 77 learners.

Instruments used

Quick Screening Procedure for Referral to Occupational Therapy (QSPOT)

The QSPOT has been designed to screen learners between the age of 4 years 0 months to 5 years II months for the presence of intrinsic barriers to learning1,3. The test meets the guidelines for the development of screening procedures, namely that they should be user-friendly with a short administration time6,11,19,20 and no lengthy standardised testing20. The QSPOT is a criterion-referenced test as learners are not compared to a normal sample, but are instead marked according to whether they are able to perform the specific skills or not21,25.

The QSPOT includes items which are divided into four tasks. Task I: Draw-a-person (DAP) / Visual Motor Integration (VMI), Task 2: Cutting, Task 3: Balance, Task 4: Catching. The tasks, which may contain one or more activity items, are used to screen body awareness, fine motor skills and visual motor perception. Task I: DAP/VMI is in the form of pencil-and-paper activities, namely drawing a person and copying shapes (horizontal and vertical lines, a cross, circle, square, triangle, a diagonal cross within a rectangle, and a diamond) from stimulus cards, followed by an added item in which the learner names the shapes. Task 2: Cutting contains a single activity item for cutting. Gross motor skills are screened using Task 3: Balance with a single activity item for static balance, and an additional descriptor for static balance with eyes closed, and Task 4: Catching, with a single activity item for catching1,3. Administration time is approximately I5 to 20 minutes3.

Each activity item has a number of descriptors or observations that are observed and scored while the learner completes the activity. One point is scored against each descriptor that was not achieved during the performance of the activity item and these scores are totalledI. Guidelines as to when to score against the descriptors are provided in the manual1. If the learner achieves the behaviour in the descriptor this is not scored so the QSPOT scores indicate areas of concern only.

Norms which should be achieved in each activity item for ages 3 years, 4 years, 5 years and 6 years are also provided on the score sheet. Failure to achieve the appropriate age norm for the activity item results in a three-point weighting being added to the corresponding total score recorded on the descriptors1, 2. A subtotal for each task item is obtained by adding the scores for descriptors and the three-point weighting if applicable. The subtotals of the tasks are added to provide a Total Score1,2.

In an initial pilot study in 2009, the content validity and some aspects of construct validity for the QSPOT2 was determined on a sample of 118 randomly-selected learners between the ages of 5 years 0 months and 5 years 11 months using the Rasch Analysis26. The results of the analysis showed, as expected, that a large majority of the learners scored in the normal range of functioning. The results also suggested that the QSPOT was able to discriminate between learners with barriers to learning, from those who are typically developing in terms of their learning and performance in the classroom2.

The Movement Assessment Battery for Children - 2ndEdition (MABC-2)8

For the purposes of determining the concurrent validity of the QSPOT in this study, only the motor skill test of the MABC-2 was used, which consisted of the Manual Dexterity Component, the Balance Component, and the Aiming and Catching Component. The MABC-2 was standardized on a sample of 1,172 children aged between 3 years 0 months to 16 years 11 months from the United Kingdom, which was stratified for gender, race, geographical location and level of parental education. Adequate reliability and validity are reported in the manual8. The MABC-2 has been found to have adequate cross-cultural validity for international populations8,27, and precise measurement capabilities28. It has also been widely used and cited in research studies conducted internationally27-29 and locally in South Africa30, including those that evaluate other checklists that identify deficits in motor skills9,30.

The Beery-Buktenica Developmental Test of Visual-Motor Integration - 6th Edition (DTVMI-VMI/VP)

The DTVMI-VMI31 assessment is a gold-standard for the assessment of visual-motor integration function. It is recognised as having adequate cross-cultural validity31-33, including predictive validity of academic performance in black South African learners from higher and lower socioeconomic backgrounds32. It has been historically used in research within South Africa33-36 and internationally37-39.

The test has been found to have adequate specificity (86%), but unacceptable sensitivity in identifying handwriting difficulties37. Adequate concurrent validity has been found with other tests assessing VMI31. Adequate relationships have also been found between the DTVMI-VMI, and the writing of individual letters38, copying sums and a written passage34, and educational performance areas of mathematics, reading and writing34.

Data collection procedures

Two research assistants were recruited to administer the MABC-2 and the DTVMI to participants, so that the researcher could be blinded to the learners' performance on those tests. The research assistants were occupational therapists with five and more years of experience in paediatrics, who either had further post-graduate training or experience in working as a tut