THE 22^ND VONA DU TOIT MEMORIAL LECTURE

 

Stepping Stones from Input to Outcomes: An Occupational Perspective

 

 

Daleen Casteleijn

PhD (Pret); MArb (Pret); Diploma Higher Education and Training Practice (Pret); Postgraduate Diploma in Vocational Rehabilitation(Pret); BOcc Ther (Hons)(med); BOcc Ther (Pret) Senior Lecturer, Occupational Therapy Department, School of Therapeutic Sciences, Faculty of Health Sciences, University of the Witwatersrand

Correspondence

 

 

---

ABSTRACT

Occupational therapists in South Africa do not seem to have, as yet, adopted routine outcome measurement in daily practice. Although there is an abundance of valuable clinical contributions by occupational therapists, there is little evidence of recorded change in clients' activity participation or functional ability. This Vona du Toit Memorial Lecture addresses this apparent shortcoming in practice. The value of the work of Vona du Toit was highlighted as well as two other pioneers in the health care arena: Mary Reilly and Georg Rash.
Seven stepping stones to consider when implementing routine outcome measurement were presented. The importance of having an occupational therapy mainstay was argued and that this mainstay should be occupational performance. Routine outcome measurement was presented as a viable strategy for basic evidence indicators of occupational performance at any time, any place and without much effort. The basic science underpinning development of such measures was clarified and presented as a way for occupational therapy to get a foot in the door to enhance its recognition as a powerful profession whilst proving the invaluable change that meaningful occupational performance can bring about.

---

 

 

INTRODUCTION

Dear friends and colleagues, I thank you so much for honouring me with the award of the prestigious Vona du Toit Memorial Lecture. It is an honour for me to stand in front of such a distinguished audience tonight to share with you my perceptions and views of our beloved profession. My passion for occupational therapy started 30 years ago and today my belief in our philosophy is stronger than ever.

When it was announced that I was selected to do this lecture, I immediately knew my topic would include outcome measurement, a passion of mine for the past ten years. However I was not sure how to address the issue in a meaningful way. When I listened to all the wonderful presentations delivered at the OTASA Congress (2012), it confirmed my conviction that the topic of outcome measurement needs to be addressed and that it has to be one of the layers of the unfolding occupational therapy story.

I finally decided to introduce you to three formidable persons from the past who inspired me in my search for a perfect strategy to enable outcome measurement in Occupational Therapy.

These three persons contributed to the scientific world in the 1960's and 1970's. Mary Reilly of the USA who contributed greatly to the study of occupational behaviour; Vona Du Toit, the definitive occupational therapist from South Africa who developed the theoretical constructs and frame-work for the Model of Creative Ability; and Georg Rasch, a Danish mathematician who developed the Rasch Measurement Model.

Allow me to take you back to 1961 when Mary Reilly was awarded the American Occupational Therapy Association's Eleanor Clark Slagle Lecture. The title of the lecture was: "Occupational Therapy Can Be One of the Great Ideas of 20th-Century Medicine"¹. Now 51 years later, I wish to concur that we indeed were one of the greatest ideas of the 20^th century medicine and I believe that we will continue to be in the 21^st century. The keynote speaker at this Congress, Prof Ikiugu, enthusiastically provided unequivocal evidence that it is Occupational Therapy, with its philosophy of meaningful occupation which promotes health and well being, that is needed to save the world².

Mary Reilly's well known hypothesis that "man, through the use of his hands as they are energized by mind and will, can influence the state of his own health"¹ became one of the ten most quoted statements to date and I am sure you have come across it³.

Mary Reilly's hypothesis formed the essence of the art and science of our profession for the past 50 years. We still believe that a person needs to be engaged and participate successfully in everyday activities, also called occupations, as this engagement influences his/her health. Mary Reilly was a pioneer in the promotion of the study of people engaging in occupations, her groundbreaking work on occupational behaviour continues to be relevant for practice.

Over time numerous frames of references and models of practice evolved and assessment and interventions were developed to enhance and support meaningful occupational behaviour as relevant for all those with occupational needs.

During a recent survey among the training institutions in SA it was interesting to note which occupational theories and frameworks drive the curricula in SA⁴. Occupational Science and the Occupational Therapy Practice Framework are taught by all eight of the universities (see Figure 1). The Model of Human Occupation, developed by Gary Kielhofner who was also inspired by Mary Reilly's work on occupational behaviour, features prominently, while five universities teach the Vona du Toit Model of Creative Ability. From this survey it was clear that we strive to equip our future occupational therapists with knowledge and skills to use theories and frameworks to address the occupational needs of the South African population.

 

 

During many years of student supervision and exposure to a variety of clinical departments I have been privileged to observe occupational therapy staff provide the most appropriate treatment to their clients regardless of severely limited resources. I have, furthermore, had access to reports reflecting very creative ways to facilitate engagement in meaningful activities. I have witnessed group treatment given to those whom other professionals have given up on. Occupational therapists have on a daily basis supported meaningful occupations in those with severe disabilities and learning disabilities, for example facilitating the establishment of support groups for various needs such as the groundbreaking Grandmothers against Poverty and Aids (GAPA)⁵ project; we even have the proudly South African Outeniqua Wheel-chair challenge, initiated by a colleague. These are but a few examples, I could easily, if time allowed, continue to reflect on the outstanding work that occupational therapists have been doing in the past, and we are still doing today.

However, I believe that the time is ripe that we, as a unique therapeutic science, use a new set of glasses to look at the evidence of our great successes. I believe that if we reflect on our evidence based achievements we would see new horizons, new goals and new challenges.

Through my research of the past ten years and my exposure to different areas of practice, I observed that although occupational therapists seem to be focusing on rendering quality client-centered services, they are, regrettably, providing little scientific evidence that their services have indeed created a change in the client.

Without basic information to show effectiveness one finds it difficult to confirm that our service is indeed an efficient commodity contributing significantly to the recovery process of the people in South Africa. In a very short space of time this quest for evidence of effective treatment has become one of the main drivers of funding in most parts of the world and perhaps more so in private health care than in government hospitals in South Africa. This lack of scientific evidence renders our great profession extremely vulnerable. If we wish to achieve wide recognition, we have to do something about providing the necessary evidence.

The critical questions we need to ask ourselves to reach this point include "Where in the occupational therapy practice will we find the evidence of change?" and "How will we find the evidence?" I firmly believe that we, as practitioners, have to adapt to the practice of routine outcome measurement and that this is eminently possible. My wish and mission is that occupational therapists will have basic evidence indicators routinely available, to use at any time, any place and without much effort.

Allow me to use the nursing profession as an example. Nurses perform basic procedures that provide powerful evidence of change; these include the measuring and recording of vital signs such as blood pressure, temperature, pulse and respiration rate to measure important health functions. These are four very basic functions but by measuring and recording them routinely, nurses not only provide evidence of change in the client, but may also indicate the quality of nursing care. These vital signs are the mainstay of the nursing process. Each vital sign tells a different story but together they describe client acuity and the need for nursing intervention. In occupational therapy we lack equivalent basic measures to use routinely to inform others of the efficacy and need for occupational therapy intervention. I realise that you might argue that we are not nurses; that our intervention is too complex and that occupation cannot be measured or reduced to numbers. The reality is that we no longer have a choice; we have to identify and measure the occupational therapy mainstay if we are to receive the recognition for our contribution to health and well-being we believe we deserve. For purposes of this paper the term mainstay is used to describe the very basic tenets of a profession.

Let me get back to my proposed strategy of routine outcome measurement, I am fully aware that attempting to embed routine outcome measurement in practice is not an easy one-step exercise; it needs to be situated within the overall measurement strategy. Through my research on routine outcome measurement I came to realise that occupational therapists do not necessarily have the knowledge and guidelines to develop the needed routine outcome measures, and when developed, they seem at a loss as to how to implement them routinely and effectively into everyday practice. I further found that occupational therapists are under the impression that assessments and outcome measures are the same thing. However, I very encouragingly, found when taught how to use and apply routine measures they gain confidence in their contribution, develop a new uniform language and they prove themselves to be indispensable to the multi-disciplinary team.

In my search to find the basic tenets that could be used in a routine Occupational Therapy measurement tool, I found the work of Vona du Toit most useful and instrumental. Significantly Vona also produced her work in the 1960's, as did Mary Reilly and Georg Rasch, thus putting them into a similar contextual framework.

 

VONA DU TOIT - A SHORT HISTORY

In keeping with the significance of this award I wish to pay tribute to the great contribution made by Vona to the development of a sound theoretical grounding for Occupational Therapy in South Africa, through the development of the the Model of Creative Ability. We have several so-called masters in the audience who knew Vona well and were taught by her. In Creative Ability circles, they have been nick named the mother ducks. I would like to acknowledge their contribution, made alongside Vona as colleagues and students, up to and even after her untimely death, also in areas other than Creative Ability. Their experience and wisdom have been invaluable for the growth of Occupational Therapy in South Africa.

We also have the new comers in the audience and to continue with the nick naming, let us call them chicks. The mother ducks are well aware of Vona's history and her contribution but our chicks might need to hear about the history and appreciate one of the greatest occupational therapists in our South African history. This is after all the 22^nd Vona du Toit Memorial Lecture.

Vona du Toit (nee van Straaten) qualified as an occupational therapist from the University of the Witwatersrand in 1946. She was one of the first five South African trained OTs. Before this qualification she obtained diplomas in Primary and Higher Education as well as a Teachers Special Class Diploma. She established and worked at several hospitals in Johannesburg and Pretoria before taking up a teaching post⁶.

Vona du Toit , became head of training, and was joined by Ilse Eggers at the Pretoria College of Occupational Therapy in 1963, her work on Creative Ability had its origins in a dissertation on 'Initiative' which she completed as part of the requirements for a Tertiary Education Diploma. The fundamentals of the Model emerged in the mid 1960's and continued to develop and become refined up until her death in 1974⁶.

A singular honour was bestowed on Vona, especially as this was during the 'Apartheid' years, when the World Federation of Occupational Therapists awarded Vona du Toit an Honorary Fellowship of WFOT in Vancouver, in 1974, honouring her for her "indestructible belief in the worth of her profession and her unflagging efforts to stimulate its growth in breadth and depth"⁶. It is significant that this honour was only very recently, 36 years later, bestowed on a second South African, Dr Rosemary Crouch, in Chilli 2010 . Vona du Toit had, despite a short life span of 52 years, left an enduring legacy. Although Vona regrettably produced only a limited number of publications she was a prolific teacher and leader. She presented her theory and convictions at many forums both nationally and internationally. Selected material was however included in a booklet (literally called "the bookie") published by the Vona and Marie du Toit Foundation 1980⁶ and in later years, the chapter by de Witt in Crouch and Alers⁷ which has been accepted as the seminal text on the model.

It always amazes me that although she had produced limited research and very few publications that her Model of Creative Ability has endured and grown in stature and is applied by thousands of practitioners to whom the use of the model is an integral part of who they are as therapists. Her thinking at the time was way ahead of that of her peers and very much in keeping with current day approaches.

Vona did many presentations at congresses and meetings, to illustrate the above statement and show the linkup with the topic of my lecture, I quote from her Creative Ability lecture of 1970 given at the National OT Congress in Cape Town, where she stated: "The medical profession demands that any new clinical procedure be validated in terms of clinical results. An assessment of the value of a procedure is done according to the negative or positive effects elicited in the client by the application of the procedure"^6:5. She suggested that this effect be measured in terms of a scale of measurement like the sequential development of recovery of creative ability^6:5. On reading these statements I again realised that Vona was not only developing a philosophy of activity participation and purposeful engagement in everyday activities but she was also on a mental journey to develop a generic measurement tool for Occupational Therapy. The levels of motivation with corresponding action are used extensively in clinical practice today. Table I illustrates these levels.

 

 

THE VONA DU TOIT MODEL OF CREATIVE ABILITY - DO THE LEVELS EXIST?

For my PhD study I had the enormous privilege of analysing Vona's theoretical assumptions, concepts and constructs⁸ and wish to share insights gained with you as part of my introduction to the real topic of my lecture.

During the late 1960's and early 70's Vona realised the importance of objective observations of actions or abilities in a person's functioning on different levels which enabled the prediction of the difficulty level of the tasks that the person should be able to complete. Her formulation of nine sequential and interrelated levels of creative ability (Table 1) and the profound conceptualisation that through observation of the actions of a person, the therapist is able to determine the direction and strength of the motivation of that person, stimulated me to investigate the validity of the existence of the levels of creative ability. My question was:" Do the levels exist?" Intuitively we know they exist, but the evidence that the levels actually exist was not scientifically investigated. My search to answer these questions helped to guide my research.

There are many methods in statistical analysis to investigate validity but the best one in my opinion and for the question posed above was the Rasch Measurement Model^9,10, making this the appropriate time to introduce the third person, Georg Rasch, the Danish mathematician who also contributed ground breaking work in the form of statistical procedures during the 1960's. Although Vona and Georg never, to my knowledge, met or communicated with one another they seemed to share the idea that a person's ability is a function of a task's difficulty. In other words persons with less ability will find the task difficult while persons with more ability will find the task easy¹⁰. It sounds so simple and logical, but history has shown to us that it is in simplicity that one finds the great discoveries of time. Vona found these basic constructs in the occupational therapy sciences and Georg Rasch found it in the mathematical sciences.

I wish space allowed for me to explain the magic of Georg Rasch's work in detail but unfortunately that is not possible. I would like to move on, or rather back to the title of this paper namely the stepping stones in routine outcome measurement and while doing that, I will touch on some of the Rasch techniques.

 

SEVEN STEPPING STONES - FROM INPUT TO OUTCOMES

To fully understand the seven stepping stones I need to take you into the world of measurement principles, which of necessity becomes very technical at some points, but is essential to enable us to come up with useable and valid data. I will use some examples of my own research to illustrate the application of these principles.

The seven stepping stones that I will present must not be viewed hierarchically or as a flight of stairs where you have to complete the first one before you can attempt the second; rather see them as equally important stepping stones that will help you to get safely across a river. The stepping stones each represent an important component in the journey from service input to treatment outcomes.

Stepping stone 1- the single target

Firstly decide what your Occupational Therapy programme sets out to achieve. Make sure it is a single target you are hoping to achieve i.e. think unidimensional. The single target in nursing is their vital signs. It consists of a few domains or concepts but together they form one construct of vital signs. You have to find your "Occupational Therapy vital signs" as they inform about the most basic aspects of the occupational change you want to assess in the client.

The survey among the universities that I referred to earlier also indicated the myriad of occupational performance areas (see Figure 2) included in our curricula in SA⁴. There seems to be consensus within the profession that these occupational performance areas are the mainstay of our profession. I do realise that to have an occupational perspective in outcome measurement, we cannot only measure these areas, we have to account for the meaning of the occupations in a person's life, what motivates the person and how these occupations are performed for example habits and routines. However, we need to remember that we are trying to provide evidence of change, we are not describing occupational performance and therefore we need to measure the most basic targets of our interventions.

Having said that, I need to reiterate the importance of the occupational perspective in our search for recognition of an occupational therapy mainstay. Since the inception of the profession, our understanding of the concept of occupation improved tremendously, there was even an era where occupation took a back step to accommodate the medical model¹¹⁴. It was called the mechanistic paradigm where performance components became the focus of assessment and treatment in occupational therapy. This was driven by a better understanding of the human body as explained by the medical model, but fortunately the era from 1980 onwards shows a renewed focus on occupation, with the emergence of Occupational Science giving us further occupational impetus.

If we do not focus on occupation and remain grounded in an occupational perspective we run the risk of training therapists who are not clearly distinguishable from other health care professionals, of the domains of the profession becoming obscure, and of therapists who fail to provide opportunities for clients to (re)- engage in everyday activities and meaningful and purposeful occupations. We know that negative occupations (obsessive compulsions, bad habits like substance abuse) are detrimental to our health, leading to illness and sick societies and a world at crisis as was demonstrated by Prof Ikiugu in his keynote address. He further made it abundantly clear that meaningful occupations could save the world at crisis².

I am sure that I do not need to do any more convincing of what the mainstay of the Occupational Therapy profession is but I anticipate that you might be somewhat dubious at this moment by thinking if and how it would be possible to measure the myriad of "vital signs" in our clients AND measure it routinely! I am happy to assure you that it is quite possible and has been done successfully.

Stepping stone 2: Measurement, evaluation and assessment

The next component is the understanding of the terms measurement, evaluation, and assessment. The literature is not very helpful in this regard. These terms are often used interchangeably to describe any of the three concepts. A measurement is an empirical value you place on an observation^12:11 e.g. if a person has a fever, it may be as high as 39,6 degrees Farhenheit where the 39,6 degrees is the measurement of temperature. After treatment the fever may have dropped to 36,6 degrees. An evaluation is the judgment made¹²⁸ between two or more similar measures e.g. the temperature has dropped by 3 degrees to 36,6 degrees over a period of three hours. An assessment is the descriptive report on the context whereby the measure has been applied e.g. the age of the client, the diagnosis, the cause of the fever, the environment and the support systems to combat the fever.

In the clinical field of occupational therapy we tend to place much emphasis on assessments without empirical measurements and therefore our evaluations remain vague, descriptive and often lack scientific validity. One needs a measure to indicate the change that was effected after treatment.

Stepping stone 3: input, output and outcomes

After clarifying the terms of measurement, evaluation and assessment, we can move on to the process of input, output and outcomes.

In organisational theories input refers to the resources that contribute to the delivery of the output while outputs are the goods and services produced¹³. Outcomes are defined as the consequences or impact of service delivery¹³.

When applying these terms to the health care arena, input refers to resources like the standards of practice for training of future professionals as well as the standards of practice in the different areas of service provision. This will include ethics, theoretical frameworks like the Vona du Toit Model of Creative Ability, and many others. In my opinion, occupational therapy as a profession in South Africa is doing well as far as inputs are concerned as the profession has excellent procedures and protocols in place. The HPCSA and Professional Board for Occupational Therapy, Medical Orthotics and Prosthetics and Arts Therapy (Board) have done extremely valuable work in compiling standards for undergraduate training programmmes as well as general standards for practice, codes of conduct and ethical rules. The Board, additionally, recently distributed a first draft of the revised Occupational Therapy Scope of Profession and Scope of Practice. This implies that our basic pa- perwork is in good order. A huge misconception however seems to exist as we often assume that good inputs guarantee good outputs and good outcomes which is absolutely not true.

Output on the other hand refers to how skilled we have become in implementing the occupational therapy procedures. This is usually reflected in the turnover and volumes of clients we can handle and at the same time keep up to date with the "paper work". And how we hate "paper work", but despite such feelings we are obliged to continue providing evidence of our output. Quality of output has been observed to vary from setting to setting, I have unfortunately observed occupational therapists who seem more concerned about the procedure itself than the desired change in a client. Filling out the statistics and ordering equipment could and do at times take up many hours at the expense of hands-on intervention. It is not only occupational therapists in the health care professions who are guilty of this mismanagement of time and outputs. I often get a sense that health care professionals feel that the client is really obstructing their ability to perform their duty. If only there were no clients, their job to provide good outputs would have been so much easier! And most unfortunately I also observed health care professionals who could not even be bothered to do the paperwork. This unethical behavior should not be tolerated by any of us.

Good outputs therefore do not infer that we have facilitated meaningful occupational change in the lives of our clients, it only means we are hardworking!

What we really need is good outcomes; we want to know what the consequences of our service delivery are or how effective we were in enabling change in our clients. The focus in outcomes is not on how much we know (inputs) or how hard we work (outputs) but how smart we are in changing the occupational behavior of clients (outcomes). We can only achieve this by looking at the clients using an occupational measure and say: "On admission to my program the client measured 50% in terms of occupational performance and at the end of my program the client was performing at 80%. Therefore my effectiveness was an increase of 30% and I did it over a period of 30 days. Therefore my efficiency was 30% over 30 days which equals 1% per day".

For example, if I am able to average about 1 % improvement per day for clients in the same impairment grouping; e.g. mood disorders, then my benchmark for mood disorders is a 1% improvement per day. This improvement rate is then linked to the occupational therapy intervention that I use, but if I change my techniques or intervention and start getting better outcomes, I start setting new trends into the outcomes analyses. In this way an entirely new approach develops around our science of occupational therapy.

I am somewhat anxious that we labour under the misconception that, if our paperwork is in order and we work hard, it is enough evidence that we are a valuable service. The problem of this tension between input and outputs is that the client may no longer be the real focus of the occupational therapy programme. We are running the risk that most of our time is directed at designing new sets of forms, setting new rules and regulations, and spending money on training staff in the new procedures. The question that surely must be asked is 'when do we make time to reflect on the consequences of the input and outputs and even more importantly, reflect on the needs of the clients?'

When we move on to measure change in the client's occupational performance, that is when the focus moves back to the client. The client can see how s/he improves and the clinician feels confident about his/her treatment and is able to provide evidence of change.

All of the above is common knowledge, simple to understand and very logical. If that is the case, then why are we short of the outcomes data to do these very basic outcomes calculations? To my mind it is simply because we do not have the appropriate measures. This brings us to the fourth basic stepping stone.

Stepping stone 4: stevens' theory of scales of measurement

Before we can look at appropriate measures to measure the effect of our intervention, we have to review scales of measurement. In the early 1940's the scientific world was a very confused place. The social and human scientists, called the 'soft scientists', evolved as a new growth point in research, whilst the 'hard mathematician scientists' were the custodians of the empirical values. They refused to let any empirical number be abused by the 'soft scientists'. The 'soft scientists' seemed at a loss until Stevens published his well-known "On the theory of scales of measurement"¹⁴ in 1946. In the world of outcomes research the work of Stevens is agreed to be a fundamental steppingstone¹⁵.

The lowest level of scaling according to Stevens is to classify or sort into groups with similar characteristics and is called the nominal scale^12:139. This is not a measurement, but very useful tool to classify and apply in the comparison of similar groups with similar outcomes in the benchmark analyses.

Stevens' second level of scaling is the ordinal scale. This is a rudimentary scale whereby numbers are assigned to observations, themes or experiences in a hierarchical set of rules. This is commonly used in a Likert scale and is also referred to as qualitatively ordered, meaning it cannot be used in the basic adding or subtracting of raw data^12:140. The correct terminology for this level of scaling is a scale that provides scores. Therefore scales and scores cannot be analysed for outcomes analyses.

Stevens' third level of scaling is an interval measure that provides measurements that are useful in the analyses of outcomes as they are concatenated. In laymen's terms this means the intervals between each category is equal to the neighbouring categories^12:143. We need to produce interval measures for scientific outcomes analyses research. This is where Georg Rasch comes into the picture. Rasch developed the mathematical model to convert ordinal scales into interval measures for us to use as outcomes measures^9,10. But not all ordinal scales can be converted. There is a certain set of criteria which the scale has to comply with for it to fit the Rasch model^9,10. In an attempt to answer a question I posed earlier as to whether the levels of Creative ability did in fact exist, I subjected the levels of Creative Ability to this test and it fitted the Rasch model gratifyingly well. There are techniques to modify a scale to fit the Rasch Model but in the case with the levels of Creative Ability, it fitted so well that no changes were necessary.

Stevens' fourth scale is a ratio scale with an arbitrary zero point^12:145 and is only of concern for the "hard scientists".

I would like to return to the point where I said that the levels of Creative Ability fitted the Rasch model well and the scale was converted from an ordinal score to an interval measure. The example comes from the outcome measure that I developed. This outcome measure measures change in activity participation as I believe that activity participation otherwise called occupation, could become the mainstay of occupational therapy. The name of the outcome measure is the Activity Participation Outcome Measure (APOM)^8,16.

The scoring system of the APOM is based on the levels of Creative Ability. It consists of eight domains namely process skills, communication and interaction skills, life skills, role performance, balanced life style, motivation, self-esteem and affect. Each domain is then represented by different items (52 items in total) that make up that domain. The single construct which underpins this entire outcome measure is, not surprisingly, activity participation^8,16.

Each item has a description of an observable behaviour on each of the first six levels of creative ability. Each level is further divided into three phases which show progression within the level. Those of you who know the Model of Creative Ability will understand the terms levels and phases but what it means in the APOM, is that the scale has eighteen categories, three categories for each one of the six levels⁸.

I will use the domain of motivation to illustrate the scoring system. The first item in Motivation is active involvement and defined as the desire to engage in tasks or activities and demonstrating maximum effort and a sense of enjoyment and satisfaction. When this is described in the levels of creative ability, the "amount" or quality of active involvement is evident and a number can be assigned to it (see Table II). The clinician (trained in the VdTMoCA) observes the behaviour of her client and decides which level descriptor fits the observation best. After the level has been determined, the phase within the level is then judged. For example, if the client is on the self-presentation level, the clinician needs to decide whether the client will score a seven (therapist-directed phase), an eight (patient-directed phase) or a nine (tranisitional). Goal directed behaviour is another example of an item described in the levels of creative ability^8:202-203. All fifty-two items that make up the APOM have been described in these levels.

The data of 209 clients were collected with the APOM and then subjected to the Rasch Measurement Model to see if the scoring system fits the Rasch Model and could thus be converted from an ordinal to interval scaling, and thus conformed to the criteria for classification as a measure.

In the picture of a ruler, the top line represents an ordinal scale where the distances between the categories are not equal; 1,2, and 3 are lying at the same point as the 2 and 3 category of the bottom line which is an interval scale.

In Rasch terms, we talk about thresholds of categories which mean that there is a "distance" between categories and each category represents more of the trait than the previous category. Thresholds are ordered when each category represents more of the trait than the previous category. When converting ordinal to interval scaling, the "distance" in the ordinal scale is adjusted to fit the properties of an interval scale. This adjustment involves rescoring of items as necessary.

When I subjected the APOM with its 18 categories to the Rasch Measurement Model, there was a concern that the scale was too long. There is a general rule that the longer the scale the risk exists that it is not a measure but only a descriptive ordinal scale but nevertheless, we went ahead and prepared the data of the 209 subjects to verify the threshold ordering.

 

Figure 3

 

Figure 4 is immensely significant and depicts the threshold map which indicates that all the thresholds were ordered in the first round of the analysis. No rescoring was necessary, a result found to be unusual for a scale with 18 categories. The meaning of this map is that all the domains in the APOM were converted from ordinal scale level to interval level without any adjustment.

Without going into the technical explanation why this graph is something to marvel about, I realised that the level descriptors of the actions in the APOM were absolutely accurate. The level descriptors are based on the levels of Creative Ability and therefore I can state with confidence that the levels of Creative Ability exist and they are extremely valid for use as a measure. It is as if you are using a ruler to measure the levels of Creative Ability in a client. I still remember my admiration for Vona's work when I saw the results of the threshold ordering for the first time. It is not common that categories of a scale of this length perform so well in the Rasch Measurement Model. If only Vona could see these fantastic results! The conclusion to be drawn from this amazing result is that a scale based on sound theory, has the potential to be a true measure and is thus appropriate and valid for outcome measurement and analysis. When theory is sound, validation of measurement tools is easily demonstrated.

Stepping stone 5: objective observations vs subjective observations

Another steppingstone is whether the outcomes measure is objective or subjective. I come across many occupational therapists who find it difficult to detect the change in their clients because they seem uncertain about what to observe, at times resorting to paper and pencil tasks to assist their deductions. They prefer to give the client a questionnaire to complete. I do not agree with the use of routine questionnaires as they are dependent on the circumstances of the client at that point in time, for instance the clients need to satisfy the therapist or the emotional status of the clients are vulnerable. Based on my research I promote the use of observational measures for the following four reasons:

If you understand the measure, you know what to look for in the client.

If you know what to look for in the client you know your client better.

If you know your client better you know your interventions and techniques better.

If you know your interventions and techniques better you are a better OT.

I realise that many practitioners might argue that there is a place for client satisfaction questionnaires but my experience has shown that subjective measures receive a great deal of criticism from the funders of services, especially when the clinician is required to provide objective evidence of change after intervention.

Stepping stone 6: Routine outcome measurement

I am a huge advocate of having longitudinal outcomes measurements embedded routinely into the occupational therapy process. I found the benefits of expressing outcomes with numbers rather than describing it with words overwhelmingly positive. It was found to create a new occupational therapy language based on the observational scores. Clinicians start talking about levels and amounts of functional participation based on the outcome measure they are using. It cuts the waste in paperwork and getting to the point as all clinicians are focused on the specific domains to measure. It provides enormous opportunities to do reflective scientific research on the longitudinal data from records, e.g. correlation studies between outcomes and techniques applied. It gives clinicians confidence to speak up about the occupational needs of their clients in the multi-disciplinary team meetings as they know that they have measured the levels of function and they have evidence of change. It provides benchmarking for the effectiveness of occupational therapy practices as those settings using the same outcome measure may start comparing the changes in clients in specific domains and learn from each other. It furthermore provides a platform to study the effectiveness of new trends in occupational therapy for instance how many sessions are needed for maximum functional gain.

Stepping stone 7: clinical utility

Clinical utility is the last stepping stone but by no means the least important. Clinical utility means that the proposed outcome measure must be acceptable, appropriate, accessible and practical to the clinician in daily practice¹⁷. It is of utmost importance to test and retest a measure in the clinical setting to ensure that it is welcomed by occupational therapy clinicians as a useful companion in their busy practices. Clinicians must perceive it as helpful in analysing the problems at hand, clear in giving directions to explore, easy to apply, enriching their occupational therapy practice, and frankly something they cannot imagine doing without. If a measure achieves this then it would have achieved Occupational Therapy utility.

 

Conclusion

I have introduced you to three great role models of the past who had visionary thinking and their work is still influencing our practice models 50 years down the line.

Vona du Toit left a basic measurement principle of observations of actions through which to infer the motivation of the person to engage in activities and furthermore to think in levels (or amounts) of ability. I hope that I have inspired you with her great thinking and if you are using her model, continue doing so and do it with confidence and absolute conviction. If you are not using the Vona du Toit Model of Creative Ability, make sure that your outcome measure is a measure that provides you with valid results.

Georg Rasch gave us a way in which to measure change in spite of ordinal scales of measurement. This allows us to break the cycle between input and output and take the leap to measurement of change in the client. We can now move the focus back to the client.

Mary Reilly and her study of the human as an occupational being reminds us of the core of our profession, our unique contribution towards health and wellness. I am confident that occupational therapy in South Africa will strive to be the greatest idea of the 21^st century and not only in health care but all sectors that effect the occupational wellbeing of our clients.

I proposed seven stepping stones from input to outcomes and although I focused on the great work of Vona du Toit, these steps should be applicable to any occupational therapy setting and any theoretical framework. The challenge I put to you is to break the cycle between input and output and proceed to outcome measurement. This will make my dream of occupational therapists having basic evidence indicators of occupational performance at their fingertips, to use at any time any place and without much effort.

The way forward

It is basic science to measure outcomes and you might not share my enthusiasm and excitement about an ordinal scale that converts to an interval scale but this basic science gives us a foot in the door to ensure recognition of occupational therapy as a powerful profession and to achieve this. I firmly believe that we have to bombard the world with the change that meaningful occupational performance can bring about.

The success of solving a problem lies in the systematic approach to the problem. If only we can pull our expertise together in a meaningful strategy, we will be able to influence the occupational health of all nations.

My final words are a quote from Theodore Roosevelt:

"In any situation,
the best thing you can do is the right thing;
the next best thing you can do is the wrong thing;
the worst thing you can do is nothing."

Theodore Roosevelt

 

DEDICATION

I would like to dedicate this lecture to someone very special; someone who always believed in me and taught me about perseverance, dedication, how to make time for doing things you enjoy and a zest for life. This person is my mother and I am extremely fortunate to have her here with me at this important event in my professional life.

 

REFERENCES

1 Reilly M. Occupational Therapy can be one of the great ideas of the 20th century: The 1961 Eleanor Clarke Slagle Lecture. The American Journal of Occupational Therapy, 1962; 16(1): 87-105.         [ Links ]

2 Igiuku MN. Meaning-Making through Occupational Performance: Occupational Scientists'/Therapists' Role in Guiding Society towards Meaningfulness, Purposefulness, and Happiness. Keynote address at 33rd National OTASA Congress, Durban July 2012. http://www.otasacongress.co.za/images/stories/presentations/WED/Umdoni2/ 09h30%20ikiugu%20plenary%20wed.pdf        [ Links ]

3 Baum C. AOTA The American Occupational Therapy Inc. Mary Reilly: Remembering a True Legend. [Online].; Feb 2012. http://www.aota.or^News/AOTANews/Reilly.aspx (31 May 2012)        [ Links ]

4 Casteleijn D. The use of core concepts and terminology in South Africa. World Federation of Occupational Therapists Bulletin. 2012; 65: 20 - 27.         [ Links ]

5 Barry A. Learning from African communal values: The leadership challenge for Occupational Therapists in the 21st century. Plenary session at 33rd National OTASA Congress, Durban July 2012. http://www.otasacongress.co.za/images/stories/presentations/Thurs/ Umdoni2/08h30%20barry%20plenary%20thurs.pdf (8 Jul 2012)        [ Links ]

6 Du Toit V Patient Volition and Action in Occupational Therapy. (2nd ed). Pretoria: Vona & Marie du Toit Foundation, 1980.         [ Links ]

7 De Wit P Creative Ability: a model for psychosocial occupational therapy. In: Crouch RB & Alers VM (Ed). Occupational Therapy in Psychiatry and Mental Health (4th Ed). London: Whurr Publishers, 2005.         [ Links ]

8 Casteleijn JMF. Development of an outcome measure for occupational therapists in mental health care practice. University of Pretoria, South Africa. Unpublished doctoral thesis, 2010. Available from: http://upetd.up.ac.za/thesis/available/etd-02102011-143303/ (30 June 2012)        [ Links ]

9 Andrich D. Rasch Models for Measurement. Series: Quantitative Applications in the Social Sciences. London: Sage Publications, 1988.         [ Links ]

10 Bond TG, Fox CM. Applying the Rasch Model: Fundamental Measurement in the Human Sciences. 2nd Ed. Mahwah NJ: Lawrence Erlbaum Associates, Inc., Publishers, 2007.         [ Links ]

11 Molineux M. Occupation in Occupational Therapy: A Labour in Vain? In M. Molinuex (Ed), Occupation for Occupational Therapists. Oxford: Blackwell Publishing Ltd., 2004.         [ Links ]

12 Laver Fawcett A. Principles of assessment and outcome measurement for occupational therapists and physiotherapists: theory, skills and application. Hoboken, NJ: John Wiley & Sons, 2007.         [ Links ]

13 Talbot C. Theories of Performance: Organizational and Service Improvement in the public domain. Oxford: Oxford University Press, 2010.         [ Links ]

14 Stevens SS. On the Theory of Scales of Measurement. Science, 1946; June: 677-680.         [ Links ]

15 Michell J. Measurement in Psychology: A critical history of a methodological concept. Cambridge: Cambridge University Press, 2004.         [ Links ]

16 Casteleijn D, Graham M. Domains for occupational therapy outcomes in mental health practices. South African Journal of Occupational Therapy, 2012; 42: 26 - 34.         [ Links ]

17 Smart A. A multi-dimensional model of clinical utility. International Journal for Quality in Health Care, 2006; 18 (5): 377-382.         [ Links ]

 

 

Correspondence:
Daleen Casteleijn
Daleen.casteleijn@wits.ac.za