SCIENTIFIC ARTICLES

 

Practice-based evidence: Evaluating the quality of occupational therapy patient records as evidence for practice

 

 

Helen Buchanan^I; Jennifer Jelsma^II; Nandi Siegfried^III

^IBSc (OccTher), MSc (OccTher), PhD (OccTher) Cape Town- Senior Lecturer, Division of Occupational Therapy, Department of Health and Rehabilitation Sciences, University of Cape Town
^IIBSc (Physio), MPhil PG Dip (Adult Ed), PG Dip (Research Ethics), PhD- Professor, Division of Physiotherapy, Department of Health & Rehabilitation Sciences, University of Cape Town
^IIIMBChB MPH (Hons) Cape Town, FCPHM (SA) DPhil Oxon- Independent Clinical Epidemiologist, Cape Town

Correspondence

 

 

---

ABSTRACT

BACKGROUND: Occupational therapy patient records are required for legal purposes, but may also be used to produce evidence for practice. Our aim was to establish how comprehensively occupational therapists documented patient records.
METHODOLOGY: We conducted a descriptive cross-sectional study of occupational therapists at public health facilities in a South African province. Trained raters audited five randomly-drawn records per participant using a checklist developed for the study. The maximum possible score was nine and the lowest was zero. Audits were checked for consistency.
RESULTS: Forty-nine occupational therapists participated and 240 records were audited. Records contained information on intervention (96%) and changes occurring at impairment (82%) and activity and participation levels (64%). Documentation of baseline assessment (impairment level: 20%; activity and participation level: 10.4%) and re-assessment (impairment level: 7%; activity and participation level: 0.0%) was limited. Audit scores were significantly better in the work practice area (H=16.10, p=0.003) and among therapists in urban areas (U=24.50, p<0.001).There was a significant negative correlation between audit score and number of clients seen per month (r_s=-0.46, p<0.001).
CONCLUSION: The low audit scores suggest that the records did not contain sufficient information to produce robust evidence. Manageable ways of documenting occupational therapy practice need to be devised.

Keywords: audit, documentation, evidence-based practice, occupational therapy, patient records, practice-based evidence, quality of records

---

 

 

INTRODUCTION

Clear and accurate documentation is essential to build a body of evidence that is based on practice. One of the main responsibilities of health care practitioners stipulated in the Health Professions Council of South Africa (HPCSA) ethical rules is to "keep accurate patient records"^l:20. A patient record, as defined in the HPCSA Guidelines, is "any relevant record made by a health care practitioner at the time of or subsequent to a consultation and / or examination or the application of health management"². The 2012 National Core Standards (NCS) for Health Establishments in South Africa gives greater clarity in defining patient records as "A collection of documents that provides an account of each episode in which a client visited or sought treatment and received care or referral from a health care facility"³. Within occupational therapy a patient record "means information, however recorded (e.g. written, electronically recorded/entered, audio, video, photographs, diskette), generated (in the case of an occupational therapy record) by the occupational therapist or an individual supervised by the occupational therapist, pertaining to occupational therapy services provided by the occupational therapist"^4:17. Patient records document "what has happened to the client in a chronological sequence", show clinical reasoning, keep team members informed, and "demonstrate the effectiveness of occupational therapy"^5:xv. They can therefore be used to monitor patient progress (or lack thereof) provided there is a "clear, simple and coherent documentation of the process and outcome of assessment and intervention procedures"^6:434. Patient notes are legal documents^2,3,7 and an important source for assessing the quality of patient care through clinical audit³.

It is argued that documentation of therapists' actions and their outcomes is essential for building professional experience as written notes may facilitate more accurate reflection on practice. Accurate documentation of routine practice also enables a profession to build a bank of quality data that can be used to create evidence to underpin practice⁸. While there is some similarity in the processes therapists use to collect evidence (information) about the best possible intervention for a client, and that of researchers, the focus of the former is "improving intervention for a particular client" while the latter is more concerned with "adding knowledge to the body of evidence available for planning assessments and conducting interventions with future clients"^6:434. The process of generating evidence from everyday practice has been coined practice-based evidence by Margison et al. within the discipline of psychotherapy⁸. We propose that practice-based evidence (evidence that can be generated through practice) is as important as evidence-based practice (implementing evidence in practice). Occupational therapists thus need to be afforded the time to document their practice mindfully so that the information recorded may be used for research purposes to create the evidence required for practice. This argument has been affirmed by Sherwood^9:2 who advocated that "evidence is best generated in relation to your own specific clients, services and interventions".

Occupational therapy records may be read and used by a variety of audiences including team members, clients and family members, facility managers and administrators. A United States survey of occupational therapists found that 64% of the respondents felt that evidence should be communicated in documentation especially where funding was concerned and when it was a departmental requirement^l0. It is therefore important that records are professional and of a satisfactory standard⁵. This implies that they should be accurate and complete and provide a clear description of what has occurred in occupational therapy⁵. Although occupational therapists regard record keeping as important, it can cause uncertainty and frustration. Participating occupational therapists in a Swedish study experienced several professional dilemmas related to their documentation practices which influenced what they recorded in their patient notes¹¹. Dilemmas included using everyday language as opposed to professional language, highly structured notes versus providing a comprehensive picture of the patient, juridical and ethical demands compared to environmental conditions, varying expectations of those accessing the notes, and conflicts between occupational therapy reasoning and medical reasoning. Factors within the workplace, such as time, routines, and tasks expected by other professionals, also affected their ability to document properly¹¹.

Unlike countries such as Canada and the United States where documentation guidelines have been developed^4,12, there are no specific guidelines for occupational therapy documentation in South Africa. The HPCSA Guidelines on patient records² which apply to all health professions under their auspices, state that records should be complete, concise, consistent, and use a standardised format². They explain a "standardised format" as notes containing "the history, physical findings, investigations, diagnosis, treatment and outcome" in that order². Provision of such a structure enables the information to be assimilated and summarised. It is evident from the language used that the criteria have been devised with medical doctors in mind, but modifications are possible to make the criteria suitable for occupational therapy as practiced in clinical settings. According to the HPCSA Guidelines therefore, occupational therapy records should include: the patients' personal particulars and bio-psychosocial history, the time, date and place of each occupational therapy encounter, assessment of the patient's condition with the findings, the proposed intervention, referral details if relevant, the patient's response to treatment, information on absence from work with reasons, and written proof of informed consent where applicable. This aligns closely with Tickle-Degnen's recommendations⁶⁴³⁵ that occupational therapy records include "descriptive attributes about the client and the context of the assessment and intervention, the client's responses during the assessment, and intervention effectiveness for the client". Occupational therapy guidelines for documentation guide therapists in what to include in their patient records in addition to the fundamental aspects required (such as the patient's name, date of contact, and the therapists name and signature)^4,12.

The implementation of the NCS for Health Facilities in South Africa which "assist in setting the benchmark of quality care against which delivery of services can be monitored"³⁸ form part of the South African Department of Health's initiatives to improve the quality of health care. Domain 6 (Operational Management) within the NCS includes a section on medical records with one of the performance standards being that "Patient information is accurately and completely recorded according to clinical, legal and ethical requirements"³. Compliance in this regard is measured with an audit checklist. Audits of patient records provide basic data about decisions and intervention^13,14. To conduct an audit, specific criteria based on clearly-defined standards need to be identified. These should relate to the focus of the audit and are used to measure performance¹⁵. Changes may be implemented after an audit, and practice re-evaluated at a specified time to establish whether there has been an improvement in quality¹⁵. This statement is supported by a Swedish study that identified a great need for quality improvements in occupational therapy documentation and advocated for additional "education and training supporting efficient documentation routines"¹⁶⁷⁹.

The purpose of this study was to examine the extent to which occupational therapy records could be used to generate evidence for practice. The aim was to explore how comprehensively occupational therapists at public health facilities in a South African province completed their patient records. The specific objectives were: 1) to establish, using an International Classification of Functioning, Disability and Health (ICF) Framework¹⁷, which aspects were routinely assessed and recorded; and 2) to explore the factors that were associated with satisfactory completion of patient records.

 

METHODS

Study design

The study used a descriptive, cross-sectional design. Cross-sectional studies collect data "from a sample of individuals (or groups) at a particular point in time as a basis for inferring the characteristics of the population from which the sample comes."^18:53 The study was descriptive as it set out to describe a particular phenomenon (the type of information captured in occupational therapy patient records) in a specific sample¹⁸.

Population and sampling

The population included occupational therapists employed by the Western Cape Department of Health (DOH). Refer to Table I for details.

 

 

Occupational therapists carrying a patient load who worked at least 20 hours per week were eligible to participate. For practical reasons, therapists working more than 115 kilometres from Cape Town were excluded. Those planning to take leave during the study, or who were leaving the employ of the DOH before the end of the study, were also excluded due to the impact on the outcome data. Recruitment occurred initially through the Metropole Occupational Therapy in Health (MOTH) Forum^a. In addition, three information sessions were held and therapists who could not attend were contacted by a research co-ordinator who explained the purpose and details of the study. Occupational therapy managers were asked to recruit staff in their departments and acted as the liaison for the principal investigator and research coordinator who checked that participants met the eligibility criteria. All occupational therapists that met the criteria and signed consent were included.

Development of the audit instrument

As the focus of our study was on measuring the extent to which participants completed their patient records rather than evaluating the quality of their practice, an audit was the most appropriate choice of method²⁰. Audit instruments apply specific best practice criteria to current practice to improve the quality of healthcare^15,21.

Although incomplete documentation and missing data were a concern, the intention was not to capture the participants' reasoning processes but rather to develop a profile of their record-keeping.

A systematic review to identify suitable data collection instru-ments²² revealed three: a chart-stimulated recall evaluation form for assessing competence in occupational therapists²³, a chart audit form measuring the use of outcome measures²⁴ and an audit tool for evaluating the quality of occupational therapy records in a province in South Africa²⁵. As none of these instruments were suitable in their entirety for the purpose of the study, an audit checklist was developed based on these instruments. Items were selected if they contributed to answering the question: "What would one expect to see in patient notes aimed at creating evidence for practice?"

Development of the grading rubric

A grading rubric (available from the first author on request) with descriptors for each item was developed to guide scoring²⁶. This was refined and finalised during the piloting process by discussing the results for each pilot test with the raters, identifying the reasons for discrepancies between scores and revising the rubric accordingly. Further information on the development and scoring of the rubric is provided in the data collection section.

Initial validation

The initial audit instrument containing six dichotomous scale (yes/ no) items was sent to an expert panel to review its face validity and content validity. The panel consisted of two international colleagues with audit experience and two local experts. Feedback from the panel led to the inclusion of the ICF²⁷ as a framework to differentiate between assessment, intervention and re-assessment at impairment, activity and participation levels, in order to gauge the focus of interventions. The clinical utility of this revised 15-item checklist (with a four-point rating scale) was pre-tested by the principal investigator who audited 10 patient records from one of the facilities from which study participants would be recruited. This resulted in the refinement of the audit criteria and the inclusion of a not applicable response option.

Recruitment and training of raters

Two occupational therapists were employed and trained to conduct the audits. Training entailed explaining each audit checklist item, clarifying terminology²⁷, discussing scoring with the rating scales and using the checklist on patient records. One record was audited jointly by the principal investigator and both raters, and three records were subsequently rated with increasing levels of independence to address inter-rater reliability (IRR). After each audit, scores were discussed and reasons for differences identified.

Pilot testing and further validation

After training was complete, four pilots were conducted with a set of 10 records each to test the IRR and refine the 15-item checklist. Audits were conducted independently by the two raters with no discussion permitted. As the first four pilots revealed poor IRR for several items, with no consistency in how each item performed, the rating categories were collapsed to form a dichotomous scale. Re-analysis of the data did not however improve its reliability, therefore the four-point scale was retained. Examination of the item scores across the four audits revealed that the disagreements occurred in the items referring to activity and participation. These two levels were therefore combined. In addition, item 4 (Recording of baseline assessments) was redundant as recording of a baseline assessment had already been scored in a previous item.

The revised 10-item checklist (four-point rating scale) was tested in a further two pilots but the IRR deteriorated further with six or more items having less than satisfactory values. Feedback from the two international panel members resulted in the rating scale being converted to a dichotomous scale (A. McCluskey, PhD, email communication, 31 July 2008), with more specific descriptors containing only one criterion being developed (M. Law, PhD, email communication, 25 August 2008). For example, to score yes for item 3, a goal had to be documented but did not have to be measurable. This revised checklist was tested again and Cohen's Kappa was computed with "stata statistical software, release 10. Landis and Koch's benchmarks for Kappa (Poor: <0.21; Fair: 0.21-0.40; Moderate: 0.41-0.60; Substantial: 0.61-0.80; Almost perfect: >0.81)²⁸ were used to judge the strength of agreement between raters. One item (Recording of intervention at impairment level) had poor IRR (kappa=-0.l54, p=0.70l) and was combined with another item (Recording of intervention at activity or participation level). The final audit checklist thus consisted of nine items (see Table II) each of which showed at least moderate agreement (kappa>0.60) (see Table III).

 

 

 

 

Data collection procedure

Data collection commenced once participants had provided informed consent. Participants submitted a list of patients seen in the specified week and a random sample of five records was drawn from each list using the ROUND RAND command in Microsoft Excel. Participants were contacted by email or telephone to inform them of the records to be audited. If five or fewer patients were seen, all records on the list were audited. Arrangements were made to visit each facility and participants were requested to have the selected records available. Audits were conducted at participants' places of work, with the exception of one municipality where participants worked in different health facilities spanning a large geographical area. In this case, participants (n=5) provided photocopies of their patient records, removed identifying features to ensure anonymity, and either left them with the occupational therapy manager for collection by the principal investigator, or faxed them to the principal investigator.

On arrival at the audit venue, the principal investigator checked that all selected records were available. In a few cases, patient lists were not received before the audit and random selection had to be done on a laptop at the audit venue. The therapist was then advised of the records that were needed. Records that were unavailable on the day of the audit were photocopied by the participant and faxed to the principal investigator later. To ensure confidentiality, all identifying information was deleted from faxed records and only initials were used to ensure that the record appeared on the randomised selection list.

Records were audited with the validated audit checklist by one of the two research assistants trained during the piloting phase. The grading rubric was used to guide scoring. The principal investigator checked all audits to verify that raters applied the criteria in the grading rubric to allocate scores. Items in the audit checklist were given a score of 0 (not evident in the record), 1 (item evident in the record) or not applicable. Items were deemed not applicable when it was not appropriate to include them. For example, if the client had not yet been discharged a final assessment would not have been expected. The maximum score was nine, unless items were rated as not applicable in which case the total possible score was calculated by subtracting the number of not applicable items from nine. Discrepancies in scoring between the rater and principal investigator were discussed until consensus was reached.

Data management

Data were entered into Microsoft Excel by one of the research assistants and double checked for accuracy by the principal investigator. Data were captured after each facility audit. Participant's study numbers were used during data entry to ensure confidentiality. Data were stored securely throughout the study.

Data analysis

Data were exported into, and analysed with, STATISTICA 12. Measures of central tendency and dispersion were determined for all descriptive variables. Categories were combined if there were few responses. This was done for work setting (both were combined with urban as the therapist was likely to have contact with other colleagues) and qualification level (all postgraduate qualifications were combined into a single category). Practice areas were collapsed into five groups: mental health, neuromusculoskeletal, paediatrics, work and combination. Audit scores for each record were calculated by adding the total number of yes responses and converting them to percentages. Median audit scores were used since the scores were ordinal. Ranges were calculated for the total number of records audited per participant. To identify trends in the contents of patient records, frequencies and proportions were computed for each audit item across all records (n=240).

As data were not normally distributed, non-parametric statistics were used throughout.The Kruskal-Wallis H-test or Mann-Whitney U-Test was conducted to determine whether there were significant differences between descriptive variables and median audit scores. Mann-Whitney tests whether there is a relationship between two groups when one is dichotomous and the other is at least ordinal, while Kruskal-Wallis tests whether there is a difference in the medians between three or more groups²⁹. To discover the factors that impacted on the quality of patient records (audit score), Spearman's rank order correlation was computed to determine whether there was a correlation between audit score and numerical variables (age, years of experience and number of clients per month). Two-sided significance tests were used throughout.

Ethical considerations

Ethical approval was obtained from the University of Cape Town Health Sciences Faculty Human Research Ethics Committee (REC REF: 259/2006) and the Western Cape Provincial Department of Health (Ref. I9/I8/RP37/2008). Letters were sent to medical superintendents or senior managers at the facilities in which occupational therapists agreed to participate to inform them about the study and obtain their support. Informed consent was obtained from all participants prior to the start of the study. Participants were allocated study numbers to ensure anonymity. As existing records were used for the audit, patient care was not compromised. Patient privacy was protected by using numbers rather than names on audit checklists. Audit checklists were numbered prior to data collection to ensure patient anonymity. Although the research assistants conducting the audit would not usually have had access to patient records, the use of numbers rather than names ensured that confidentiality was maintained as far as possible. The audit should not, therefore, have exposed patients to additional harm.

 

RESULTS

Of the 89 potential participants, 12 did not meet the inclusion criteria, thus 77 were eligible. Forty-nine agreed to participate (63.6%) and 28 declined due to time pressures (n=16), lack of interest (n=3) and failure to submit a consent form (n=2) (see Table IV). Seven participants did not disclose their reasons for declining. In total, 240 records were audited at 15 sites. Most participants submitted five records for audit (46/49, 93.9%). Three had fewer than five records audited, because they had either not seen five different patients in the selected week or the record was unavailable^b at the time of the audit. Although it was possible that participants could have made changes to their records prior to the audit, there was no evidence of this.

 

 

Demographic and practice characteristics

A comparison of the number of participants in the sample and population per facility type is available in Table V to enable a judgment to be made about the representation of the sample. Less than 50% of the population of occupational therapists in district health and specialised services participated.

 

 

Participants' median age was 28.0 (Range: 22.0-50.0) years*"" and median experience was 5.0 (Range: 0.5-31.0) years. Most participants were female (95.9%) and had only an undergraduate qualification (87.8%). Slightly less than half worked in tertiary facilities (40.8%) with most participants being based in urban areas (87.8%) (refer to Table VI).

 

 

Audit of patient records

The overall performance per audit item is shown in Figure 1. Most records documented information about the intervention provided and changes in the patient at impairment and/or activity and participation levels. Few records included baseline assessment or re-assessment data, occupational therapy goals or mention of an evidence-base for the intervention.

 

 

The median overall audit percentage score was 40.0% (Range: 13.7-76.0%). Median audit percentages for qualification level, setting, level of care and practice area are shown in Table VII. There were wide ranges in scores across all variables. Audit scores were lowest in rural areas, at primary level and for participants working in a combination of areas. In terms of practice area, work obtained the highest median score.

 

 

There was a significant negative correlation between audit score and clients seen per month (r_s=-0.46, p<0.00l). Correlations between audit score, years of experience and age were not significant. The difference in audit score for practice areas was significant (H=I6.I0, p=0.003) with work scoring significantly higher (median=60.0, range: 56.0-76.0) than the other areas. Audit scores were also significantly higher for participants in urban (median: 42.9, range: 22.7 - 76.0) compared to rural (median: 22.9, range: 13.7 - 35.4) settings (U=24.50, p<0.00l). There were no significant differences in audit scores for qualification or level of care. Results for statistical tests are displayed in Table VIII. Significant p-values are indicated in bold.

 

 

DISCUSSION

The low median audit scores reflect the generally poor quality of the content of the occupational therapy records audited in this study.

This not only limits the possibilities for creating evidence from the data available in patient records, it may have implications for the quality of care. Zegers et al³⁰ found an association between poor quality information in hospital patient records and higher rates of adverse events, and concluded that the quality of information may be a predictor of the quality of care. Incomplete documentation has been reported in other occupational therapy studies. For example, an audit of 46 occupational therapy stroke records at an academic hospital in South Africa revealed that all records were incomplete, leading the researchers' to express concern about the potential implications for patients, the hospital and the profession³¹. Similar limitations have been reported for occupational therapy records in schools for learners with special needs in the Western Cape of South Africa³². In the latter study, most information was recorded less than 50% of the time. In light of Tickle-Degnen's claim that "frequent and prospective documentation can aid session-by-session decision-making by demonstrating client response and change in a clear and concrete manner"⁶⁴³⁵, the high number of incomplete records in the current study raises questions about the possible impact on occupational therapy practice.

Participants in the current study recorded information about patient intervention (95.8%) and monitored changes in the patient at impairment (81.5%) and/or activity and participation (63.7%) levels. The inadequate reporting of baseline assessments (impairment level: 20.4%; activity and participation level: 10.4%), reassessments (impairment level: 7.1%; activity and participation level: 0.0%), occupational therapy goals (34.6%) and evidence for the intervention (0.4%) echoes the findings of a South African study which identified limitations in documenting assessment information (reported in 15% of records), treatment plans (11%), treatment sessions (28%), and discharge information (13%)³². A second South African occupational therapy study found that only 9% of stroke records contained information about patient improvement (or lack thereof), and that evaluation and treatment was least documented (only 35% included an occupational therapy plan)³¹. A Swedish study of community-based occupational therapists similarly revealed sparse recording of assessment (2%), treatment plans (2%), goals (6%), follow up (7%) and evaluation (1%) relating to prescribing technical aids¹⁶.

The sparse documentation of the evidence-base for intervention in the current study was not surprising considering that few clinicians in the Department of Health have access to evidence sources^33,34. Davis et al¹⁰ found that participants with access to evidence sources at work were more likely to cite evidence in their documentation (r=0.305, p>0.0l). They further identified barriers to citing evidence in documentation, namely time limitations at work (55.6%), patient quota requirements (l8.3%), not understanding the research (l2.7%) and not knowing how to apply the outcomes of research to clients (12.7%)¹⁰. It seems that including evidence in patient records may be more necessary under particular circumstances. For example, an American study found that although most occupational therapy respondents (63.5%) felt evidence should be included in documentation, it was considered most important to include when funding was at stake or when dictated by their department^l0. Thus, it may not be necessary or feasible for evidence to be cited in all occupational therapy records. The absence of a standard or model to guide practitioners in how to communicate evidence in clinical documentation is likely to be a strong factor influencing the dearth of recording such information¹⁰.

The practice of incomplete documentation is concerning considering that information on assessment, intervention (including goals), the patient's response to intervention and the outcome of the intervention are regarded as essential elements of occupational therapy patient documents^4,6,12. Furthermore, the paucity in documenting baseline assessment and re-assessment information has implications for determining the outcomes of occupational therapy, and thus for establishing its evidence-base. While recording of assessment information is important, the type of assessments used need to be considered carefully so that relevant data are collected. To generate high quality evidence for occupational therapy practice, assessments must be standardised and suitable for measuring occupational performance outcomes⁹. Failure to collect and document all necessary information will result in patient records that are not useful for creating evidence. In support of this statement, an analysis of the electronic records of mental health occupational therapists in the United Kingdom revealed few records with completed follow-up assessments and concluded that the possibilities for demonstrating changes in clinical outcomes across services was thus limited³⁵.

The focus on impairment as opposed to activity and participation in the current study is not really surprising considering the dominant influence of the medical model within many Department of Health facilities in South Africa due to the quadruple burden of disease. It is, however, disturbing considering that occupational therapy is "concerned with promoting health and well being through occupa-tion"³⁶, and that "The primary goal of occupational therapy is to enable people to participate in the activities of everyday life"³⁶. The limited recording of activity and participation in the current study echoes that of a Swedish study that similarly found inadequate documentation of occupational performance (activity and participation) (7%, n=l82) in occupational therapy records¹⁶. This may be due to occupational therapists adjusting their documentation by using medical language to be more respected in the multi-disciplinary team, and omitting the language of occupation and occupational performance^ll. In contrast to this, a second Swedish study found that although few occupational therapy records were complete, the majority captured activity and participation³⁷. A United Kingdom review of the occupational therapy service within a National Health Service Trust likewise found that records focussed on activities of daily living³⁸ (an aspect of activity and participation). The poor representation of the main focus of occupational therapy in the records audited in the current study may have potentially serious consequences for the development of the profession and thus requires further investigation.

Possible reasons for incomplete occupational therapy patient records have been discussed in previous studies. Rischmuller and Franzsen³² attributed the gaps in occupational therapy records to several factors, including the roles and expectations of occupational therapists, lack of accountability and the absence of clear guidelines³². As in the study by Lundgren Pierre and Sonn¹¹, it is possible that the HPCSA requirement for records to be as short and concise as possible caused difficulties for the participants who, in adhering to this requirement, omitted information that is important in occupational therapy. It is also possible that the underground nature of occupational therapy practice (the parts of practice that are not documented but are highly valued and form an integral part of the clinical reasoning process) may have resulted in the study participants not documenting all they knew about a patient³⁹. Occupational therapists seem to consider more than purely legal and ethical aspects when rating their satisfaction with their patient records. For example, Lundgren Pierre found that even when the occupational therapists followed the occupational therapy process to document their practice, some were dissatisfied with their records because they were "uncertain about putting words to or naming some aspects of the professional assessments, the small things going on all day long, intertwined in legitimate tools and well-defined professional tasks, although they valued them highly"^40:l74. Lastly, due to the short-term nature of many of the facilities where participants in the current study were employed, priority may have been given to interventions to enable patients to be independent at home once discharged which may have prevented participants from conducting formal baseline assessments and re-assessments. Thus, documenting all the aspects contained in the audit checklist may not have been feasible in these settings, however desirable they may be.

Exploration of the factors that influenced the quality of the records in the current study showed that practice area, place of work (urban/rural) and the number of clients seen per month played a role. The fact that records in the area of work scored significantly higher than those in other areas (H=16.10, p=0.003) is probably because these records are often used for medico-legal purposes, and are therefore open to closer scrutiny. One explanation for therapists in urban settings obtaining significantly higher median audit scores than their counterparts in rural settings, may be due to the former having greater access to resources in terms of academic literature as well as supervision and support structures. The finding that the level of care had no bearing on the quality of records supports a previous study that likewise found no relationship between these variables²⁵. We hypothesised that therapists with a postgraduate qualification may be more inclined to document their practice in more detail. This was however not the case as the median audit scores were similar (postgraduate: 43.4%; undergraduate 40.0%) and the results of the Mann-Whitney Test were not significant (U=l02.00; p=0.429). This may have been due to the small sample size for postgraduate qualifications (n=6). It may also be that postgraduate coursework programmes do not include documentation as part of the curriculum.

Responsibility for ensuring that the content of records reflects the core values of occupational therapy rests with the profession. It is encouraging to note therefore, that some occupational therapy departments have developed their own documents outlining the domains of occupational therapy and that these concepts are reflected in their documentation (E. Williams, email communication, 23 September 2015). In the current context of quality improvement initiatives within the Department of Health, occupational therapists will need to be increasingly accountable for what they document in patient records. As part of the NCS, medical records, including those of occupational therapy, will be audited. Although the audit checklist that will be used focusses primarily on services provided by doctors and nurses, much of the general information is appropriate for occupational therapy records. The quality assurance programme implemented by the Gauteng Department of Health includes an audit tool for evaluating the standard of record keeping by allied health professions⁴¹. The tool is framed within the ICF and the items reflect the type of information outlined in occupational therapy documentation guidelines developed elsewhere^4,l2. This holds great promise for gathering data that can be used to create evidence that is context-specific and relevant for occupational therapy practice in South Africa.

 

STRENGTHS AND LIMITATIONS OF THE STUDY

This is the first study to document the content of occupational therapy records in South Africa from the perspective of practice-based evidence. It provides an overall picture of the content of occupational therapy records and gives some direction as to what the profession should consider in terms of documentation guidelines. Strategies that strengthened the quality of the data included pilot testing of the audit instrument until adequate IRR was achieved, training of the raters, double-checking audit ratings and data entry for accuracy, and use of objective criteria to evaluate records and guide raters (grading rubric).

A limitation is the inclusion of participants employed at Department of Health facilities in one province of South Africa and the under-representation of participants from district health and specialised services. The findings can therefore not be generalised to all occupational therapists employed by the Department of Health nor can they be applied to occupational therapists in other sectors, such as private practice, education or non-profit organisations.

 

IMPLICATIONS AND RECOMMENDATIONS

Several implications for practice, education and research emerge from this study.

Practice

The overall poor quality of the records audited in this study has serious ethical and legal implications. It also impacts the occupational therapy contribution within the multi-disciplinary team. The fact that the quality of the records decreased as patient numbers increased points to the need for more effective human resource planning to provide sufficient time for accurate and thorough patient record keeping. Of concern is the lack of congruence between the information documented in the records, and the core philosophy of occupational therapy. Failure to document practice that reflects the unique contribution of occupational therapy could affect funding of occupational therapy services.

The occupational therapy profession is missing an opportunity to use data from patient records as evidence for practice. It would be valuable for the profession to devise a guideline for documentation that reflects the core values of the profession, and to develop criteria for information that should be included routinely in patient records for adoption across sectors. This would need to be reflected in the record keeping audit tools used across the country. Moving from written records to electronic records may facilitate easier retrieval of patient information for synthesis and analysis purposes.

Education

The high rate of incomplete documentation points to a need for strengthening education around documentation at both undergraduate and postgraduate levels. Undergraduate students need to understand the record keeping requirements set out in the HPCSA ethical rules and be given sufficient opportunities to develop basic competence in writing patient records. In-service training covering good documentation practices should be routinely included as part of continuous professional development programmes to improve the quality of occupational therapy patient records.

Research

Further investigation should be undertaken to determine the underlying reasons for poor record keeping and to understand how and why therapists choose to document what they do. Viable ways for documenting occupational therapy practice should be developed. This should ideally be a partnership between clinicians and academics so that a practical and realistic method is devised that has potential for use in research to support practice and develop the profession.

 

CONCLUSION

The current study reiterates Rischmuller and Franzsen's³² conclusion that occupational therapists are not meeting the requirements for record keeping laid down by the HPCSA. It is also clear that an evidence-based approach to occupational therapy documentation is currently not in use, and as a result the data cannot be used to produce evidence. The generally poor quality of occupational therapy progress notes is concerning in light of ethical standards and the imperative for occupational therapy to demonstrate its effectiveness. The lack of emphasis on occupational performance as opposed to performance components may reflect a failure of occupational therapists to adopt a client-centred approach that focuses on patient needs in everyday life. There is an urgent need to raise awareness about the importance of accurate and complete documentation and to devise manageable ways for occupational therapists to document their practice consistently and in line with occupational therapy core values.

 

ACKNOWLEDGEMENTS

Sincere appreciation is extended to the study participants and to Drs Mary Law and Annie McCluskey who generously provided expert guidance.

This research was funded by the National Research Foundation (TTK200604 I9000I8) and the Faculty of Health Sciences University Research Committee, University of Cape Town.

 

REFERENCES

1. Health Professions Council of South Africa. Guidelines of good practice in the health care professions. Ethical and professional rules of the Health Professions Council of South Africa as promulgated in Government Gazette R717/2006 Pretoria: Health Professions Council of South Africa, 2008. <http://www.hpcsa.co.za/Uploads/editor/UserFiles/downloads/conduct_ethics/rules/generic_ethi-al_rules/booklet_2_generic_ethical_rules_with_anexures.pdf> (14 April 2015)        [ Links ]

2. Health Professions Council of South Africa. Guidelines on the keeping of patient records. Booklet l4. Pretoria: Health Professions Council of South Africa, 2008. (14 April 2015)        [ Links ]

3. Republic of South Africa Department of Health. Towards quality care for patients. National Core Standards for Health establishments in South Africa. Tshwane: Department of Health, 2012.         [ Links ]

4. College of Occupational Therapists of Ontario. Standards for record keeping<http://www.coto.org/pdf/COTO_Standards_Record-Keeping_2008.pdf>: 2008. (6 Oct 2015).         [ Links ]

5. Sames K. Documenting occupational therapy practice. 2nd ed. Boston: Pearson, 2010.         [ Links ]

6. Tickle-Degnen L. Monitoring and documenting evidence during assessment and intervention. American Journal of Occupational Therapy, 2000; 54: 434-6.         [ Links ]

7. Meriano C and Latella D. Occupational therapy interventions. Function and occupations. Thorofare: SLACK, 2007.         [ Links ]

8. Margison F McGrath G, Barkham M, Clark J, Audin K, Connell J and Evans C. Measurement and psychotherapy: Evidence-based practice and practice-based evidence. British Journal of Psychiatry, 2000; 177: 123-30.         [ Links ]

9. Sherwood W. Good intentions - POTS keynote address. Focus, 2011; 2: 1-5.         [ Links ]

10. Davis J, Zayat E, Urton M, Belgum A and Hill M. Communicating evidence in clinical documentation. Australian Journal of Occupational Therapy. 2008; 55: 249-255.         [ Links ]

11. Lundgren Pierre B and Sonn U. Occupational therapy as documented in patients' records. Part II. What is proper documentation? Contradictions and aspects of concern from the perspective of OTs. Scandinavian Journal of Occupational Therapy, 1999; 6: 3-10.         [ Links ]

12. Clark G and Youngstrom M. Guidelines for documentation of occupational therapy. American Journal of Occupational Therapy, 2008; 62: 684-90.         [ Links ]

13. Jennett P Scott S, Atkinson M, Crutcher R, Hogan D, Elford R MacCannell K and Baumber J. Patient charts and physician office management decisions: chart audit and chart stimulated recall [Abstract]. Journal of Continuing Education in the Health Professions, 1995; 15: 31-9.         [ Links ]

14. Jennett P and Affleck L. Chart audit and chart-stimulated recall as methods of needs assessment in continuing professional health education [Abstract]. Journal of Continuing Education in the HealthProfessions, 1998; 18: 163-71.         [ Links ]

15. Sasaru S, Sheward Y and Sasaru S. Audit in allied health professional practice. In: Clouston T and Westcott L, (eds.). Working in health and social care. An introduction for allied health professionals. Edinburgh: Elsevier Churchill Livingstone, 2005, p. 145-60.         [ Links ]

16. Hedberg-Kristensson and Iwarsson S. Documentation quality in occupational therapy patient records: focusing on the technical aid prescription process. Scandinavian journal of Occupational Therapy, 2003; 10: 72-80.         [ Links ]

17. World Health Organization. Towards a common language for functioning, disability and health. International classification of functioning, disability and health. Geneva: World Health Organization, 2002.         [ Links ]

18. Bynner J. Cross-sectional survey. The SAGE Dictionary of Social Research Methods. SAGE Publications, Ltd. London, England: SAGE Publications, Ltd, p.53-5.         [ Links ]

19. MOTH Forum. Draft Constitution. Metro Occupational Therapy in Health Forum, Cape Town, 2008.         [ Links ]

20. Goulet F, Jacques A, Gagnon R, Bourbeau D, Laberge D, Melanson J, Ménard C, Racette P and Rivest R. Performance assessment. Family physicians in Montreal meet the mark! Canadian Journal of Family Physicians, 2002; 48: 1337-44.         [ Links ]

21. Bowling A. Research methods in health. Investigating health and health services. 3rd ed. Berkshire: Open University Press, 2009.         [ Links ]

22. Buchanan H. Evidence-based practice in occupational therapy in South Africa and the Western Cape. Department of Health and Rehabilitation Sciences. Cape Town: University of Cape Town, 2011.         [ Links ]

23. Salvatori P and Ward M. Development of a tool to measure clinical competence in occupational therapy: a pilot study. Canadian Journal of Occupational Therapy, 2000; 67: 51-60.         [ Links ]

24. MacDermid J, Solomon P Law M, Russell D and Stratford P Defining the effect and mediators of two knowledge translation strategies designed to alter knowledge, intent and clinical utilization of rehabilitation outcome measures: a study protocol. Implementation Science 1: 14, 2006. doi: 10.1186/1748-5908-1-14 (29 November 2006).         [ Links ]

25. Foote H, Lamont S, Burger E and Leishman A. The introduction of a quality assurance programme in Gauteng Health hospital occupational therapy services. South African Journal of Occupational Therapy, 2006; 36: 6-10.         [ Links ]

26. Hosking J, Newhouse M, Bagniewska A and Hawkins B. Data collection and transcription. Controlled Clinical Trials. 1995; 16: 66S-103S.         [ Links ]

27. World Health Organization. ICF checklist, version 2.Ia, Clinician form for International Classification of Functioning, Disability and Health<http://e-bility.gr/eutexnos/Includes/icf-checklist.pdf>: 2001. (25 Sept 2003).         [ Links ]

28. Landis J and Koch G. The measurement of observer agreement for categorical data. Biometrics, 1977; 33: 159-74.         [ Links ]

29. Plichta SB and Garzon LS. Statistics for nursing and allied health. Philadelphia: Wolters Kluwer/Lippincott Williams & Wilkins Health, 2009.         [ Links ]

30. Zegers M, de Bruijne M, Spreeuwenberg P Wagner C, Groenewegen P and van der Wal G. Quality of patient record keeping: an indicator of the quality of care? BMJ Quality and Safety, 2011; 20: 314-8.         [ Links ]

31. Mlambo T, Amosun S and Concha M. Assessing the quality of occupational therapy records on stroke patients at one academic hospital in South Africa. South African Journal of Occupational Therapy, 2004; 34: 10-3.         [ Links ]

32. Rischmüller R and Franzsen D. Assessment of record keeping at schools for learners with special educational needs in the Western Cape. South African Journal of Occupational Therapy, 2012; 42: 13-20.         [ Links ]

33. Joubert R. Evidence-based practice: a critique based on occupational therapy within the SA context. South African Journal of Occupational Therapy, 2005; 35: 8-13.         [ Links ]

34. Duncan M and Alsop A. Practice and service learning in context. In: Lorenzo T, Duncan M, Buchanan H and Alsop A, (eds.). Practice and service learning in occupational therapy. Enhancing potential in context. Chichester: Wiley, 2006: 7019. 7-19.         [ Links ]

35. Morley M. Evidencing what works: are occupational therapists using clinical information effectively? British Journal of Occupational Therapy, 2014; 77: 601-4.         [ Links ]

36. World Federation of Occupational Therapists. "Statement on occupational therapy." 2010. <http://www.wfot.org/Portals/0/PDF/STATEMENT%20ON%20OCCUPATIONAL% 20THERAPY%20300811.pdf> (6 October 2015)        [ Links ]

37. Backman A, Kawe K and Bjorklund A. Relevance and focal view point in occupational therapists' documentation in patient case records. Scandinavian Journal of Occupational Therapy, 2008; 15: 212-20.         [ Links ]

38. Welch A and Forster S. A clinical audit of the outcome of occupational therapy assessment and negotiated patient goals in the acute setting. British Journal of Occupational Therapy, 2003; 66: 363-8.         [ Links ]

39. Mattingly C and Fleming M. Clinical reasoning: Forms of inquiry in a therapeutic practice. Philadelphia: FA Davis Company, 1994.         [ Links ]

40. Lundgren Pierre B. Occupational therapy as documented in patient's records - Part III. Valued but not documented. Underground practice in the context of professional written communication. Scandinavian Journal of Occupational Therapy, 2001; 8: 174-83.         [ Links ]

41. Gauteng Department of Health. Allied Health Professionals Standards and Audit Tools (Hospital). Gauteng: Department of Health, 2016.         [ Links ]

 

 

Correspondence:
Helen Buchanan
Helen.Buchahan@uct.ac.za

 

 

a MOTH Forum meetings are held monthly and attended by a representative, usually the head of department, from each occupational therapy department or service. The purpose of the Forum is 'to enable and facilitate collaborative efforts, in striving for an occupational therapy service of excellence, within the public sector'.
b Records were in the doctor's office or being used by other members of the multi-disciplinary team at the time of the audit.