RESEARCH

 

The readability of informed consent forms for research studies conducted in South Africa

 

 

A E Fischer^I; W D F Venter^II; S Collins^III; M Carman^IV; S T Lalla-Edward^V

^IMPH; Ezintsha, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa
^IIPhD, FCP(SA); Ezintsha, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa
^IIIHIV i-Base, London, UK
^IVMA, PhD; Department of Philosophy, School of Social Sciences, University of the Witwatersrand, Johannesburg, South Africa
^VPhD; Ezintsha, Faculty of Health Sciences, University of the Witwatersrand, Johannesburg, South Africa

Correspondence

 

 

---

ABSTRACT

BACKGROUND: Informed consent forms (ICFs) are used to obtain consent from participants. However the complexity and comprehensiveness of these forms may not be appropriate. Readability can be quantified by formulas in Microsoft (MS) Word, such as the Flesch Reading Ease test. The South African (SA) ethics guidelines suggest that the MS Word Flesch-Kincaid Reading Grade score should be used to assess the complexity of ICFs and should be the equivalent of grade 8 level, or lower.
OBJECTIVES: To use readability formulas to determine whether current SA ICFs are appropriate for the general population.
METHODS: This was a descriptive study of a sample of English ICFs (solicited from our studies, as well as from local researchers) which received approval from local ethical review boards during the past 5 years, for prospective (>6 months) drug studies that explored treatment and prevention of HIV, tuberculosis, diabetes or cardiovascular disease. ICFs were evaluated in MS Word for Flesch Reading Ease and Flesch-Kincaid Reading Grade, with the Simple Measure of Gobbledygook (SMOG) index calculated using www.readabilityformulas.com. Recommended targets for easy readability are above 60 for the Flesch Reading Ease score, and less than or equal to a grade 8 reading level for the Flesch-Kincaid Reading Grade and SMOG.
RESULTS: A total of 75 consent forms from 35 individual research studies conducted in SA over the last 5 years were included. The consent forms had been approved by six ethics committees across seven of the SA provinces. The median (interquartile range (IQR)) Flesch Reading Ease score was 55.8 (48.7 - 59.7) and 18 (25.0%) of the ICFs had easy or standard readability, while the median (IQR) Flesch-Kincaid Grade was 10.2 (8.8 - 11.4), with 23 (30.6%) at least a grade 8 level or lower. The median (IQR) SMOG index was 9.8 (9.0 - 11.1) and 4 (5.3%) scored below grade 8 level.
CONCLUSIONS: Two-thirds of the ICFs from this study fail to meet the SA readability standard, a result matched by using alternative readability formulas. Readability can be improved with simple techniques and by actively monitoring readability metrics.

---

 

 

Voluntary informed consent is an ethical and legal requirement for participation in clinical research studies, and informed consent forms (ICFs) are required to share information with participants who may come from a wide variety of backgrounds. These ICFs must effectively communicate information including participant rights, potential risks and benefits, complex medical information, and comprehensive study procedures. The complexity of these ICFs is compounded by additional requirements from legal departments and funders to ensure that internal policies and requirements are met.^[1,2] Collectively, these components may lead to the creation of ICFs that certain populations may find difficult to comprehend.

The purpose of the ICF is to ensure that the potential research participant is given the essential information in a manner which is easy to comprehend so that their decision is a truly informed one.^[3] Aside from the ethical aspects, the value of ensuring proper understanding by a potential participant is the increased likelihood of their compliance with study procedures and retention in the study^[4] However, a study on the understanding of ICFs among low-income participants in the USA found that only 45% of participants read the entire consent form, and 27% admitted that they only understood the study 'a little'.^[5] Additional studies among different populations found that lengthy ICFs with medical jargon and complex concepts may take over an hour to fully read and comprehend.^[1,6] An accepted metric of evaluation for these ICFs, and other medical information documents, is readability, which can be quantified using a number of tests, such as the Flesch Reading Ease test.^[5] These readability tests have been globally applied to medical information documents and have constantly identified documents, including ICFs, that far exceeded local literacy levels.^[8-10]

In South Africa (SA), informed consent has guidance from the Constitution of SA,^[11] and the National Health Act 61 of 2003,^[12] as well as international documents like the Declaration of Helsinki.^[13] This has led to the creation of national guidelines to advise informed consent, which include Good Practice in the Conduct of Clinical Trials with Human Participants in South African Guidelines for Good Practice in Health Care Professions: Seeking Patients' Informed Consent^[15] and Ethics in Health Research: Principles, Structures and Processes^[16] Previous research has shown that these country-specific documents may not legally and ethically fulfil the complete requirements for informed consent;^[17] however, in SA, universities have human research ethics committees (HRECs) that provide another level of guidance. HRECs may provide researchers with specific guidelines and templates for ICFs as part of the management of the entire ethics approval process.^[18-20]

The adult literacy rate in SA increased from 91.9% in 2009 to 94.3% in 2017, while functional literacy (the ability to read at a grade 7 level) increased from 72.7% in 2002 to 86.3% in 2017, but despite the increases in literacy rates, there is still a significant proportion of the population who struggle with these forms. When adults with a level of education lower than grade 7 were asked about filling out forms, 24% stated that they had difficulties doing so, while 36% were unable to fill out the forms at all.^[21]Ethics in Health Research: Principles, Structures and Processes^[16] is the only national document that directly speaks to readability, and states that readability levels should be appropriate to the participants' level of understanding, and suggests a complexity level of no more than grade 8. No readability studies have been conducted on the ICFs in SA. The objective of this study was to analyse the readability of ICFs used to obtain voluntary consent in SA research studies to determine whether the ICFs, as written, are appropriate for the general population.

 

Methods

Study design

This was a descriptive study in which parametric and non-parametric measures of central tendency and variability were used to describe the various algorithms used to evaluate ICFs. ICFs were included if they were English, had been approved by an SA HREC in the past 5 years, and were investigator-led prospective medium- to long-term (>6 months) drug studies that explored treatment and prevention of HIV, tuberculosis (TB), diabetes or cardiovascular disease.

Recruitment

Participating ICFs were mined from a variety of sources from May 2019 until August 2019. A search of PubMed and Google Scholar was done to identify studies that met the abovementioned criteria; then the corresponding authors were invited via email to participate, by sharing their ICFs. Research and academic institutions were also contacted to invite researchers to participate. The email invitation included a line encouraging the researcher to forward this request to participate to their colleagues who may have also conducted eligible studies (email included in supplementary information).

Data collection

Consenting researchers were asked to provide a Microsoft (MS) Word document of their ICF for analysis. Where only PDF versions were available, it was exported as an MS Word document, then manually checked to ensure that all punctuation and formatting remained correct.

Readability metrics

The Flesch Reading Ease test incorporates average sentence length and average word length in syllables into a formula to compute reading ease, where the higher scores are easiest to read^[20] Scores from 70 to 100 are considered easy to read (appropriate for 4th to 6th grade), while scores from 60 to 70 are considered standard (7th or 8th grade) and scores below 60 are considered hard (high school or college). A variation of the Flesch Reading Ease test is the Flesch-Kincaid Reading Grade, which uses the same variables, but weights them differently in order to provide a reading grade that directly parallels US school grades. Although developed for the American population, the Flesch Reading Ease test is one of the most accurate readability measures. It is commonly used globally and appropriate for our study^[1,9,10] Similarly, the Simple Measure of Gobbledygook (SMOG) index also provides a score that parallels years of education and has been proven highly effective for healthcare applications, including ICFs (Fig. 1).^[23,24]

 

 

Data extraction

A pre-defined data collection tool was created for data extraction of descriptive ICF characteristics (ICF type, HREC, study location, study type, disease of study, sample size, date, font type, font size and page length), variables associated with readability (word count, sentence count, sentence length, word length, total syllables, syllables per word and number of words with three or more syllables) and readability metrics (Flesch Reading Ease test, Flesch-Kincaid Reading Grade and SMOG index). Most ICF characteristics were collected by reading the documents, while readability variables and readability metrics were recorded from the MS Word 'Tools' toolbar, the online resource for readability metrics (www.readabilityformulas.com) (total syllables, word >3 syllables and SMOG index) or calculated (syllables per word and percentage of words with >3 syllables) from these collected variables.

In some instances, the word count in MS Word was not the same as the word count from the online resource because of certain formulas grading formatted symbols like bullet points or underlined areas for signatures differently. In these instances, the MS Word word count was taken as true. The online readability resource could only process documents up to 3 000 words, so large documents were processed as 3 000-word sections, then proportionally averaged back together based on the weights of each section. Syllables per word and words with 3 or more syllables were extracted from the online resource, but their respective average or percentage were calculated against the true MS Word word count, as the online calculator did not give results accurate to one decimal, and they were calculated with the internal word counter.

Data analysis

Extracted data were entered into MS Excel (Microsoft Corp., USA) for analysis. Data from ICF characteristic variables were counted, and then presented as frequencies and percentages. Median and interquartile ranges (IQRs) were calculated for all readability variables and the readability test scores. Means were not calculated to avoid presenting results skewed by a 20-page ICF. For each readability test, the scores were categorised as easy, medium and hard, and each of these categories was presented as a frequency and percentage.

Patient and public involvement

A member of a local institutional review board read and commented on the manuscript for tone and messaging. Aside from this, no patients or public were involved in the study design, or in the recruitment to, and conduct of, the study.

 

Results

ICF characteristics

There was a total of 75 ICFs, from 35 individual studies, with many of them including secondary consent forms. The most common secondary consent form was for bio-storage (9; 12.2%), followed by under-18 assent (5; 6.8%), parental consent (4; 5.4%), sub-studies (4; 5.4%), sampling (3; 4.1%), pregnant participants (2; 2.7%) and a variety of others. The consent forms were from six ethics committees across SA, with University of the Witwatersrand and University of Cape Town being the highest, with 22 (62.9%) and 6 (17.1%), respectively. Seven provinces were represented, with 20 (57.1%) from Gauteng Province and 7 (20.0%) from Western Cape Province being the highest. General research studies were the most prevalent type of study, with 13 (37.1%), followed by randomised controlled studies (9; 25.7%). Twenty-two studies (62.9%) focused on HIV, 7 (20.0%) on TB, 7 (20.0%) on non-communicable diseases and 6 (17.1%) on maternal health. The remainder focused on other health areas. Eleven studies (31.4%) had a sample size of 101 - 500, while 8 (22.9%) were below 100 and 6 (17.1%) were above 500; 10 (28.6%) did not provide sample size. The most common fonts were Calibri and Arial, 15 (42.9%) and 12 (34.3%), respectively, while the most common font sizes were 11 and 12, both appearing 15 (42.9%) times.

ICF readability variables

The median (IQR) ICF length was 5 pages (4 - 7), 1 608 words (1 121 - 2 298), and 69 sentences (48 - 100). For sentence length, median (IQR) values were 19 words per sentence (17 - 20), and for word length, 4.7 characters/word (4.5 - 4.8). The median (IQR) ICF total number of syllables was 2 620 (1 739 - 3 653), with a median of 1.6 syllables/word (1.5 - 1.6). The median (IQR) number of words with 3 or more syllables was 248 (163 - 325), with a median percentage for the total words of 14.3% (13.3 - 16.8%) (Table 1).

 

 

ICF readability metrics

The Flesch Reading Ease scores identified that the median (IQR) score was 55.8 (48.7 - 59.7), while 1 (1.3%) of the 75 ICFs had easy readability and 17 (22.7%) had standard readability. The median (IQR) Flesch-Kincaid Grade level was 10.2 (8.8 - 11.4), with 23 (30.6%) at a grade 8 level or lower. The median (IQR) SMOG index was 9.8 (9.0 - 11.1), while only 4 (5.3%) were at or below a grade 8 level (Tables 1 and 2).

 

 

Discussion

This study has determined that over two-thirds of the ICFs analysed were hard to read and required an average reading comprehension level equivalent to at least a US grade 10. This exceeds the national functional literacy level of grade 7,^[21] as well as the recommended grade 8 level as outlined in Ethics in Health Research.^[16]These findings are in line with similar studies from different regions, such as a 2017 study in the UK that identified the Flesch Reading Ease score of a standardised orthopaedic procedure consent form to be 55.6, which may impede a significant percentage of patients from providing informed consent.^[2] Another study revealed that patient information leaflets in the UK required graduate level reading ability, which is too high for most HIV-positive individuals to effectively provide informed consent.^[10]

Despite these readability and literacy disparities, research studies continue to enrol study participants, which raises the concern of whether participants' consent is sufficiently informed. Concerns about participant understanding of ICFs, independent of readability considerations, have been documented since as early as 1981.^[25] Since then, there is very little evidence to show that efforts have been made to improve the understanding of ICFs; instead, they are actually increasing in complexity.^[26,27] In 2005 the HIV Prevention and Trials Network proposed a framework for enhancing informed consent, highlighting the importance of writing at no more than a grade 8 reading level; however, in the sub-Saharan African context, comprehension can be further compromised by low literacy rates, and vocabulary and translation difficulties.^[28]

For research studies that target the general population in SA, providing consent may prove challenging for all individuals that read below the grade 10 level, including the 14% of the population who are not functionally literate (below grade 7). This challenge is exacerbated in ageing populations who are increasingly required to participate in non-communicable disease research.^[21] In order to remove this readability barrier from ICFs, the Flesch-Kincaid Reading Grade and SMOG index should target grade 8 or less for the general SA population, while the Flesch Reading Ease test should target scores above 70 for the general population. This is evident as general interest magazines and tabloids have Flesch Reading Ease scores of 80 and 95, respectively, while intellectual newspapers like The New York Times and The Wall Street Journal have scores in the 40s.^[10]

The two main variables that impact readability are the number of words per sentence and the number of syllables per word, with long sentences and words containing three or more syllables resulting in poor readability scores.^[10] By utilising the free and accessible readability tests online and in MS Word, researchers are able to evaluate and modify the readability of their ICFs to ensure that they are appropriate for their target population. This could remove barriers to entry for participants with lower reading comprehension levels, creating a more complete sample, while also ensuring that participants are aware of the potential benefits and risks of participation. Additional resources available to assist researchers in reaching their readability goals include the Program for Readability in Science and Medicine (PRISM) Readability Toolkit which guides researchers through the principles of readability, how to determine readability, a quick-guide to improving readability (including an editing checklist), samples of easily readable ICF excerpts and examples of plain language replacements for complex medical jargon.^[7] Equipped with this knowledge, researchers should be able to improve readability by keeping sentences short (below 15 words) and by ensuring that their ICFs consist of mostly one- and two-syllable words.

Study strengths and limitations

To our knowledge this is the first report of ICF readability in sub-Saharan Africa. The method of recruitment may have led to a sampling bias, as researchers may have declined to participate based on the complexity of their ICFs, underestimating readability Network- and consortium-managed studies had strict access and sharing policies, which decreased the number of ICFs and studies that could be included. The readability formulas did not take into account other factors that may affect readability such as font, paragraph spacing and content.

 

Conclusions

The SA publication Ethics in Health Research^[16] clearly outlines that the Flesch-Kincaid Reading Grade tool should be used to assess the complexity of ICFs and that the targeted complexity should be no more than a grade 8 level equivalent. Two-thirds of the ICFs included in this study exceed these recommendations. However by monitoring readability metrics and employing simple techniques to increase readability, national targets could be met. Furthermore, these targets could be monitored by HRECs to ensure that these readability barriers are decreased.

Declaration. The University of the Witwatersrand Human Research Ethics Committee approved this evaluation (ref. no. H19/03/05).

Acknowledgements. The authors would like to thank the researchers who volunteered their ICFs for inclusion in this analysis.

Author contributions. Designed the study: AEF, WDFV, SC, STL-E. Analysed the data and interpreted results: AEF, STL-E. Wrote the initial draft: AEF. STL-E. All authors critically reviewed and approved of the final draft.

Funding. None.

Conflicts of interest. Some of the included ICFs were from studies in which AEF, WDFV and STL-E were part of the study team. All authors have no further competing interests to declare.

 

References

1. Pandiya A. Readability and comprehensibility of informed consent forms for clinical trials. Perspecl Clin Res 2010;1(3):98-100.         [ Links ]

2. Sivanadarajah N, El-Daly I, Mamarelis G, et al. Informed consent and the readability of the written consent form. RCS Annals 2017;99(8):645-649. https://doi.org/10.1308%2Frcsann.2017.0188        [ Links ]

3. Nijhawan LP, Janodia M, Muddukrishna B, et al. Informed consent. Issues and challenges. J Adv Pharm TechnolRes 2013;4(3):134-140. https://doi.org/10.4103%2F2231-4040.116779        [ Links ]

4. Kadam RA. Informed consent process. A step further towards making it meaningful! Perspect Clin Res 2017;8(3):107-112. https://doi.org/10.4103%2Fpicr.PICR_147_16        [ Links ]

5. Ittenbach R Senft E, Huang G, et al Readability and understanding of informed consent among participants with low incomes. A preliminary report. J Empir Res Hum Res Ethics 2015;10(5):444-448. https://doi.org/10.1177/1556264615615006        [ Links ]

6. Massett H, Miskin G, Paul R et al. Readability and compliance of NCI's informed consent documents (ICDs) with its revised ICD template. J Clin Oncol 2016;34(15):e18162-e18162. https://doi.org/10.1200/JCO.2016.34.15_suppl.e18162        [ Links ]

7. Ridpath JR Greene SM, Wiese CJ. PRISM Readability Toolkit. 3rd ed. Seattle. Group Health Research Institute, 2007.         [ Links ]

8. Paasche-Orlow M, Taylor H, Brancati F, et al. Readability standards for informed-consent forms as compared with actual readability. N Engl J Med 2003;348:721-726. https://doi.org/10.1056/nejmsa021212        [ Links ]

9. Wen G, Liu X, Huang L, et al Readability and content assessment of informed consent forms for Phase II-IV clinical trials in China. PLoS ONE 2016;11(10):e0164251. https://doi.org/10.1371/journalpone.0164251        [ Links ]

10. Collins S, Martin L, Wilhams M, et al Patient information leaflets (PILs) currently require graduate level literacy levels. Equivalent to The Guardian or The Telegraph newspapers. BHIVA Conference 2015, Brighton.         [ Links ]

11. South Africa. Constitution of the Republic of South Africa Act No. 108 of 1996.         [ Links ]

12. South Africa. National Health Act No. 61 of 2003.         [ Links ]

13. World Medical Association. World Medical Association Declaration of Helsinki, 64th WMA General Assembly, Fortaleza, Brazil, October 2013. JAMA 2013;310(20):2191-2194.         [ Links ]

14. National Department of Health. Guidelines for Good Practice in the Conduct of Clinical Trials with Human Participants in South Africa. Pretoria. NDoH, 2006.         [ Links ]

15. Health Professions Council of South Africa. Guidelines for Good Practice in Health Care Professions. Seeking Patients' Informed Consent. The Ethical Considerations. Booklet 9. Pretoria. HPCSA, 2008.         [ Links ]

16. National Department of Health. Ethics in Health Research. Principles, Processes and Structures. Pretoria. NDoH, 2015.         [ Links ]

17. Britz R, Íe Roux-Kemp A. Voluntary informed consent and good clinical practice for clinical research in South Africa! Ethical and legal perspectives. S Afr Med J 2012;102(9):746-748. https://doi.org/10.7196/samj.5498        [ Links ]

18. Human Research Ethics Committee, University of Cape Town. Informed Consent Standard Operating Procedure. Cape Town. UCT, 2016.         [ Links ]

19. Human Research Ethics Committee, University of the Witwatersrand. HREC Application Form (Medical) - 2018 - version 8. Johannesburg. University of the Witwatersrand, 2018.         [ Links ]

20. Human Research Ethics Committee, University of the Witwatersrand. Template for University of the Witwatersrand Human Research Ethics Committee Informed Consent. Johannesburg University of the Witwatersrand, 2018.         [ Links ]

21. Statistics South Africa. General Household Survey, 2017. Pretoria. Stats SA, 2018.         [ Links ]

22. Flesch R. A new readability yardstick J Applied Psychology 1948;32(3):221-233. https://psycnet.apa.org/doi/10.1037/h0057532        [ Links ]

23. Wang L, Miller M, Schmitt M, Wen F. Assessing readability formula differences with written health information materials. Application, results, and recommendations. Res Social Adm Pharm 2013;9(5):503-516. https://doi.org/10.1016/j.sapharm.2012.05.009        [ Links ]

24. McLaughlin G. SMOG Grading. A new readability formula. Journal of Reading 1969;12(8):639-646.         [ Links ]

25. Howard J, DeMets D. How informed is informed consent? The BHAT experience. Control Clin Trials 1981;2(4):287-303. https://doi.org/10.1016/0197-2456(81)90019-2        [ Links ]

26. Albala I, Doyle M, Appelbaum P. The evolution of consent forms for research. A quarter century of changes. IRB. Ethics & Human Research 2010;32(3):7-11.         [ Links ]

27. Beardsley E, Jefford M, Mileshkin L, et al. Longer consent forms for clinical trials compromise patient understanding So why are they lengthening? J Clin Oncol 2007;25(9):e13-e14. https://doi.org/10.1200/jco.2006.10.3341        [ Links ]

28. Woodsong C, Karim QA. A model designed to enhance informed consent. Experiences from the HIV Prevention Trials Network. Am J Public Health 2005;95:412-419. https://doi.org/10.2105%2FAJPH.2004.041624        [ Links ]

 

 

Correspondence:
A E Fischer
fischermhealth@gmail.com

Accepted 28 July 2020