ORIGINAL RESEARCH

 

The medial tibial stress syndrome score: item generation for a new patient reported outcome measure

 

 

M Winters^I; M Franklyn^II; MH Moen^{III, IV}; A Weir^V; FJG Backx^VI; EWP Bakker^II

^IMSc; University Medical Centre Utrecht, Rehabilitation, Nursing Science and Sports Department, Utrecht, The Netherlands
^IIPhD; Department of Mechanical Engineering, The University of Melbourne, Melbourne, Australia
^IIIMD, PhD; Bergman Clinics, Naarden, The Netherlands
^IVMD, PhD; The Sportsphysician Group, OLVG West, Amsterdam, The Netherlands
^VMBBS, PhD; Aspetar Orthopaedic and Sports Medicine Hospital, Doha, Qatar
^VIMD, PhD; Division of Clinical Methods and Public Health, Academic Medical

Correspondence

 

 

---

ABSTRACT

BACKGROUND: There is no valid and reliable instrument that evaluates injury severity and treatment effects for medial tibial stress syndrome (MTSS) patients.
OBJECTIVE: The aim was to generate items for the MTSS score, a new patient-reported outcome measure for patients with MTSS.
METHODS: The authors consulted experts in the fi eld of MTSS to generate items that measure the severity of MTSS and to reach consensus on the relevance of items for the MTSS score. This research consisted of a pilot study and two Delphi rounds. The Delphi approach entails the consultation of experts about a topic for which no evidence is available during which consensus is sought on this topic. Additionally, 20 MTSS patients appraised the MTSS score on readability and comprehension.
RESULTS: Nineteen experts consented to participate, 13 of whom reached consensus. Generated items address the following domains: 'limitation in sporting activities', 'pain while performing sporting activities', 'pain while performing activities of daily living' and 'pain at rest. Patients with MTSS confirmed the good readability and comprehension of the items.
CONCLUSION: This study supports the importance of items in the aforementioned domains while evaluating treatment effects in atients with MTSS.

Keywords: shin splints, item generation, Delphi technique, cross-cultural translation

---

 

 

Medial tibial stress syndrome (MTSS) is one of the most common lower leg injuries in athletes and military personnel.^[1,2] It is an overuse injury with pain along the distal medial border of the tibia that is thought to be due to overloading of the bone.^[3] A recent systematic review highlighted a lack of good studies on the treatment of MTSS.^[4] One commonly used definition for MTSS is provided by Yates and White:^[5] 'the presence of exercise-induced pain along the posteromedial border of the tibia over five or more consecutive centimetres that is elicited by palpation'.^[5] In previous research, numerous outcome variables have been used to assess treatment effects on MTSS patients; e.g. visual analogue scales, global perceived effect scales, and time to recovery.^[4] Over the past two decades, the opinion of the patient has received increasing attention when determining treatment effects in clinical trials and practice. Hence, the use of Patient Reported Outcome Measures (PROMs) has been recommended to quantify the effect of interventions in randomised controlled trials and clinical settings.^[6] A recent systematic review on MTSS showed there is a need for a standardised outcome measure as no validated outcome measures have yet been developed.^[4] Item generation is the first step in creating a new PROM. Therefore the aim of this study was to generate items for a new PROM for MTSS patients and have these items' relevance and comprehension subsequently appraised by patients with MTSS. This PROM should evaluate severity and treatment effects, and also incorporate the perception of the patient.

 

Methods and materials

The authors used a Delphi consensus study to combine expert opinions and reach consensus. A Delphi approach entails the consultation of experts about a topic for which no evidence is available. These experts are blind to the other experts involved in the study; thus their opinion are not influenced by other expert opinions. In a Delphi study a consensus of opinion is sought from those regarded as experts in their fields. These expert opinions are solicited "blind".^[7-9] For this study the authors received permission from the local medical ethics committees of the provinces of Utrecht (12-542/C) and Zuid-Holland (12-092).

Identifying and inviting MTSS experts

The authors aimed to include experts in the field of MTSS who were currently actively involved in MTSS research and who also had clinical experience with MTSS patients. Therefore they firstly identified experts in the field of MTSS by contacting national sports medicine associations, (the American College of Sports Medicine, the American Orthopaedic Society for Sports Medicine, the Australasian College of Sports Physicians, the British Association of Sport and Exercise Medicine, the Canadian Academy of Sport and Exercise Medicine, the Danish Association of Sports Medicine, German Federation for Sports Medicine and Prevention, and the Swedish Society of Exercise and Sports Medicine) and requested they provide the contact information of their key experts in the field of MTSS. In addition, those who had published studies in the field of MTSS were contacted. Based on their network of clinical experts, the authors also approached a number of people in the Netherlands. All experts were invited to participate by email.

Delphi study

A pilot study among the experts in the authors' own network (N=9) was conducted prior to starting the study in order to generate preliminary items. This network consisted of sports physicians and sports physiotherapists in the field of MTSS with whom collaboration had taken place in previous research projects in The Netherlands. In the first round of the Delphi study, all experts were requested to comment on the preliminary items and asked to suggest new items.

In consecutive rounds, these new items were included. These experts were asked to indicate their level of agreement with regard to the inclusion of the preliminary items in the MTSS score on a five-point scale: strongly disagree, disagree, no opinion, agree, strongly agree. They were also requested to suggest additional items. Consensus was reached upon an item when 67% of the experts voted for its inclusion or exclusion.^[6] The Delphi study was completed when consensus was reached upon all items and no further items were proposed. No maximum number of rounds was set. After consensus was reached, all items were translated into Dutch by a native Dutch speaker with a medical background who was also proficient in English.

Appraisal by patients

A sports medicine physician diagnosed MTSS if exercise-induced pain along the posteromedial border of the tibia was elicited by palpation on the posteromedial border of the tibia over a length of five or more consecutive centimeters.^[5] Patients were eligible for participation when they were > 16 years of age and had had symptoms for > three weeks. When focal tibial pain, indicative of a stress fracture, or a medical history with a cruris fracture was present, subjects were excluded.^[10]

After item generation, the patients appraised the items in two rounds. In the first round, the authors asked 15 MTSS patients to provide feedback on readability and comprehension using a semi-structured interview. They subsequently modified the items according to their feedback. In the second round, an additional five patients with MTSS were requested to appraise the items.

Cross-cultural translation

All the generated items for the MTSS score were translated from Dutch to English. Steps One to Four from the cross-cultural validation process, as described by Sousa and Rojjanasrirat^[11] and Beaton et al.^[12] were performed. The translation contained forward and back-translations. A steering committee, in which the translators and all authors (except MF), were represented, reviewed both the forward and back-translations and decided on the final English version. The decision making process was based on consensus, which was reached when 67% of the committee members present agreed. In case consensus could not be reached for all items, the authors planned to have them translated again using different translators. ^[11,12]

 

Results

Twenty-one international and eleven Dutch experts were invited to participate, 19 of whom consented to participate: four Americans, four Australians, one Canadian, nine Dutch and one from England. There were eight sports physiotherapists, six sports physicians, one podiatrist, one surgeon in sports medicine, one podiatric surgeon, one exercise and sports specialist and one biomedical engineer. Figure 1 is the study's flow diagram. One expert withdrew his participation during the pilot study and five were lost to follow-up during the second round of the Delphi study. Those experts (N=13: 8 Dutch, 3 Australian, 2 American) with whom consensus was reached are named in the Acknowledgements section, except for one expert (MF) who co-wrote this manuscript. The supplementary online material presents all the items generated.

 

 

Pilot study

The pilot study included 16 items on the limitations of activities in daily life (ADL) (N=10) and sporting activities (N=6). These items were scored from 0 to 4, with 0 indicating 'no problem' to 4 indicating an 'extreme problem. The remaining items: 3, 5, 9, 10, 11, 13 and 15 were developed during the pilot study.

Round 1

In Round 1, the main feedback provided was that there were too many items related specifically to running and sporting activities. Furthermore, participants proposed that each outcome should have a descriptor, which was accordingly included for all items. Items on sprinting, uphill running, and sudden accelerations and decelerations when running were removed as suggested by the majority of the experts, as these items were irrelevant to MTSS patients that do not usually run. Items 1, 2, 7, 8, 12 and 14 were produced in Round 1 (see Appendices 1 and 2 for items in Dutch and English).

Round 2

In this round, consensus was reached on all but two items. One item was proposed in the second round but did not reach the pre-validation stage. This item looked at provoking pain during hopping. This item was considered irrelevant for the study's objective as most patients do not usually hop. Two items (4 and 6) were suggested in Round 2; however, these items were not included in the additional round of the Delphi study. Items 4 and 6 both cover pain during sporting activities. Table 1 provides an overview of the final level of consensus reached for each item. Consensus was reached on all items formulated in English.

Contribution of experts who stopped participating

During the study, five ofthe 19 experts did not respond to the authors' attempts to seek contact and contributions from the project. The expert who stopped participating during the pilot study suggested, together with other experts, to use an item on the current content of sporting activity (Item 3). In the second round of the Delphi study, five experts discontinued responding to the authors' emails. The first of these experts stated that the questionnaire was complete in the first round and therefore did not respond to the authors' further emails. The second expert suggested including items on the current content of sporting activities (Item 3) and current sporting activity (Item 1). One expert suggested including an item on night pain (Item 14), and on pain experienced after sporting activities (Item 8). The importance of the latter was supported by one of the other experts who also withdrew their participation. The fifth expert suggested including an item that differentiated between the various types of pathophysiology (e.g. stress fracture, compartment syndrome, MTSS) of shin pain. However, it was decided not to include this item in the Delphi study as it discriminates between types of lower leg pain instead of the severity of perceived complaints.

Appraisal by patients (Figure 1)

Fourteen patients (seven women and seven men) commented on the newly developed MTSS score. They completed the questionnaire and provided feedback on the questionnaire's readability, comprehension and ease of use. The first concept of the questionnaire was shaped according to the example of the VISA-A questionnaire with a guide to continue or skip to the next item. To continue or to skip an item depended on whether the patient was still involved in their usual sporting activity, was involved in alternative sporting activities only, or was not involved in any sporting activity at all.'¹³¹ However, some of the patients did not understand this structure. Therefore the preliminary MTSS score was modified so that every patient had to complete all the items. Item 15 was not well understood. This item was aimed at the measurement of pain on touch. It started with descriptors for three different degrees of touch followed by statements of when pain was induced at touch. This was changed by using the various degrees of touch (e.g. bumping, pressing, rubbing) in the response options. Other patients' suggestions concerned alternative words for pain. Changes were made based on the feedback provided. In addition, six patients (three women and three men) provided comments on the updated MTSS score. No further comments were made and the MTSS score was considered ready for validation.

The MTSS score

The MTSS score consists of 15 items: current sporting activities, current amount of sporting activities, current content of sporting activities, pain while performing sporting activities, time to onset of pain during sporting activities, pain throughout sporting activities (Item 6 of the total set, see Appendices 1 and 2), pain throughout sporting activities (Item 7 of the total set, see Appendices 1 and 2), pain after sporting activities, pain while standing, pain while walking, pain while walking up or down stairs, pain while performing common daily activities, pain at rest, pain at night and pain to touch (Table 1).

Cross-cultural translation

The MTSS score was translated from Dutch to English according to the appropriate guidelines.^[1U2] All minor discrepancies between translators of the forward and back-translations were resolved at consensus meetings. The forward and back-translations of the MTSS score were critically reviewed by a steering committee comprising of all authors and translators. The back-translation highlighted a few minor discrepancies between the forward translation and the original version: "I feel ..." instead of "I have ..." (Items 9, 10 and 11). Other discrepancies were seen in Item 5 where 'sporting activities' was included in the response options. All discrepancies were resolved so that the English version was a correct cross-cultural translation of the original Dutch version.

 

Discussion

This study provides expert-generated and patient-appraised items for a new patient reported outcome measure for MTSS. Consensus was reached on all generated items that were included during the Delphi study. Items generated relate to limitation in sporting activities, pain while performing sporting activities, pain while performing activities of daily living and pain at rest. Patients appraised the generated items as to their ease of understanding and relevance to the injury. In previous research, a great variety of pain scales were used and definitions of when patients have recovered differ greatly between studies. This hampers comparison of results across studies. This present study aimed to generate items for a new standardised instrument to evaluate treatment effects in MTSS patients. Furthermore, the MTSS score meets the need for an instrument that evaluates effects and incorporates the perceptions of the patient.

The MTSS score was developed using the Delphi technique, a widely used method to reach consensus among experts in fields for which no evidence is available. One of the most important advantages is that experts are unaware as to who their co-participants are. Therefore the experts opinions are free from the influence of other panel members.^[7-9] The strong aspects in this Delphi study include the size of the expert panel and the wide variety of experts with different backgrounds. In addition, the items have been appraised by a total of 20 patients with MTSS, in two rounds. Although five experts stopped participating during course of the Delphi study, all experts contributed to the development of the MTSS score's items. Furthermore, the quality of the contributions were considered as more important than the quantity of the contributions.

There were also some limitations in the current study. Consensus was not sought on two items (Items 4 and 6). These items were proposed in Round 2. As five experts did not respond to the authors' emails after Round 1, there was concern that more experts may drop out in additional rounds, thus leaving little or no consensus on the items. These two items were appraised by the authors' group and were found to be useful. The content validity for Items 4 and 6 is acknowledged and therefore less supported by expert consultation. The authors are confident that their decision to not seek consensus on these two items enabled a broad consensus on all other items. In the Delphi method, there is no widely accepted threshold for when consensus among experts is met. Previous reports suggested using thresholds between 50% and 70%.^[8,9] In this research project, it was decided to set the threshold at 67%; however, there was >75% agreement for all but one item.^[7] A report on the validation study, in which items for the MTSS score were selected and its reliability, validity and responsiveness is assessed elsewhere.^[14]

 

Conclusion

This study reports on the item generation process for the MTSS score, a new patient-reported outcome measure for patients with MTSS. The results support the importance of items in the domains of pain, limitations in activities of daily living and sporting activities while measuring the severity of MTSS from the patient's perspective. The items generated in this study cover all these domains.

Acknowledgements: The authors are very grateful for the valuable contributions of the expert panel members. The experts that participated in the final round of the Delphi study were: Carl Barten, BPt; Steef Bredeweg, MD, PhD; Fred Hartgens, MD, PhD; Shannon Munteanu, PhD; Carol Otis, PhD; Rodney Pope, BPt, PhD; Steven Rayer, MSPT; Michiel Schipper, MSPT; Victor Steeneken, MSc; Jack Taunton, MD, PhD; Joost Vollaard, MSPT; Wessel Zimmermann, MD. We thank Sophia Stone and Jesse Tjebbes (forward-) and Elmar Kal, PhD, and Jacintha Kal (back-) for their valuable cross-cultural translations.

Competing interests: No competing interests to be declared.

Funding: None.

 

References

1. Clanton TO, Solcher BW. Chronic leg pain in the athlete. Clin Sports Med 1994 Oct;13(4):743-759. [http://dx.doi.org/10.1177/1941738111426115]        [ Links ]

2. Taunton JE, Ryan MB, Clement DB, et al. A retrospective case-control analysis of 2002 running injuries. Br J Sports Med 2002;36(2):95-101. [http://dx.doi.org/10.1136/bjsm.36.2.95]        [ Links ]

3. Moen MH, Tol JL, Weir A, et al. Medial tibial stress syndrome: A critical review. Sports Med 2009;39(7):523-546. [http://dx.doi.org/10.2165/00007256-200939070-00002]        [ Links ]

4. Winters M, Eskes M, Weir A, et al. Treatment of medial tibial stress syndrome: a systematic review. Sports Med 2013;43(12):1315-1333. [http://dx.doi.org/10.1007/s40279-013-0087-0]        [ Links ]

5. Yates B, White S. The incidence and risk factors in the development of medial tibial stress syndrome among naval recruits. Am J Sports Med 2004;32(3):772-780. [http://dx.doi.org/10.1177/0095399703258776]        [ Links ]

6. Dawson J, Doll H, Fitzpatrick R, et al. The routine use of patient reported outcome measures in healthcare settings. BMJ 2010;340:c 186. [http://dx.doi.org/10.1136/bmj.c186]        [ Links ]

7. Boyce W, Gowland C, Russell D, et al. Consensus methodology in the development and content validation of a gross performance measure. Physiotherapy Canada 1993;45:94-100.         [ Links ]

8. McKenna HP. The Delphi technique: a worthwhile research approach for nursing? J Adv Nurs 1994 Jun;19(6):1221-1225. [http://dx.doi.org/10.1111/j.1365-2648.1994.tb01207.x]        [ Links ]

9. McKenna H, Hasson F, Smith M. A delphi survey of midwives and midwifery students to identify non-midwifery duties. Midwifery 2002 Dec;18(4):314-322. [http://dx.doi.org/10.1054/midw.2002.0327]        [ Links ]

10. Edwards PH Jr., Wright ML, Hartman JF. A practical approach for the differential diagnosis of chronic leg pain in the athlete. Am J Sports Med 2005;33(8):1241-1249 [http://dx.doi.org/10.1177/0363546505278305]        [ Links ]

11. Sousa VD, Rojjanasrirat W. Translation, adaptation and validation of instruments or scales for use in cross-cultural health care research: clear and user-friendly guideline. J Eval Clin Pract 2011 Apr;17(2):268-274. [http://dx.doi.org/10.1111/j.1365-2753.2010.01434.x]        [ Links ]

12. Beaton DE, Bombardier C, Guillemin F, et al. Guidelines for the process of cross- cultural adaptation of self-report measures. Spine (Phila Pa 1976) 2000 Dec 15;25(24):3186-3191. [http://dx.doi.org/10.1097/00007632-200012150-00014]        [ Links ]

13. Robinson JM, Cook JL, Purdam C, et al. The VISA-A questionnaire: a valid and reliable index of the clinical severity of Achilles tendinopathy. Br J Sports Med 2001;35:335-341. [http://dx.doi.org/10.1136/bjsm.35.5.335]        [ Links ]

14. Winters M, Moen MH, Zimmermann WO et al. The medial tibial stress syndrome score: a new patient-reported outcome measure. Br J Sports Med 2015 Oct 28. pii: bjsports-2015-095060. [http://dx.doi.org/10.1136/bjsports-2015-095060]        [ Links ]

 

 

Correspondence:
M Winters
marinuswinters@hotmail.com