Article Text
PDF
Abstract
Introduction As patient-reported outcome measures (PROMs) have become of significant importance in evaluation of care and clinical research, adequately selecting the appropriate instrument is an integral part of paediatric orthopaedic research and clinical practice. This systematic review aims to provide a comprehensive overview of PROMs targeted at children with impairment of the lower limb, and to critically appraise and summarise the quality of their measurement properties by applying the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology.
Method and analysis A systematic search of the MEDLINE and EMBASE databases will be performed to identify relevant publications reporting on the development and/or validation of PROMs used for evaluating children with impairment of the lower extremity. Data extraction and quality assessment of the included studies will be undertaken by two reviewers independently and in accordance with COSMIN guidelines.
Ethics and dissemination It is not necessary to obtain ethical approval for this systematic review. The results will be published in a peer-reviewed journal and will be presented at relevant conferences to enhance information dissemination.
PROSPERO registration number CRD42021287323.
- Information Technology
- Data Collection
Data availability statement
No data are available.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Using patient-reported outcome measures (PROMs), in clinic and in research, to evaluate the changes in a patients (mental) health status has become common practice.
Good and easy-to-use PROMs for children with lower extremity problems are scarce, and a clear overview is currently unavailable.
In general, the majority of existing paediatric PROMs are used without proper scientific justification and lack a prior high-quality validation study.
WHAT THIS STUDY ADDS
This study will provide insight into the knowledge gap existent within the proper usage of PROMs for children with impairment of the lower extremity.
A high-quality methodological assessment using the COSMIN Guidelines to report the psychometric properties from multiple validation studies.
A categorised overview and comprehensive recommendation of all viable PROMS for children with impairment of the lower extremity.
Introduction
The patient experience and patient related outcomes (PROs) have become the conventional approach to describing the effects of their perceived healthcare services both in clinic and in trials.1 PROs, measured using patient-reported outcome measures (PROM), provide a comprehensive assessment of the patients’ current health condition.2 3 Orthopaedic surgeons and researchers often use PROMs to evaluate the health status of a patient and track the change over time or after treatment.1 2 4 A recent publication by Arguelles et al 5 identified the major challenges clinicians and researchers face when using PROMs in paediatric orthopaedic research; for example, most PROMs are used without formal validation for both content and context. Thus, the clinical or researcher interpreting these results could potentially make a misleading recommendation or perform an unnecessary intervention.6
In orthopaedic, clinical practice most often PROMs are used to evaluate the physical and mental status of a patient before or after an intervention, and PROMs are used to objectively measure subjective physical and/or behavioural patterns in patients with a physical impairment due to a disease to the musculoskeletal system. The selection of a PROM is vital to its own effectiveness in providing valuable information to the physician, if the PROM has not been validated for the proposed use and the intended joint, it will provide false information to the physician.
Therefore, clinicians and researchers need to use an age-appropriate PROM specifically designed for the potential diagnosis or injury of the patient; the content validity should fit both the disease and the population. Even though a variety of studies on the quality of development and validation of PROMs are readily available for clinicians and researchers, it remains challenging to judge whether the results of these studies justify an appropriate use of PROM for the intended patient population.4 7 This may result in a PROM being used in either an unvalidated patient population with a validated disease or in a validated patient population with an unvalidated disease.5 Up to 18 years of age, children will develop most of their cognitive physical skills. If a child were to develop an impairment to the upper or lower extremity due to a disease of the musculoskeletal system, the orthopaedic surgeon would use a PROM to objectively measure the progression of the disease. The outcome of the PROMs in children needs to verify if their progression/regression is due to normal physiological growth or due to a disease.
When developing and using PROMs in paediatric research, clinicians and researchers must take developmental influences such as age-dependent disease-awareness and cognitive–linguistic ability, into careful consideration for paediatric qualitative research.8 These conditions impede the proper development of validated and easy-to-use paediatric PROMs, which results in a significant sparseness; consequently increasing the difficulty of proper PROM selection in the paediatric orthopaedic patient population.7 The improper selection is exacerbated by the wide variety of individual PROMs per joint.9
The COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN) initiative has developed the COSMIN checklist and guideline for systematic reviews of outcome measurement instruments.10 11 This checklist was specifically developed to evaluate the quality of the PROMs development and the methodological quality of individual studies on psychometric properties of patient-related outcomes measurements in concordance with the GRADE approach, the PRISMA statement and the Cochrane handbook for systematic reviews of interventions and for diagnostic test accuracy.10 12 13
Objectives
The primary aim of the review is to construct a comprehensive index of all paediatric orthopaedic PROMs validated for children with impairments of the lower extremity, and to critically appraise and summarise the quality of their measurement properties with use of the COSMIN checklist. The secondary goal of this review is to provide an evidence-based recommendation for PROM selection in paediatric orthopaedic research and clinical practice for patients with an impairment of the lower extremity.
Methods
Design
This protocol was developed using the Preferred Reporting for Items for Systematic Reviews and Meta-Analyses-Protocols (PRISMA-P) checklist14 and the 10-step procedure in the COSMIN Risk-of-Bias checklist for systematic reviews of PROMs.11 12 15 The conduct and reporting of this review will be in accordance with the PRISMA statement and in accordance with the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) approach.
Information sources and search strategy
The systematic review will involve a broad search according to the COSMIN guidelines. The electronic databases of PubMed and Embase will be systematically searched using a search designed with the help of a clinical librarian, with an emphasis on sensitivity, not specificity, to ensure identification of all relevant articles. The search will consist of four distinct elements: (1) search terms describing the population of interest with a validated paediatric study search filter by Leclercq et al,16 (2) the comprehensive PROM filter developed by the PROM Group of the University of Oxford,17 and two validated filters by Terwee et al 18: (3) a highly sensitive measurement property filter and (4) an exclusion filter. The search will be restricted to English articles only by using language filters (a preliminary search for studies in the native languages spoken by the authors (Dutch) resulted in no viable studies to be included), and eligible articles have to be published after 1 January 2000 to identify contemporary outcome measures. The reference lists of included articles will be manually screened to identify additional relevant studies or citations. The search strings used for PubMed and Embase can be found in online supplemental material 1.
Supplemental material
Eligibility criteria, inclusion and exclusion criteria
A study will be included if a full-text original version of the article is available. The article must report on an original study describing the development and/or the evaluation of one or more measurement properties of a disease-specific patient-reported and/or proxy-reported questionnaire of any language. The study population must consist of children (0–18 years old) with an orthopaedic impairment in the lower extremity region.
The exclusion criteria are: (1) any study design in which the patient-reported and/or parent/proxy-reported questionnaire was only used as an outcome measurement instrument (eg, randomised controlled trials, longitudinal studies, systematic reviews or meta-analysis); (2) any study in which the questionnaires evaluated the use of prosthetic limbs.
Patient and public involvement
No patients will be involved during this systematic review of the current worldwide available literature on PROMs for paediatric patients with an impairment of the lower extremity.
Study selection
The literature search will be uploaded to Endnote (X9, Clarivate Analytics, London, UK) and all duplicate records will be removed from the results. A full-text selection will be made during a two-phase selection procedure. During the first part, the two reviewers (TFFS and RK) will independently identify all eligible articles from title and abstract for full-text review based on the predefined eligibility criteria using the Rayyan web application,19 which facilitates this blinded selection procedure. During the second part, the two reviewers (TFFS and RK) will independently screen all full-text articles based on the eligibility criteria and conclude the final inclusion. Disagreements during both phases will be resolved by a third reviewer (IS or CvB).
Quality appraisal
The methodological quality of studies on measurement properties included in this review will be assessed according to the extensive and recently improved COSMIN methodology for qualitatively evaluating studies on PROMs.16 Detailed information on the COSMIN taxonomy, the stepwise approach of the COSMIN methodology and the COSMIN checklists applied in this review, can be found in the corresponding publications by Mokkink et al.12 13 Prinsen et al,15 and Terwee et al.11
The modular tool, providing risk-of-bias scores for the psychometric properties, uses a four-point rating scale: ‘very good’, ‘adequate’, ‘doubtful’ or ‘inadequate’.12 ‘The worst score counts’ principle will be applied to come to an overall methodological quality rating for each individual study on a measurement property.15 To evaluate reports on content validity and/or PROM development, a separate COSMIN guideline will be used: the COSMIN methodology for evaluating content validity.11
During selection, quality appraisal and data extraction, the inter-rater agreement of the reviewers (TFFS and RK) will be considered appropriate when reviewers reach >80% agreement.20 Per recommendation (Mokkink et al 20), if the inter-rate agreement falls below 80%, both reviewers will review and redefine the inclusion and selection criteria during selection, quality appraisal and data-extraction, and repeat the entire process until>80% agreement is reached.
Data extraction
Two reviewers (TFFS and RK) will independently extract data from the included studies. The COSMIN guidelines have provided a guidance document15 with tables and figures for data extraction, which will be used to standardise the information that will be gathered. All information gathered will be divided into three categories with multiple subcategories (box 1).
Visual overview of the standardised parameters gathered (three main categories and multiple subcategories) from each individual study on patient-reported outcome measures (PROMs)
Characteristics of the included PROMs
Testable construct
Target population
Mode of administration
Recall period
Subscales (number of items in questionnaire)
Response options
Range of scores/scoring
Original language
Available translations
Characteristics of the included study populations
Number of patients
Age (mean and dispersion) in years
Gender (% of females)
Disease characteristics
Disease, duration in years, severity
Instrument administration
Setting, country, language
Response rate
Evidence of the psychometric measurement properties of the studied PROMs
Data synthesis
A qualitative synthesis of the evidence per measurement property per PROM will be constructed to come to an overall conclusion of PROM quality. If consistent (ie, ≥75% of the results are either rated ‘sufficient’ or ‘insufficient’), the results of the individual studies on measurement properties will be qualitatively summarised and rated against the criteria for good measurement properties. If inconsistent, an explanation for this inconsistency will be sought. If the inconsistency remains unexplained, the overall result will be rated as ‘inconsistent’ (±). An ‘indeterminate’ (?) rating will be given if the individual results are all rated as ‘indeterminate’.
After qualitatively synthesising and rating the overall results per measurement property, per PROM, the quality of this evidence will be graded according to the modified GRADE approach.21 22 The summarised results will be graded as ‘high’, ‘moderate’, ‘low’ or ‘very low’, based on three factors: risk of bias (based on methodological quality), inconsistency and imprecision (ie, sample size). The fourth factor ‘indirectness’ will not be taken into consideration in evaluating evidence quality; this review will only include studies with a predefined and fixed patient population. If the quality of the summarised result is rated ‘inconsistent’ or ‘indeterminate’, the quality of the evidence cannot be graded.
Patient and public involvement
No patients are involved in this study.
Discussion
The use of questionnaires/PROMs in orthopaedic medical care will become the futures’ main focus to determine patient-related clinical outcomes in both daily practice and in research.4 23 The research done by Arguelles et al 5 has shown that the majority of PROMs are used without supportive evidence for content validity and construct validity.
With this study on validated PROMs use, we hope to highlight an area of expertise, which will benefit from a research focus on validating a PROM for a specific patient population, a specific disease or a combination of disease and population. Increasing awareness about the improper PROMs usage will encourage the future research to focus more on proper development and validation studies on PROMs.
Data availability statement
No data are available.
Ethics statements
Patient consent for publication
Ethics approval
There is no need to obtain ethical approval for this systematic review protocol. The results will be submitted for publication in a peer-reviewed scientific journal for easy distribution among clinicians and researchers. The results will be submitted to relevant conferences to spread more awareness on adequate patient-reported outcome measure selection.
Acknowledgments
A special thanks to Paulien H. Wiersma, MSc, faculty liaison medical sciences (University Library Utrecht, UMC Utrecht), for helping to create our search strings.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors TFFS, main author and first reviewer; design of the review, constructing and executing the search strategy, screening publications, analysis and interpretation of data, conception and writing of the manuscript. RK, second reviewer; constructing and executing the search strategy, screening publications, analysis and interpretation of data. IS, second supervisor; providing critical revisions and help with interpretation of data. DE, third supervisor; provided orthopaedic and scientific expertise and critical revisions to the study protocol. CvB, first supervisor; conception and design of the review, provided orthopaedic and scientific expertise and critical revisions to the study protocol.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; internally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.