Review

Identifying common health indicators from paediatric core outcome sets: a systematic review with narrative synthesis using the WHO International Classification of Functioning, Health and Disability

Abstract

Background Indicators of child health have the potential to inform societal conversations, decision-making and prioritisation. Paediatric core outcome sets are an increasingly common way of identifying a minimum set of outcomes for trials within clinical groups. Exploring commonality across existing sets may give insight into universally important and inclusive child health indicators.

Methods A search of the Core Outcome Measures in Effectiveness Trial register from 2008 to 2022 was carried out. Eligible articles were those reporting on core outcome sets focused on children and young people aged 0–18 years old. The International Classification of Functioning, Disability and Health (ICF) was used as a framework to categorise extracted outcomes. Information about the involvement of children, young people and their families in the development of sets was also extracted.

Results 206 articles were identified, of which 36 were included. 441 unique outcomes were extracted, mapping to 22 outcome clusters present across multiple sets. Medical diagnostic outcomes were the biggest cluster, followed by pain, communication and social interaction, mobility, self-care and school. Children and young people’s views were under-represented across core outcome sets, with only 36% of reviewed studies including them at any stage of development.

Conclusions Existing paediatric core outcome sets show overlap in key outcomes, suggesting the potential for generic child health measurement frameworks. It is unclear whether existing sets best reflect health dimensions important to children and young people, and there is a need for better child and young person involvement in health indicator development to address this.

What is already known on this topic

  • There are aspects of children and young people’s health that are not routinely, universally measured.

  • This lack of data about children and young people’s health can hinder health policy, care and research.

  • Core outcome sets are a way of providing guidance on what to measure, but often relate to specific clinical populations only.

What this study hopes to add

  • This study identifies common, universally shared themes across core outcome sets.

  • These themes help identify key universal indicators of child health that should be measured across populations.

  • The study further identifies gaps and areas for improvement in the involvement of children and their families in identifying key health indicators.

Introduction

Measure what is measurable, and make measurable what is not. (Isaac Newton)

How society measures characteristics of health is important because the act of measuring is an act of making an aspect of health visible—in societal discussion, decision-making and resource allocation. Measurable health characteristics are often referred to as indicators, and how they are defined for measurement shapes the information that is collected and available for decisions. Measuring health through routine and other large-scale data sets is increasingly common, and the use of the resulting data in societal conversations, decision-making and prioritisation likely has major consequences for people’s lives.

Many important aspects of child health are not yet routinely measured. While some health indicators such as births, deaths and vaccinations are long-established, many others lack robust data. The COVID-19 pandemic provides an illustrative example, where the absence of data about children’s physical and mental health and development has hindered analyses of the impacts of the pandemic on children and, arguably, limited families’, decision-makers’ and professionals’ ability to articulate their concerns. This, in turn, has hindered effective decision-making about important policy decisions such as opening of schools. The need to rethink the measurement of child health is gaining policy traction internationally1–3 and the choice over what indicators to measure is now paramount.4

Measuring child health is hindered by lack of agreement about the important, universally applicable and clinically meaningful indicators of child health and well-being. The past decade has seen a positive, increased focus on common research outcomes relevant across professional disciplines, illustrated for example in the development of core outcome sets. Core outcome sets provide an agreed, standardised minimum set of outcomes to be reported for a specific clinical group in clinical research and increasingly in clinical practice.5 There is no standardised methodology for core outcome set development, but they generally involve stakeholders agreeing the most important outcomes to report using consensus methodology.6 However, although patient involvement in core outcome set development has been recognised important,7 the sets continue to be fundamentally organised around clinical and diagnostic groups to service healthcare and trials. They do not currently provide an off-the-shelf set for universally important indicators of child health that could be applied across diagnostic categories, and across healthy and clinical populations.

We believe there is a need for a generic, universal, inclusive set of core child health and well-being indicators, and we think such a set should: (1) take the child’s perspective (as opposed to, for example, medical or educational); (2) adopt a life-course view, that is, consider future health and well-being prospects as well as present health; (3) be practically oriented, with real potential to inform routine data collection and societal discourse, policy and interventions; and (4) be inclusive of diverse health and developmental trajectories.

As the first step, in 2017–2020, we engaged with young people, families and decision-makers to discuss broad ideas about health indicators that might matter to them. Informed by those discussions, the present study sought to identify common themes in existing paediatric core outcome sets, as a way to contribute to the wider efforts to progress towards a generic, universal, inclusive set of core child health and well-being indicators. The present paper reports on two specific objectives: (1) to identify common indicators of health included in published paediatric core outcome sets for children and youth (0–18 years); and (2) to explore how children, young people and their families’ perspectives informed the selection of these outcomes.

Methods

This study used established review methods8 9 to identify and select papers reporting on published paediatric core outcome sets. Narrative synthesis was used to analyse the data.10 A protocol was agreed by the review team, with objectives, selection criteria and data extraction techniques agreed in advance. The results are reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta Analyses 2020 statement.11 The review objectives fell outside the scope of the International Prospective Register of Systematic Reviews.12 We used the WHO International Classification of Functioning, Disability and Health (ICF),13 an existing consensus framework of health. The ICF is an internationally recognised set of domains of health and functioning that provides a structured way to understand and classify an individual’s health and functioning. It considers health indicators relating to body functions and structures, activities and participation and provides multidisciplinary terminology to classify them.14

Search

Paediatric core outcome sets were identified from the Core Outcome Measures in Effectiveness Trial (COMET) register.15 The register is the internationally recognised database of ongoing and completed studies relating to core outcome set development. It is updated by systematic review annually, and by authors registering their sets on an ad hoc basis. The last review carried out by COMET, identifying new published core outcome sets for addition to the database was published in January 2021 including studies up to and including December 2018.16 To identify paediatric sets, we systematically searched the COMET register15 from January 2008 to March 2022 using the terms indicated in box 1. The search was updated to incorporate the COMET database review up to and including December 2019.17

Box 1

COMET Database Search Criteria

Health Area-disease category

Child Health

AND

Publication year

2008 onwards

AND

Published/unpublished

Published

OR

Minimum age

0

AND

Maximum age

18

AND

Publication year

2008 onwards

AND

Published/unpublished

Published

Selection

Articles were eligible for inclusion in this review if: (1) they reported on a disease or diagnostic group-specific core outcome sets developed using consensus methods (defined as reporting a clear criterion for determining outcome inclusion/exclusion, the number of people in each stakeholder group at each stage, and all outcome scoring7; (2) focused on children and young people (0–18 years) as the health beneficiaries; and (3) was published after 2008 (a previous systematic review18 identifies work prior to this date). Systematic reviews, core outcome set protocols and core outcome sets without reported consensus methods were excluded, as were articles reporting core outcome sets spanning children and adults. Titles and abstracts were first independently reviewed by one author (VH) with a randomly selected 25% double screened by a second author (ER). Full texts of articles meeting the inclusion criteria were then further reviewed, with a randomly selected 50% double screened. Reasons for exclusion were recorded at both stages. Agreement between authors at screening and full-text stages was 94% and 82%, respectively, and discrepancies were resolved through discussion.

Data extraction and analysis

Study-specific forms were developed, and piloted; and subsequently used by two authors (VH, ER) to independently extract data on: authorship; the core outcome set scope, use and outcomes listed; and consensus methodology used, including method type, stakeholder numbers and patient/public involvement. The ICF was used as a framework for coding the outcome data; this provided a common, internationally agreed terminology for naming and describing health outcomes. Outcomes were considered health indicators if they directly related to the child. Outcomes related to care inputs and processes (eg, resources, length of stay, attendance) were excluded. Outcomes were coded using a published ICF decision tree.14 For the full mapping, please see the full data set.19 Article screening was done in EndNote; data extraction in Microsoft Word; and analysis in Microsoft Excel.

Results

We identified 206 records, of which 68 duplicates were removed. A further 86 records were excluded following title and abstract screening, 16 were excluded following full-text assessment (figure 1) and 36 studies were selected for inclusion in full review (table 1). Selected articles described 36 core outcome sets related to: gastrointestinal conditions (n=8); neurological conditions (n=7); ear, nose and throat (n=5); orthopaedics (n=4); general paediatrics (n=3); neonatology (n=2); respiratory (n=2); metabolic disease (n=2); and rheumatology, oncology and dentistry specialities (n=1 each) (table 1). The number of outcomes in a core set ranged from 3 to 39, with a median of 9.

Figure 1
Figure 1

Summary of screening strategy. COMET- Core Outcome Measures in Effectiveness Trial. COS- Core Outcome Set

Table 1
|
Overview of included paediatric core outcome sets

Common outcome domains

From the 36 core outcome sets included, 441 outcomes were extracted. Mapping these to the ICF resulted in 22 clusters of outcomes, linked to 25 unique ICF codes (table 2). Medical diagnoses formed the largest cluster, activity and participation forming the majority of the other larger clusters. Several smaller clusters related to body functions were identified as well as a cluster relating to growth. Personal factors relating to emotional well-being were clustered and showed relatively higher commonality. Environmental factors were included in several sets and collectively formed one, large cluster. table 3 summarises the top 10 clusters by size after medical diagnostic outcomes and shows how many sets each outcome cluster was represented in.

Table 2
|
Overview of coded and grouped outcomes extracted from paediatric core outcome sets
Table 3
|
Top 10 outcome clusters by size (excluding medical diagnostic outcomes)

Stakeholder involvement

Table 4 summarises the stakeholder involvement in the three main sections of core outcome set development: (1) generation of an outcome longlist; (2) consensus process; and (3) final consensus meeting. The diversity of stakeholders involved varied (table 4). Five (14%, 5/36) only sought opinion from clinicians/researchers throughout their development, including no parents or children or young people (CYP) at any stage. The remaining 31 (86%, 31/36) included parents/caregivers in at least 1 part of the development, with 16 including parents/caregivers in both outcome listing and consensus stages. Overall, 11 (31%, 11/36) included CYP in at least 1 part, with 9 (25%, 9/36) including them in both outcome listing and consensus (table 4). Seven sets related to neonatal or infant populations; three of these were for conditions likely to impact into later childhood (gastroschisis, brachial plexus injury, infants receiving neonatal care) and so could have considered including older CYP. Parent/caregiver and young person stakeholder representation had increased over time (figure 2).

Table 4
|
Summary of stakeholder group involvement in paediatric core outcome set development
Figure 2
Figure 2

Stakeholder groups involved in (1) outcome long listing and (2) consensus process by year. Left hand bar represents outcome long listing, right hand bar represents consensus process. CYP- children or young people.

The median number of rounds used for consensus methodologies was 2 (range 1–7), with only 3 studies (8%, 3/36) using only one consensus round. Across the remaining 33 studies using multiple rounds, attrition rates of included stakeholders varied with a mean of 21% (range 0%–63%) for clinicians, 29% (range 0%–95%) for studies including parents in more than one round and 19% (range 0%–73%) for studies including young people in more than one round. Seven (19%, 7/36) mitigated concerns around attrition by recruiting new parent or CYP stakeholder groups for subsequent rounds. Patient and public involvement (PPI) in study design was not universally included; only fourteen (39%, 14/36) included parents or young people, either through inclusion in the study steering group, through consultation with an advisory group or through the piloting of methods. There was some evidence that the use of PPI in designing study methods impacted consensus attrition rates, with mean stakeholder attrition lower for studies that utilised PPI (figure 3, table 5).

Figure 3
Figure 3

Mean attrition between consensus rounds by stakeholder group, comparing studies including patient and public involvement (PPI) in study design to those without.

Table 5
|
Attrition of different stakeholder groups throughout consensus rounds

Figure 4 shows all 36 core outcome sets, plotted by stakeholder (parents and/or children) involvement in longlisting (x-axis) and the consensus process (y-axis). This figure highlights that the sets with greater stakeholder involvement tended to be those including a smaller number of domains. The bubble sizes represent the number of individual outcome domains included in the final core outcome set. Sets nearest the top right-hand corner are those with higher proportions of involvement at both stages of the process (representing higher proportions of stakeholder involvement). Clustered around zero are those sets with very minimal, or no stakeholder involvement at either stage, and those to the left of the plot represent those that used reviews only for the longlisting stage, or for which data around stakeholder participation was unavailable.

Figure 4
Figure 4

Stakeholder involvement in outcome long listing and outcome consensus for all paediatric core outcome sets. Bubble size represents final paediatric core outcome set size. Studies to the left of the y-axis had no stakeholder involvement in the outcomes listing phase as outcome longlists were derived from systematic reviews, or the stakeholder breakdown was not published. CYP- children or young people.

Discussion

We extracted 441 outcome variables from 36 paediatric core outcome sets, and mapped them to 22 outcome clusters of the ICF as well as on environmental factors potentially affecting these outcomes. These 22 areas represent potential child health indicators for measurement in routine and large-scale data. The most common indicator cluster was a child’s diagnosis (in 76% of sets). The second most common was pain (in 33% of sets), followed by activity and participation indicators related to self-care, school, personal well-being, community and civic life (all in 31% of sets), communication and social interactions and mobility (both in 25% of sets). These were followed by body structure and function indicators relating to sleep (22%), mental functions (16%) and growth (in 11% of sets). In addition, three categories of common indicators residing outside the ICF were identified: mortality (in 33% of sets), dental/oral health (in 8%) and quality of life (in 53% of sets). Overall, while there is an in-principle commitment to patient involvement in core outcome set development, the selection and prioritisation of indicators in the included sets were more informed by clinicians and researchers than children, young people or parents who were under-represented.

The present review used a robust search and data extraction strategy, independent double screening and data extraction by two authors, and a published decision tree for the coding of data on the ICF. The search was limited to the COMET initiative registry, which itself is rigorously updated annually, with the update published as a peer-reviewed systematic review. We can therefore be relatively certain that the present study successfully identified articles up to and including the last update of the registry (December 2019); the inclusion after that relies on ad hoc author registration and so it is possible that later sets were not included. We did not assess the quality of the paediatric core outcome sets as this was not one of the aims on this review.

There were some limitations to this work. The search excluded core outcome sets that spanned both children and adults. This was as the focus was on transdiagnostic health indicators specific to children and young people. The priorities, life experiences and opinions of CYP differ compared with adults20 and we felt including studies with a broader age range would have added adult focused outcomes that were inconsistent with our aims. A further limitation was that while independent double screening and data extraction was undertaken by two authors, only a randomly selected proportion was reviewed by the second author.

The outcome clusters identified in the present review align with other, concurrent initiatives. To date, two approaches to selecting indicators and outcomes have dominated: the public health, and the clinical. In the public health approach, coarse indicators (eg, births, deaths, vaccination rates) are selected and used for national and regional reporting and comparison. One such prominent example is the State of Child Health by Royal College of Paediatrics and Child Health.2 This uses key child health indicators to monitor trends and provide policy recommendation across the UK. Mortality, weight, oral health and mental health are all included, corresponding to some of the domains in the present review. Another example is the Public Health England (PHE) child health indicators, used to monitor trends and inform policy.21 A challenge with the current public health indicators and related data are children at highest risk of long-term ill health are not well represented—including children with health conditions or marginalised due to sociodemographic circumstances. They are missed from data collection, or the data does not cover key indicators relevant to them, for example, the PHE data does not include pain or sleep. In contrast, in the clinical approach, highly specialised end points (eg, particular illness symptoms, treatment side effects) are selected on the basis of their relevance to specific interventions and clinical groups; these indicators and measurement rarely touch on all children universally. There may be a tacit assumption that missing the highest risk children and young people from the public health data is compensated for by the clinical data. However, this is problematic as it neglects some universally important aspects of child health for the clinical populations, and creates a two-strand system where children are viewed through a binary ‘typical’ versus ‘clinical’ lens which is then translated into segregated policy and decision-making. For example, the UK Chief Medical Officers developed separate physical activity guidelines for typically developing and disabled children, in large part based on an argument that there was no compatible data to inform a joint, integrated guideline. Similar concerns apply to other major health areas of obesity, smoking and drinking, and uptake of vaccines.

Arguably, both of the public health and clinical approaches have emerged from measuring what can be measured within the current system and with existing instruments and been heavily influenced by the dominant expert paradigms of public health and clinical medicine. Both approaches can be criticised for overlooking important aspects of health and well-being valued by CYP, families and decision makers—and thus for resulting in data and findings with limited ability to inform decision-making. In the present review, we sought to adopt a third perspective to complement these two approaches, namely to identify ways forward for making more visible and measurable some of the important aspects of health that are not yet strongly featured in either of the existing approaches. Our findings here suggest that there indeed are shared, recognisable, universal health indicators that are likely to apply both to public health and clinical enquiries, such as self-care, pain, sleep and social interactions. These findings converge with, and further add to, the recent report from another international group that, independently and concurrently to us, sought to advance the thinking around child health measurement.22 That concurrent work retained a healthcare paradigm and focused on existing standardised measurement tools that could be used across diagnostic groups (explicitly excluding health indicators that were deemed not yet measurable through standardised instruments). Similarly to our review, they found universally important health indicators such as survival, growth, pain, school attendance and social functioning. Collectively, the findings from these two studies provide a strong foundation for the development of a universal, common child health indicator framework that spans traditional discipline and sector boundaries to complement existing core outcome sets for interventions as well as inform routine public health data collection. Developing such approach has the potential to facilitate more integrated, inclusive policy, practice and research across child health by focusing attention to universally important health and well-being goals that matter to all children, including children who may be clinically unwell.

To progress a meaningful child health indicator, framework will require further, substantial development. Key to this will be the involvement of a wider pool of stakeholders, particularly CYP, in deciding what the key indicators should be and how these are best operationalised for measurement and data collection. The findings from the present review align with those of the most recent COMET annual update17 that found only 16% of paediatric core sets included direct input from CYP. While this is lower than 31% of sets found in this review, the authors included sets spanning both adult and CYP populations, suggesting that with a broader review strategy CYP inclusion is even poorer. CYP and parents often differ in their priorities to health experts,4 23 24 and it cannot be assumed that parents hold the same views as children and young people.25 Therefore, including CYP as well as parents is crucial. There is evidence that CYP both understand and are keen to be involved in the development of health indicator and outcome sets20 and in neonatal or infant conditions involvement of older children with relevant experiences may be useful. A key lesson from the present review is that those engaging higher numbers of CYP differed from standard Delphi methodology, perhaps paying particular attention to adapting the methods. Furthermore, focusing on a manageable number of indicators may facilitate stakeholder involvement.

There are three immediately actionable recommendations from the present study. First, national data set administrators should consider adding pain and sleep as key health indicators. Second, anyone developing core outcome or indicator sets should ensure they involve children, young people and parents—with adaptations to the methods to make this feasible. Third, clinical evaluators should consider inclusion of key, universally important child health outcomes that may be relevant to their interventions but absent from the current core sets.