Article Text
PDF
Abstract
Objectives For 300 paediatric trials, we evaluated the reporting of: a data monitoring committee (DMC); interim analyses, stopping rules and early stopping; and adverse events and harm-related endpoints.
Methods For this cross-sectional evaluation, we randomly selected 300 paediatric trials published in 2012 from the Cochrane Central Register of Controlled Trials. We collected data on the reporting of a DMC; interim analyses, stopping rules and early stopping; and adverse events and harm-related endpoints. We reported the findings descriptively and stratified by trial characteristics.
Results Eighty-five (28%) of the trials investigated drugs, and 18% (n=55/300) reported a DMC. The reporting of a DMC was more common among multicentre than single centre trials (n=41/132, 31% vs n=14/139, 10%, p<0.001) and industry-sponsored trials compared with those sponsored by other sources (n=16/50, 32% vs n=39/250, 16%, p=0.009). Trials that reported a DMC enrolled more participants than those that did not (median [range]): 224 (10–60480) vs 91 (10–9528) (p<0.001). Only 25% of these trials reported interim analyses, and 42% reported stopping rules. Less than half (n=143/300, 48%) of trials reported on adverse events, and 72% (n=215/300) reported on harm-related endpoints. Trials that reported a DMC compared with those that did not were more likely to report adverse events (n=43/55, 78% vs 100/245, 41%, p<0.001) and harm-related endpoints (n=52/55, 95% vs. 163/245, 67%, p<0.001). Only 32% of drug trials reported a DMC; 18% and 19% did not report on adverse events or harm-related endpoints, respectively.
Conclusions The reporting of a DMC was infrequent, even among drug trials. Few trials reported stopping rules or interim analyses. Reporting of adverse events and harm-related endpoints was suboptimal.
- data collection
- ethics
- general paediatrics
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
What is already known on this topic?
Data monitoring committees aim to safeguard participants and ensure rigorous conduct in trials. They are recommended for trials that recruit from vulnerable populations, including children.
Reviews of trials published from 1996 to 2002 and 2005 to 2007 showed that the reporting of data monitoring committees was infrequent in paediatric trials.
Despite not always requiring an independent data monitoring committee, the monitoring of safety data is always warranted in paediatric trials.
What this study hopes to add?
In a randomly selected sample of 300 paediatric trials published in 2012, 18% reported a data monitoring committee.
Fifty-two per cent of trials did not report any adverse events data.
Only 32% of drug trials reported a data monitoring committee; 18% and 19% did not report on adverse events or harm-related endpoints, respectively.
Introduction
Data monitoring committees (DMCs) help to ensure ethical conduct and participant safety in trials via frequent risk–benefit appraisals to identify ‘definitive evidence of benefit, convincing evidence of harm, or sufficient evidence of no potential benefit’.1 These periodic appraisals (ie, interim analyses) are used to inform recommendations regarding trial modification, continuation or termination (ie, early stopping) based on pre-established stopping rules.2–4 In order to provide credible and unbiased monitoring of ongoing trials, members of the DMC must be independent of the trial sponsor and typically include a statistician and a clinical expert in the therapeutic area being investigated.5 In trials that investigate high-risk interventions and/or that recruit from vulnerable populations, the inclusion of bioethicists and patient or parent advocates should also be considered.5
Although safety and efficacy data should be monitored in all trials, formal establishment of a DMC might not be needed in trials where the intervention(s) are known to cause minimal risk, or trials of behavioural interventions or that analyse administrative data.6 For other trials, deciding whether a DMC is required should be based on the level of safety concern (eg, unknown risks or known risks), the practicality of having a DMC and whether having a DMC would help ensure the scientific validity of the trial.6 DMCs are always required for trials that evaluate new drugs, biologicals or devices. In those that recruit from vulnerable populations, their establishment should be strongly considered.2 3
As children are typically considered to be vulnerable individuals, DMCs are frequently warranted in paediatric trials; however, earlier reviews showed that DMCs were seldom reported.7–9 Moreover, reviews of trials investigating treatments for common paediatric conditions have found their reporting of harms to be suboptimal, limiting their utility for clinical decision making.10 11 In an evaluation of a random sample of 300 paediatric trials published in 2007, at which time only limited evidence-based guidance was available for paediatric trials, just 5% reported a DMC and 43% reported adverse events data.9 Since that time, Standards for Research in (StaR) Child Health published six evidence-based standards addressing priority issues regarding the conduct and reporting of paediatric trials, including guidance on the establishment of DMCs.12–18
As the use of DMCs in trials continues to evolve, and in light of the newly published guidance for the conduct and reporting of paediatric trials, we evaluated a sample of paediatric trials published in 2012 to determine the reporting of three distinct but related issues: (A) a DMC, its members and their responsibilities; (B) interim analyses, stopping rules and early stopping; and (C) adverse events and harm-related endpoints.
Methods
Context
Our methods have been detailed in previous reports.19 20 A brief description follows.
Sample selection
In November 2013, we searched the Cochrane Central Register of Controlled Trials (CENTRAL) for trials published in 2012.19 20 CENTRAL is a comprehensive database of reports of randomised and quasirandomised trials, taken mainly from MEDLINE and Embase.21 As this project was part of an ongoing surveillance initiative,19 the 2012 publication date was chosen because it was 5 years following an earlier evaluation of a random sample of 300 paediatric trials undertaken in 2007.9 The date also coincided with the publication of the StaR Child Health Standards, allowing for the establishment of baseline data for the reporting of priority items outlined within each.
We randomly ordered the 2296 unique records retrieved by the search using a computer-generated list in Excel (V.2016, Microsoft Corporation, Redmond, Washington, USA). Next, a single researcher screened the records by title and abstract and selected the first 300 (13%) trials that reported on outcomes for participants aged 0–18 years or that recruited both children and adults with an upper age limit of 21 years. The sample size was selected based on our previous evaluation of trials published in 2007.9 We did not restrict the sample by language, condition, intervention or outcome type.
Data extraction
We extracted data from each trial using a standard form in Research Electronic Data Capture22 pertaining to the reporting of: the presence of a DMC (yes or no), its members (defined by their professional role, eg, statistician and healthcare provider) and their responsibilities (eg, adjustments to enrolment and reviewing safety data); interim analyses (yes or no), stopping rules (yes or no) and early stopping (yes or no, and reasons); and the monitoring for and occurrence of adverse events (yes or no, and type) and harm-related endpoints (yes or no, and type).
As part of the larger study, we collected data on characteristics of the publication, trial design, intervention, trial conduct, trial sample, consent and recruitment, outcomes, conclusions, trial registration and risk of bias.19. 20 Our data extraction guide was modelled after that used in the 2007 study,9 with new items added following consultation with clinical and methodological experts. The complete data extraction guide is available in a previous report,19 whereas that for the variables presented in this study is in online supplementary appendix 1. We classified the primary diagnostic category for each trial following the WHO’s International Statistical Classification of Diseases and Related Health Problems 10th Revision.23 Table 1 shows our classification scheme for other relevant trial characteristics. Data related to consent and recruitment, study design, trial registration and risk of bias have been reported elsewhere.19 20
Supplementary file 1
We used trial registers, published protocols and/or companion articles to complement data extraction when available. When a registration record was not cited in the publication, we searched the International Clinical Trials Registry Platform (http://apps.who.int/trialsearch/), ISRCTN Registry (http://www.isrctn.com/) and Google (http://www.google.ca/). We located registration records for 46% (n=138/300) of the trials.19 We used protocols and companion articles only when they were cited in the published reports. All data were extracted from the published trials by one researcher and verified by another (AG or MPD) to identify and correct errors or omissions.
Data extraction classification scheme
Analyses
We analysed the data descriptively in SPSS Statistics (V.25, IBM Corporation, Armonk, New York, USA). We investigated differences in reporting of DMCs, adverse events and harm-related endpoints by trial characteristics using the Fisher’s exact test and by sample size using the Mann-Whitney test in StatXact (V.10.0, Cytel, Cambridge, Maryland, USA).
Patient and public involvement
Patients and the public were not directly involved in any aspect of this research.
Results
Trial characteristics
The characteristics of the 300 trials have been reported19 and are provided for context. Most (n=242/300, 81%) trials used a parallel design and were efficacy or superiority trials (n=279/300, 93%). Thirty-three (11%) were described as pilot or exploratory. The most common funding source was government funding (n=135/300, 45%), followed by private (n=81/300, 27%), academic (n=71/300, 24%), pharmaceutical (n=41/300, 14%) and industry funding (n=9/300, 3%). The most common treatments investigated included drugs (n=85/300, 28%), communication, organisational or educational programmes (n=52/300, 17%), rehabilitation or psychosocial interventions (n=30/300, 10%) and medical devices (n=29/300, 10%). Nearly half (n=140/300, 47%) of the trials were undertaken at a single centre. The most common diagnostic categories included mental and behavioural disorders (n=50/300, 17%), infectious and parasitic diseases (n=39/300, 13%), conditions of the respiratory system (n=30/300, 10%) and conditions originating during the perinatal period (n=28/300, 9%). The trials reported data for the following categories of primary outcomes: behavioural (n=46/300, 15%); biomarker (n=55/300, 18%); pain (n=14/300, 5%); physiological (n=130/300, 43%); psychological (n=28/300, 9%); techniques/training (n=13/300, 4%); and quality of life (n=5/300, 2%). Nine (3%) trials investigated primary outcomes that did not fit into any of these categories, for example, knowledge and healthcare costs.
Data monitoring committees
About one-fifth (n=55/300, 18%) of trials reported a DMC (table 2). Among these, just 20% (n=11/55) reported on its composition. Membership most commonly included physicians (n=9/11, 82%) and statisticians (n=6/11, 55%). No trial (n=0/11) reported the membership of a patient or consumer or community advocate. Sixty percent (n=33/55) of trials that reported a DMC also reported the responsibilities to which it was assigned. Among these, the most common were reviewing safety data (n=26/33, 79%), adjusting enrolment (n=7/33, 21%), and making recommendations regarding trial termination (n=6/33, 18%) and trial conduct (n=6/33, 18%).
Reporting of DMCs, interim analyses, stopping rules and early stopping
Reporting of a DMC was more common among multicentre than single centre trials (n=41/132, 31% vs n=14/139, 10%; p<0.001) (table 3). Trials that reported a DMC randomised larger numbers of participants than those that did not (median [range]: 224 (10–60480) vs 91 (10–9528); p<0.001). Reporting a DMC was more common among trials that tested drugs (n=27/85, 32%), vaccines (n=5/14, 36%), alternative therapeutic interventions (n=4/14, 29%) and prevention or screening interventions (n=3/14, 21%) compared with those that tested communication, organisational or educational programmes (n=4/52, 8%), medical devices (n=2/29, 7%) and rehabilitation or psychosocial interventions (n=1/30, 3%) (p=0.001). None (n=0/9) of the trials that tested surgeries or radiotherapy reported a DMC. Reporting of a DMC did not differ by primary outcome type (p=0.16). Trials with an industry or pharmaceutical sponsor were more likely than those with other forms of sponsorship to report a DMC (n=16/50, 32% vs n=39/250, 16%) (p=0.009).
Reported presence of a data monitoring committee stratified by trial characteristics
Interim analyses, stopping rules and early stopping
Few trials that reported a DMC reported on any interim analyses (n=14/55, 25%) (table 2). Only 22% (n=12/355) of the trials reported stopping rules. Thirteen trials (4%) reported early stopping; reasons included inadequate recruitment (n=5/13, 38%), futility (n=5/13, 38%), benefit of the treatment (n=2/13, 15%) and funding limitations (n=1/13, 8%). No trial reported early stopping due to harms. Less than one-third (n=4/13, 31%) of trials that reported early stopping also reported stopping rules.
Adverse events and harm-related endpoints
Less than half (n=134/300, 45%) of the trials reported a plan to collect data on adverse events in the methods section of the publication (table 4). About one-third (n=109/300, 36%) of trials specified the method by which they planned to collect adverse events data.
Reporting of adverse events and harm-related endpoints
More than half (n=157/300, 52%) of the trials did not report any data related to adverse events. This included 11% (n=15/134) of the trials that reported a plan to collect and 12% (n=13/109) of the trials that specified a method for collecting adverse events data. Among the 48% (n=143/300) of trials that reported data on adverse events, 36% (n=52/143) reported severe harms, 11% (n=16/143) reported any harm (not individually described), 9% (n=13/143) reported organ system level harms and 74% (n=106/143) reported specific harms. Twenty-two trials (n=22/143, 15%) reported that no adverse events occurred. When adverse events data were reported, most trials (n=119/143, 83%) reported these by group (ie, intervention vs control, as opposed to aggregated data).
Seventy-two per cent (n=215/300) of trials reported information on harm-related endpoints. Among these, 25% (n=54/215) reported discontinuations due to adverse events and 22% reported deaths during the trial (n=47/215). Fifty-three per cent (n=114/215) of these trials reported withdrawals for which the reason was either unknown or not disclosed by the authors. About one-quarter (n=57/215, 27%) of these trials reported that there were no withdrawals or discontinuations due to adverse events.
Trials that reported the presence of a DMC were more likely to report data on adverse events (n=43/55, 78% vs n=100/245, 41%; p<0.001) and harm-related endpoints (n=52/55, 95% vs n=163/245, 67%; p<0.001) (table 5). Adverse events data were most commonly reported among trials that examined vaccines (n=12/14, 86%) and drugs (n=70/85, 82%) and infrequently reported among trials that examined communication, organisational or educational programmes (n=4/52, 8%) and rehabilitation or psychosocial interventions (n=4/30, 13%) (p<0.001). Harm-related endpoints data were most commonly reported among trials that examined vaccines (n=14/14, 100%), drugs (n=69/85, 81%), medical devices (n=20/29, 69%), surgery or radiotherapy (n=6/9, 67%) and rehabilitation or psychosocial interventions (n=20/30, 67%). They were less commonly reported among trials that examined prevention or screening programmes (n=8/14, 57%), communication, organisational or educational programmes (n=28/52, 54%) and alternative therapeutic interventions (n=8/14, 57%) (p=0.002).
Reporting of adverse events and harm-related endpoints stratified by the nature of the intervention
Discussion
Of the trials that we evaluated, 18% reported a DMC. This compares to 14% for paediatric trials published in 2005–20077 and 2% for those published in 1996–2002,8 according to earlier reviews. As children are a vulnerable population, some would suggest that all paediatric trials should be overseen by a DMC.10 Nevertheless, the decision whether to establish a DMC in a paediatric trial is dependent on various considerations (clinical, methodological and otherwise),15 most of which are not available in published reports. It is thus likely that a number of the trials in our sample did not require a DMC; however, it is encouraging that their establishment was more frequent among those that investigated drugs, vaccines and alternative therapeutic interventions compared with those that investigated behavioural, rehabilitation or psychosocial programmes. Notably, the reporting of a DMC was infrequent among trials that investigated surgeries, radiotherapy or devices where, especially in paediatric populations, their establishment may be warranted.
Less than half of the trials in our sample reported data on harms, a finding that compares to previous reviews of trials in specific topic areas. For example, Hum et al 10 noted suboptimal reporting of harms in paediatric trials of antibiotics for acute otitis media. Moreover, Leung et al 11 identified several methodological issues related to the identification and reporting of adverse events in paediatric studies of asthma medications. Incomplete reports of trials limit healthcare providers’ ability to make decisions based on consideration of both the benefits and risks of available treatments.24 25 We found that the reporting of adverse events was infrequent among trials that may be presumed to pose lesser risk (eg, communication, organisational or educational programmes and rehabilitation or psychosocial interventions); however, even in low-risk populations and putatively low-risk interventions, ‘the balance of harms and benefits may easily lean toward harm’.26
Of the 143 trials that did report data on harms, 36% reported severe harms. Moreover, of the 215 trials that reported on harm-related endpoints, 54% reported discontinuations due to adverse events. By contrast, none of the trials in our sample reported early stopping due to harms. Ethically, trials must stop early when the findings of interim analyses show that exposing participants to additional potential risk by participating in the trial is not justified.27 Thus, the occurrence of harms is not an indication to stop a trial, unless the accruing harms data show unreasonable risk from participation compared with the anticipated benefits.27 An important issue is that more than half of the trials we analysed did not report any data related to harms. Because it is not possible to uphold ethical standards for trial conduct if harms data are not collected and monitored, this likely reflects a reporting issue. Similarly, a review of adverse event reporting in published and unpublished reports of studies of healthcare interventions found strong evidence that much of the information on adverse events remains unpublished.28
Implications for research and practice
Many trialists cite inadequate knowledge and paediatric-specific methodological training as serious barriers to the rigorous conduct and reporting of trials involving children.29–31 Encouragement of prospective protocol publication, learning opportunities for trialists and trainees and the vigilant review of the reporting of DMCs and adverse events data by reviewers and editors of academic journals may contribute to improvements in conduct and reporting. As it was not feasible in this study to appraise the independency of members of the DMCs from trial sponsors or investigators (which is necessary to ensure unbiased monitoring), we cannot draw any conclusions regarding DMC conduct. Future studies may consider addressing this knowledge gap.
Strengths and limitations
We evaluated trials published in 2012, providing a baseline for ongoing evaluation of safety monitoring procedures in paediatric trials; however, the findings may not be reflective of present-day conduct and reporting. Moreover, because we investigated a random sample of trials, not all of the trials would have required a DMC. Nevertheless, ongoing evaluation of the state of the research is needed to evaluate changes over time and identify the areas in most need of attention. The random nature of our sample facilitated comparisons with previous studies, including a similar descriptive analysis of paediatric trials published in 2007.9
Limitations of our findings stem from our reliance on the data provided in published reports. Because the reporting of serious adverse events is a regulatory requirement for many clinical trials, it is likely that our findings represent reporting shortcomings. Moreover, we examined only whether adverse events were reported, not whether the adverse events investigated were appropriate or adequate. Because we sampled trials published in 2012 covering various conditions, interventions and outcomes, our findings may not be generalisable to trials of specific conditions or interventions, measuring specific outcomes or published in other years.
Conclusions
The reporting of a DMC was infrequent within our sample. It was more common among trials that investigated drugs, vaccines and alternative therapies, multicentre trials, industry-sponsored trials and those that enrolled larger samples. Adverse events data were reported in less than half of the trials, which has important implications for the ability of paediatric trials to inform clinical decision making.24 25 None of the trials in our sample reported early stopping due to harms.
Acknowledgments
We would like to thank Robin Featherstone (MLIS, University of Alberta, Edmonton, Canada) for running the search and Sarah Bryson, Huiru Dong, Aimee Gonzalez, Hamza Jafri, Karandeep Jassal, Megan Nuspl, Kassi Shave, Jocelyn Shulhan and Jaskiran Sidhu (research assistants at the Alberta Research Centre for Health Evidence, University of Alberta, Edmonton, Canada) for contributing to data extraction.
References
Footnotes
Contributors MPD conceptualised the study, designed the data collection instrument, oversaw all aspects of the work, verified and analysed the extracted data and contributed to drafting the manuscript. AG verified and analysed the extracted data, contributed to drafting the manuscript and revised the manuscript following input from the coauthors. PC, SC, LD, LH, LEK, RMF, KaW and KeW contributed to the interpretation of the extracted data and revised manuscript drafts critically for important intellectual content. BV contributed to the data analysis and interpretation of the extracted data and revised manuscript drafts critically for important intellectual content. All authors approved the manuscript as submitted and agree to be accountable for all aspects of the work.
Funding This work was supported by the Canadian Institutes of Health Research (#KRS 140989).
Disclaimer The funder played no role in the design and conduct of the study; the collection, management, analysis and interpretation of the data; the preparation, review or approval of the manuscript; or the decision to submit the manuscript for publication.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement The data collected and analysed from trials included in this study will be available to researchers via reasonable request from the corresponding authors. The data will be available immediately following and for 5 years after article publication. Our data extraction guide is available as a supplementary file.
Presented at Parts of this research were presented as a poster at our local research day (Pediatric Research Day, Department of Pediatrics, University of Alberta, Edmonton, Canada) on 16 May 2018 under the title: ’The safe and ethical participation of children in research: a descriptive analysis of the conduct and reporting of trials published in 2012'.
Patient consent for publication Not required.