Article Text
PDF
Abstract
Background An estimated 10%–24% of children attending emergency departments with a burn are maltreated.
Objective To test whether a clinical prediction tool (Burns Risk assessment for Neglect or abuse Tool; BuRN-Tool) improved the recognition of maltreatment and increased the referral of high-risk children to safeguarding services for assessment.
Methods A prospective study of children presenting with burns to four UK hospitals (2015–2018), each centre providing a minimum of 200 cases before and after the introduction of the BuRN-Tool. The proportions of children referred to safeguarding services were compared preintervention and postintervention, and the relationship between referral and the recommended cut-off for concern (BuRN-Tool score (BT-score) ≥3) was explored.
Results The sample was 2443 children (median age 2 years). Nurses and junior doctors mainly completed the BuRN-Tool, and a BT-score was available for 90.8% of cases. After intervention, 28.4% (334/1174) had a BT-score ≥3 and were nearly five times more likely to be discussed with a senior clinician than those with a BT-score <3 (65.3% vs 13.4%, p<0.001). There was no overall difference in the proportion of safeguarding referrals preintervention and postintervention. After intervention, the proportion of referrals for safeguarding concerns was greater when the BT-score was ≥3 (p=0.05) but not for scores <3 (p=0.60). A BT-score of 3 as a cut-off for referral had a sensitivity of 72.1, a specificity of 82.7 and a positive likelihood ratio of 4.2.
Conclusions A BT-score ≥3 encouraged discussion of cases of concern with senior colleagues and increased the referral of <5 year-olds with safeguarding concerns to children’s social care.
- child abuse
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
What is known about the subject?
Burns are common injuries to young children, and 10%–24% are reported to be associated with maltreatment (physical abuse or neglect).
Identifying which children with burns presenting to emergency departments (ED) are at risk of maltreatment is challenging.
Burns Risk assessment for Neglect and maltreatment in children Tool (BuRN-Tool), a clinical prediction tool for children’s burns, has been designed, developed, validated for use in ED.
What this study adds?
A prospective, multicentre study in EDs and burn units has demonstrated that the BuRN-Tool was accepted by clinicians and completed in 90.8% of cases.
Children with a BuRN-Tool score (BT-score) ≥3 were more likely to have safeguarding concerns discussed with senior colleagues and be referred for safeguarding assessment.
A BT-score ≥3 for referral had a sensitivity of 72.1, a specificity of 82.7 and a positive likelihood ratio of 4.2
Introduction
Every year, 40 000 children attend emergency departments (ED) in the UK with burns.1 Studies estimate that 10%–24% of these injuries are associated with maltreatment.2–5 Neglect arising from lack of supervision or inadequate safety provision in the home is more common than deliberate injury.2 Children less than 3 years old with a burn from any cause have been shown to be seven times more likely than controls to suffer maltreatment by their sixth birthday,6 and more likely to be hospitalised for unrelated healthcare needs.7
Identifying maltreatment in young children presenting to busy EDs is challenging given the high volume of patients, pressure to manage and discharge patients quickly and a high staff turnover with variable paediatric experience. This could be improved by a standardised injury assessment and a clinical prediction tool (CPT) to identify suspected maltreatment. However, systematic reviews have concluded that there were few high-quality published studies and no screening tools with sensitivities and specificities suitable for widespread introduction.8 9 The reviews suggested that injury-specific tools or scoring systems based on multiple factors performed better than the generic tools that aimed to identify broad categories of maltreatment.
Following Medical Research Council guidance10 and applying the highest quality methodological standards for the development of CPTs in EDs,11 we have designed, developed and evaluated a CPT for use in children’s burns. An evidence-based proforma (the Burns and Scalds Assessment Template; BaSAT) was designed to encourage a standardised clinical assessment of each child attending the ED with a burn or scald and to ensure that key variables relevant to the risk of maltreatment were recorded (online supplemental appendix 1). A CPT, the Burns Risk assessment for Neglect and maltreatment in children Tool (BuRN-Tool) (figure 1), was then developed to enable a score (BT-score) to be calculated from these key variables to quantify the risk of maltreatment. The BuRN-Tool was tested on a separate population of children presenting with burn injury12 with sensitivity and specificity of 87.5% and 81.5% for scalds and 82.4% and 78.7% for non-scalds.13 A feasibility study of the BuRN-Tool concluded that it is acceptable and valued in clinical practice.12
Supplemental material
The Burns Risk assessment for Neglect and maltreatment in children Tool (BuRN-Tool).
This prospective, multicentre study tests the efficacy of using the BuRN-Tool in routine clinical practice in EDs and burn units and aims to evaluate the recognition of maltreatment using the BaSAT and BuRN-Tool and subsequent actions based on the BT-score, namely discussion with colleagues and referral to safeguarding services for further assessment.
Methods
Study design
A prospective before-and-after design was adopted to test the intervention of the BuRN-Tool to identify potential maltreatment associated with burns. The BaSAT was introduced to record a standardised assessment of all children less than 16 years of age with a burn who attended four large EDs (two of which had colocated burn units) in England and Wales between 1 April 2015 and 31 March 2018. Victims of house fires were excluded. After each centre had collected a minimum of 200 consecutive cases (preintervention), the BuRN-Tool was added to the BaSAT and a further minimum of 200 consecutive cases then collected (postintervention). At each stage, the clinical teams were provided with standardised training on the use of the BaSAT and BuRN-Tool. Training was updated when staff changed and was available online to new staff throughout the study.
The primary outcome measures compared the proportion of children referred preintervention and postintervention to the hospital safeguarding team (HSG) or to children’s social care (CSC), and the relationship between the referrals made and the BT-score. The behaviour of clinicians when using the BuRN-Tool was measured by evaluating the extent to which clinicians completed the information on the BaSAT required to derive or calculate a BT-score, the proportion of cases discussed with senior colleagues and the agreement between the BT-score calculated by the clinician against retrospectively derived BT-scores by the study team based on information recorded within the BaSAT.
Setting
Three centres were paediatric EDs (centres A, B and D) and centre C was a general ED. Two regional burn centres in centres C and D were included. A fifth general ED was a control centre that used the BaSAT in hard copy format, and recorded the background safeguarding referral rate from ED without using the BuRN-Tool.
Data collection
The BaSAT includes information required to inform seven evidence-based risk factors for maltreatment (physical abuse or neglect) (online supplemental appendix 1). Centre B used an electronic version of the BaSAT (a clinical proforma in the MEDWAY Hospital Database with the BuRN-Tool (figure 1) embedded and switched on at intervention). The other three centres used a hard copy version of the BaSAT and BuRN-Tool.
The BT-score was calculated for each child from seven individually weighted risk factors for maltreatment (figure 1). The summative score could range from 0 to 12: a score of 0–2 equated to a low risk of a child maltreatment, and a BT-score of 3 or more suggested a risk of physical abuse or neglect. For BT-scores ≥3, the clinician was advised to follow the local child protection pathway for escalating their concerns, namely discussion with a senior National Health Service colleague or a referral to HSG or CSC for further assessment. In the centres using hard copies, the attending clinician calculated the BT-score. In centre B, the BT-score was calculated automatically once the BaSAT was completed. For comparison purposes, the research team retrospectively derived an equivalent BT-score preintervention from the seven recorded features on the BaSAT for each case.
The following outcomes were collected preintervention and postintervention: treat and discharge (no safeguarding pathway initiated), refer to HSG and refer to CSC. Postintervention discussions with senior colleague were recorded as a binary yes or no.
Data management
Each case was anonymised and appointed a unique study number. Study data from the BaSAT and BuRN-Tool were entered onto a Research Electronic Data Capture database.14
Data analysis
The completeness of data within the BaSAT that were relevant to inform the recording of the seven BuRN-Tool risk factors is provided in online supplemental appendix 2. The proportions of complete records to inform a derived BT-score (preintervention) or calculated BT-score (postintervention) were compared. We compared the derived BT-score with that calculated from the BuRN-Tool by the clinician (or generated electronically) postintervention using Krippendorff’s alpha which estimates the disagreement between the two. Values range from 0 to 1, where 0 is perfect disagreement and 1 is perfect agreement. Krippendorff suggests an alpha ≥0.667 is the lowest acceptable limit to draw tentative conclusions.15
Supplemental material
The proportions of cases referred to HSG or CSC preintervention and postintervention were calculated and compared. Data were analysed for preschool children (ie, those who have not yet reached their fifth birthday) where the determination of child maltreatment is often most challenging as children cannot give their own version of events and for those ≥5 years of age. All analyses were conducted in Stata V.1516 and comparisons were made using non-parametric statistical tests. Any analyses comparing variables preintervention to postintervention within centres used Mann-Whitney U tests for numerical variables or χ2 tests for categorical variables. To compare across centres within preintervention and within postintervention, we used Kruskal-Wallis tests for numerical variables and χ2 tests for categorical variables. Logistic regressions were used to calculate the ORs for referral to HSG or CSC (combined) for BT-scores <3 and ≥3 separately for preintervention (derived BaSAT score used) and postintervention (clinician score used).
Receiver operating characteristic (ROC) curve analysis was performed with the ‘reference standard’ being the total number of children referred to HSG and CSC (combined) against each BT-score value as a threshold (a threshold of 3 having been previously established as the most appropriate cut-off13). Area under the curve (AUC), sensitivity, specificity and likelihood ratios were obtained. The AUC shows the ability of the BT-score to distinguish between referral and no referral (ranging from 0 to 1, the higher the better).
Patient and public involvement
Patients were not involved in the study design due to their young age and the anonymous nature of the study. There was considerable involvement of stakeholders in the iteration and version control of the BaSAT and BuRN-Tool with associated qualitative studies evaluating their acceptability.12
Results
Sample
There were 2478 clinical records within the study. Thirty-five duplicate records within the same centre were excluded, leaving 2443 children from the four study centres (figure 2). The median age of children was 2 years preintervention and postintervention; however, the age profile of children did differ between centres both preintervention (p<0.001) and postintervention (p<0.001) (table 1). There was no difference in gender distribution across centres either preintervention (p=0.09) or postintervention (p=0.90) (table 1). Scalds were the most common burn type, but the proportion of scalds to contact burns varied between centres both preintervention (p<0.001) and postintervention (p<0.001).
Flow of the cases included in the study.
Age, sex, burn type and designation of ED clinician completing the BaSAT and BuRN-Tool before and after the intervention in each of the four study centres
The BaSAT and the BuRN-Tool were completed predominantly by nurse practitioners or the most junior doctors, except for centre D where the majority were completed by more senior trainee doctors (p<0.001). For centres A and B, the distribution of who completed the forms changed from preintervention to postintervention (table 1).
Data completeness
Before the introduction of the BuRN-Tool, the completeness of key variables on the BaSAT necessary to inform the risk factors and to derive a BT-score was good (>90%) across all centres except centre D, which had lower completion rates for burn severity (85.6%) and symmetry of a scald (61.4%) (online supplemental appendix 2); however, this improved significantly postintervention.
After intervention, a BT-score was recorded by clinicians for 90.8% of cases overall (table 2). In centre A, recording of key variables needed to inform a BT-score fell from 81.4% preintervention to 69.6% postintervention, but the high proportion of BT-scores calculated in this centre suggests that some clinicians made the calculation but did not record the data within the BaSAT (table 2). In centre C the reverse was true, key variables were completed in 87.7% of cases but the proportion of calculated BT-scores was lower (table 2). The most complete data both preintervention and postintervention were from centre B using electronic proformas. Completeness of referral outcome data ranged from 88.0%–99.0% preintervention to 92.0%–97.0% postintervention.
Proportion of completed fields on the BaSAT needed for the BuRN-Tool score (BT-score) preintervention and postintervention and proportion of records with a BT-score postintervention
Distribution of BT-scores preintervention and postintervention
There were 80.7% (928/1150) of cases with a derived BT-score preintervention and 90.8% (1174/1293) with a calculated BT-score postintervention. The distribution of BT-scores was very similar preintervention and postintervention and across the four centres with the majority of cases scoring 2 (figure 3). The proportion of cases scoring ≥3 preintervention and postintervention was no different (p=0.15). Before intervention, 31% (291/928) had a derived score ≥3 and after intervention, 28.4% (334/1174) had a clinician score ≥3.
Distribution of BuRN-Tool scores (BT-scores) preintervention (n=928) and postintervention (n=1174). Sample size (only those with complete data on scores) in each centre preintervention: A=171, B=373, C=211, D=173, and postintervention: A=372, B=391, C=194, D=217. BuRN-Tool, Burns Risk assessment for Neglect and maltreatment in children Tool.
Accuracy of clinician BT-score postintervention
When comparing the BT-score calculated by the clinician with a BT-score derived by the research team from the key variables in the BaSAT postintervention (table 3), the agreement as measured by Krippendorff’s alpha was 0.85. Agreement was higher for those older than 5 years of age (0.86) than those <5 years of age (0.67). The automated system in centre B resulted in better agreement than those relying on hard copy versions (table 3).
Agreement between BuRN-Tool scores (BT-score) calculated by clinician and BT-score derived from BaSAT postintervention
Relationship between BT-score and discussion with senior colleagues
Cases with a BT-score ≥3 were nearly five times more likely to be discussed with a senior colleague compared with those with a BT-score <3 (65.3% vs 13.4%, p<0.001) (table 4). The proportion of ≥3 cases discussed with seniors was greatest in centres B and C where more junior clinicians or nurses completed the BuRN-Tool (table 1).
The number and proportion of cases discussed with a senior colleague postintervention according to BuRN-Tool score (BT-score)
Referrals to CSC or HSG
Although there was no overall difference in the total safeguarding referral proportion (CSC+HSG) from preintervention to postintervention, for children <5 years of age, referrals to CSC increased from 4.8% to 7.7% (p=0.02) with referrals to HSG falling from 6.0% to 3.7% (p=0.04) (table 5A). Centre C had the highest percentage of total (CSC+HSG) referrals (table 5A).
Number and proportion of referrals to children’s social care (CSC) or hospital safeguarding teams (HSG)
After intervention, the proportion of referrals for safeguarding concerns to CSC or HSG was greater in all centres when the BT-score was ≥3 (p=0.05) but not for scores <3 (p=0.60) (table 5B). After intervention, the overall OR (95% CI) of total referral to HSG or CSC across centres for clinician-calculated BT-score ≥3 compared with <3 was 12.37 (95% CI 7.92 to 19.32). Before intervention, using the derived BaSAT score, the same OR was lower at 7.28 (95% CI 4.47 to 11.85). Overall, for scores ≥3, CSC referrals increased from 11.0% preintervention to 21.2% postintervention (p=0.002) whereas the HSG referrals remained similar at 12.4% and 10.2% (p=0.42).
Data collected from the control centre for the years 2016–2018 (n=707) showed that for children <5 years of age (n=401), safeguarding referrals (CSC+HSG) increased from 8.4% to 16.6% (p=0.08), and for children ≥5 years of age (n=306), there was an increase from 6.6% to 12.5% (p=0.13). However, neither reached statistical significance.
ROC curve analyses postintervention
As established in our previous study,13 the ROC curve analyses confirmed that a cut-off BT-score of 3 corresponded to the optimum sensitivity (72.1 (62.8–80.2)) and specificity (82.7 (80.3–85.0)) for safeguarding referral postintervention, maximising the identification of suspected maltreatment cases. Overall predictive accuracy (AUC) was 0.81 (0.79–0.83) postintervention (considered good), with some variation across centres (figure 4). A cut-off score of 3 gave a positive likelihood ratio of 4.2 and a negative likelihood ratio of 0.3 (figure 4).
Receiver operating characteristic (ROC) curve for sensitivity and specificity of the BuRN-Tool score (BT-score) against safeguarding referral postintervention. AUC, area under the curve; BuRN-Tool, Burns Risk assessment for Neglect and maltreatment in children Tool; LR+, positive likelihood ratio; LR−, negative likelihood ratio.
Discussion
This multicentre, prospective study of the impact of a CPT specifically designed to identify maltreatment associated with burn injury in young children has confirmed that for children <5 years of age, the use of the BuRN-Tool improved the identification of risk of maltreatment and increased the referral of those at high risk (BT-score ≥3) to safeguarding teams. The use of the BuRN-Tool with children ≥5 years of age did not change referral patterns significantly.
The centre that used electronic records had significantly better completion rates and agreement between risk factors recorded on the BaSAT and BT-score given by clinicians than those using hard copy records where some clinicians calculated the BT-score without completing all the fields on the BaSAT. Building the BuRN-Tool into healthcare records that automatically generate the BT-score therefore improved the efficacy of the CPT.
The BuRN-Tool raised maltreatment concerns for an estimated 1 in 3 children across all settings which is a higher than previous prevalence rate quoted.2–5 The spread of the BT-scores was similar preintervention and postintervention and across centres despite differences in completion rates, burn types seen, burn service location and the level of staff experience who completed the BaSAT and BuRN-Tool. These findings suggest the BuRN-Tool works consistently in different settings and provides a practical threshold for raising concern of maltreatment and discussing or investigating a case further.
The threshold for concern for maltreatment (BT-score ≥3) was confirmed by the ROC curve analysis against safeguarding referral. Although not directly comparable as this study did not split the analysis by burn type, the sensitivity was lower than that reported in the derivation and validation study13 for both scalds (87.5) and non-scalds (82.4), specificity was slightly higher than previously (scalds: 81.5, non-scalds: 78.7) and AUC (0.81) was slightly lower (scalds: 0.87, non-scalds: 0.85). This variation may be due to the large number of different clinicians of different grades using the CPT.
The cases with a calculated BT-score ≥3 were five times more likely to be discussed with a senior clinician in ED, particularly in centres where the majority of cases were assessed by a nurse or junior doctor. These findings suggest that the BT does support the clinician in taking the right actions towards safeguarding the child and is consistent with qualitative evidence collected from interviews with ED staff,12 suggesting that the BT-score is most useful in helping less experienced clinicians assess safeguarding risk associated with a paediatric burn. The majority of burns related to child maltreatment arise from neglect rather than physical abuse with an estimated ratio of 9:1,3 and the clinical team must undertake an evaluation which contrasts with the more forensic evaluation of children who present with other traumatic injuries (such as bruising or fractures) with which ED clinicians are more familiar. Burns from neglect often arise from inadequate levels of supervision, a subjective judgement that can be difficult to make. The BuRN-Tool directs the clinician to consider and make an objective assessment of supervision as described in figure 1.
Although the overall percentage of safeguarding referrals increased in the control centre at the same time period consistent with the recent temporal trend for referrals to CSC,17 this increase did not reach statistical significance. A higher percentage of referrals in the study centres was only seen in those less than 5 years of age who were recognised as high risk with a BT-score ≥3, suggesting that the BT-score enabled a more specific type of referral. A low BT-score <3 appeared to reassure clinicians of a low safeguarding risk. The nature of referrals and actions taken by CSC is discussed in detail in a forthcoming publication.
There are a number of CPTs in the maltreatment field, particularly in the identification of abusive head trauma, yet only few have been validated in clinical practice.18–20 In the Netherlands, generic screening checklists for child maltreatment (SPUTOVAMO and Child Abuse Inventory at Emergency Rooms (Chain-ER)) have been used in EDs with high-associated false-positive rates.21 22 The utility of the SPUTOVAMO checklist in children’s burn presentations was assessed on admissions to a large specialist burns unit,23 but this retrospective analysis was not able to show how the SPUTOVAMO influenced clinician behaviour. In comparison, the present study was prospective, and captured clinicians’ actions with and without the CPT. It is one of few studies that have attempted to prospectively measure the impact of a CPT for child maltreatment in clinical practice.
The strengths of this study are: that the BuRN-Tool has been derived from evidence specific to maltreatment associated with paediatric burns; was previously validated; and the efficacy of the tool has now been tested in large EDs and regional burn units in four centres in the UK with a wide range of clinical staff of different grades and experience. The completion rates of BaSAT were high, any selection bias or children missed may have been incurred by locum or temporary staff who did not access training in the use of the BaSAT or BuRN-Tool. However, ascertainment rates across centres were estimated as 85% of children seen during the study period. The primary outcome measure for this study is based on referrals of suspected maltreatment and does not evaluate the appropriateness of the referral, and follow-up data on the referrals collected from CSC teams are being published elsewhere.
In conclusion, this evaluation of a CPT specific to the identification of maltreatment associated with paediatric burns has shown that the BuRN-Tool is generally completed correctly, the efficacy of the BaSAT and BuRN-Tool was significantly better when combined into the child’s electronic medical record. The recommended cut-off for concern (BT-score of 3) gave acceptable sensitivity and specificity for referral. A BT-score ≥3 encouraged discussion of cases of concern with senior colleagues and increased the referral of young children less than 5 years of age with safeguarding concerns to CSC.
Acknowledgments
We thank the lead ED clinicians in each research centre, Dr Mark Lyttle, Dr Zoe Roberts, Dr Stuart Hartshorn, Dr Kirsty Dixon Jardine and Professor Tom Potokar, the research nurses Karen Coy, Grainne O’Connor, Amanda Cook, Louise Rogers Steve Gregory for database management and Dr Emma Johnson for her help with literature search.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @veritybennett3, @BristolCCAH
Contributors AK: conceived the study design, sought funding, supervised the research team and the analysis and was one of the four main cowriters. AME: main collaborator on the project, participated in study design, monitoring and supervision, cowrote the manuscript. DN: data collection, coordination of the multicentres involved. LH: cowriter and undertook the statistical analysis. CVB: organised the data extraction and data cleaning, set out the paper outline and order of the results section, drafted the original tables for the results and helped with the writing and approval of the penultimate draft. LH and CVB’s input into the paper was equal and should be considered as joint first authors of the manuscript.
Funding This work was supported by Health Care Research Wales (grant number RfPPB-16a-1262) with thanks to The Scar Free Foundation who funded the Children’s Burns Research Centre from 2012 to 2017 that underpinned this research.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval Approval for waived consent was given by the Confidentiality Advisory Group (CAG 1-06 (PR7)/2013) as it was unlikely that consent would be given by those parents who were of interest. The study is based on phase 2 diagnostic accuracy (before and after study design) (MREC No: 15_WA_0259;15/CAG/0203). Ethics approval date: 29 September 2015.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available upon reasonable request. Data are available from corresponding author on reasonable request.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.