Article Text
PDF
Abstract
Background Early-onset neonatal sepsis (EONS) remains an important disease entity due to very serious adverse outcomes if left untreated. Lack of diagnostic tools in identifying healthy from diseased neonates, and clinicians’ fear of the missing positive-culture sepsis babies, or babies with clinical sepsis have led to overtreating and unnecessary antibiotic exposure. Kaiser Permanente EONS risk calculator is an internally validated tool that can predict EONS. This sepsis risk calculator (SRC) classifies neonates into three subgroups: (1) ill-appearing, (2) equivocal and (3) well-appearing. We propose a modification to this tool that aims to use it solely for well-appearing babies. This modification represents a more conservative approach to decrease antibiotic exposure and offers an alternative for those hesitant to fully implement this tool.
Methods This is a dual-centre retrospective study where data were extracted from the electronic medical records. Our primary outcome was to validate the modified use of the SRC with a two-by-two table. Specificity, negative predictive value and expected antibiotic reduction were used to evaluate the tool’s feasibility.
Result Among 770 babies suspected of EONS, the feasibility of the modified use was tested. The expected antibiotic exposure reduction rate on the modification was 40.4% overall. The proposed modification resulted in a specificity and negative predictive value of 99.28% (95% CI: 97.92% to 99.85%) and 99.5% (95% CI: 99% to 99.8%), respectively.
Conclusion The modified use of the sepsis risk calculator has shown that it can safely reduce antibiotic exposure in well-appearing babies. The modified use is used as a ‘rule out’ test that can identify very low risk of EONS babies, and safely minimise antibiotic exposure. Further prospective studies are needed to examine the efficacy of this use, and quality improvement projects are required to evaluate its applicability in different clinical settings.
- Neonatology
Data availability statement
Data are available upon reasonable request. Data collected and analyzed in this study will be available upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Kaiser Permanente early-onset neonatal sepsis risk calculator has been well used in many centres across the USA. The implementation of the calculator in the UK and other centres across the world has been limited. One of the important limitations of the calculator is missing babies with a negative blood culture but who are clinically ill.
WHAT THIS STUDY ADDS
This study suggests a modified use of the original calculator. This modification aims to decrease the risk of missing clinically ill babies and to use this risk prediction model for solely well-appearing babies.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
The suggested modified use of the sepsis risk calculator provides an alternative to implementing the original calculator. For institutions that are hesitant to adopt the calculator and doubt its efficacy, this modified use may be implemented as a first step to decrease antibiotic exposure. The expected reduction in antibiotic exposure is similar to the studies that implemented the original calculator. The modified use of Kaiser Permanente early-onset sepsis risk calculator may be implemented in practice and tested to independently rule out early-onset sepsis in low-risk neonates.
Introduction
Early-onset neonatal sepsis (EONS) is caused mainly by the dissemination of bacterial agents from the mother’s genitalia during delivery.1 The most common causative agent is group B Streptococcus (GBS), while Escherichia coli is the leading cause of mortality.2 The incidence of EONS in Arab countries is 1.5 per 1000 live births.3 Another study conducted in Saudi Arabia showed the incidence of neonatal septicaemia to be 4.9 per 1000 live births.4 In addition, a 15.2% mortality rate from neonatal sepsis was attributed to EONS in Riyadh, Saudi Arabia.5
EONS diagnosis is still extremely challenging for clinicians, and up-to-date laboratory tests are neither sensitive nor specific for setting up a definitive diagnosis.6 Therefore, clinicians tend to start broad-spectrum empirical antimicrobial treatment for suspected cases of EONS, which is generally limited once the agent is identified. This has resulted in the excessive use of antibiotics, despite the decrease in the incidence of EONS.7–9 The risk-benefit ratio for antibiotic exposure must be taken into consideration, as the treatment of uninfected neonates will only assume risks without benefit. A study revealed that 11% of infants were treated empirically, while only 0.04% had confirmed infection with positive blood culture.7 Moreover, it has been suggested that transient bacteraemia could be attributed to 20%–30% of neonates with early positive blood cultures, which leads to an overestimation of sepsis risk.10
Early unnecessary exposure to antibiotics is a major contributor to antimicrobial resistance and it impacts the risk of being overweight in later childhood, developing asthma and allergies at 6 years of age, and having autoimmune disorders.11–15 On the other hand, over-investigation and overuse of empirical antibiotics remain concerns in EONS management.16 One way to optimise antibiotic use in babies suspected of EONS that is currently in practice is the Kaiser early-onset sepsis calculator developed by the Kaiser Permanente health system in California, which provides an estimated risk of sepsis among newborns.17 The sepsis risk calculator (SRC) is an online tool used in newborns ≥34 weeks of gestation that integrates maternal risk factors and clinical findings to assess the need for antibiotic use and neonatal intensive care unit (NICU) admissions among suspected neonates.18
The adoption of the calculator led to reduced diagnostic testing and empirical antibiotic treatment.10 Currently, the SRC is well used and is the standard of care in Kaiser Permanente hospitals across the USA. Two systemic reviews have evaluated the EONS calculator separately, and although there is evidence that the implementation of this tool will decrease antibiotic exposure, a large proportion of cases were missed.16 19 Thus, it is still challenging for many centres to adopt this tool, including our centres. As an institution that is very liberal with antibiotic treatment when approaching babies suspected of early-onset neonatal sepsis (EONS), we desired to implement a practice that aims to decrease antibiotic exposure but with limited risk. In this study, we aimed to propose a modification to the SRC which may serve as a bridge toward antibiotic optimisation that we can rely on without the fear of leaving babies with ‘clinical sepsis’ untreated. Although it may not be very different from the original use of the calculator, it can offer an alternative to those who are hesitant to fully use the tool. This modified use aims to use the SRC solely for the well-appearing group (low-risk), thus, decreasing antibiotic exposure and limiting the number of false negative cases. This approach can be perceived as a more conservative approach than the original use of the SRC. However, it is not expected to be superior in terms of antibiotic optimisation.20 Sepsis risk calculators are infrequently employed in many regions of the world, and this is in part because they have not been validated in these contexts so clinicians are reluctant to make decisions using them. Here, we suggest a pragmatic use of such a calculator, focusing just on well-appearing infants as a way of introducing the concept to new clinicians and lowering the barriers to use the SRC. By this approach, the calculator may be more applicable to be implemented in similar clinical settings. The primary outcome of interest was to estimate the reduction in the antibiotic exposure rate if the tool was to be implemented based on our proposed modification, which is targeting solely well-appearing babies.
Methods
Study design, area, and settings
This was a retrospective cohort study from 2018 to 2022 in which we validated the Kaiser Permanente neonatal early-onset sepsis calculator, which was developed to guide antibiotic treatment and neonatal intensive care unit admission for neonates ≥34+0 weeks of gestational age. The study was conducted in two tertiary centres. The authors collected the data using a data collection sheet through electronic medical records from both centres. Subjects eligible for the calculator were subsequently entered through the tool website. (Available at https://neonatalsepsiscalculator.kaiserpermanente.org/).17
Study population
All neonates evaluated for early-onset sepsis who were admitted to be investigated were screened. The inclusion criteria for individuals were as follows: (1) neonates ≥34+0 gestational age, (2) who were inborn and (3) who received antibiotic treatment within the first 24 hours of life due to suspected EOS after cultures were taken. The exclusion criteria were those who had major anomalies, as defined by the Vermont-Oxford Neonatal Network, and neonates who were born outside of the institution (outborn).21
Sample size and sampling technique
We used an online software sample size calculator. Based on unit audits, the rate of antibiotic exposure in all deliveries is approximately 4%. Considering a 50% expected reduction in antibiotic exposure, a CI of 100 (1-alpha) of 95% and a power of 80%, the minimum sample size was 630 babies to detect a significant difference in antibiotic exposure.22 The sampling technique was clustered random sampling based on the centres involved.
Clinical data and measurements
Detailed information, including demographic characteristics, the SRC variables, mode of delivery of antibiotics, antibiotic exposure within the first 24 hours of birth, NICU admission, and blood and cerebrospinal fluid (CSF) culture results, was collected from the patients’ electronic medical records. Neonatal EOS was defined as the isolation of bacterial pathogens in a positive culture of blood and/or CSF. The incidence of early-onset neonatal sepsis was calculated based on culture results.
Algorithm explanation and details
The sepsis calculator developed by Kaiser Permanente North California is a risk-based prediction model for managing neonates at risk of early-onset sepsis. The SRC uses maternal risk factors (prolonged rupture of membrane, antepartum maternal temperature, GBS test, type and timing of antepartum antibiotics), gestational age, institution’s incidence of EOS and clinical examination status of the baby to estimate the risk of EOS.17 The calculator then provides a clinical recommendation suggesting whether antibiotic treatment should be started. The clinical examination status of the babies was classified as follows: (1) well-appearing; (2) no persistent physiologic abnormalities; (2A) persistent physiologic abnormality >4 hour [tachycardia (HR >160); tachypnea (RR >60); temperature instability (>100.4 °F or <97.5 °F); respiratory distress (grunting, flaring or retracting) not requiring supplemental O2]; (2B) two or more physiologic abnormalities lasting for >2 hours [tachycardia (HR >160); tachypnea (RR >60); temperature instability (>100.4 °F or <97.5 °F); respiratory distress (grunting, flaring or retracting) not requiring supplemental O2]; and (3) ill-appearing; persistent need for nasal continuous positive airway pressure (NCPAP)/high flow nasal cannula (HFNC)/mechanical ventilation (outside of the delivery room), haemodynamic instability requiring vasoactive drugs, seizures, Apgar score @ 5 min <5 or a need for supplemental O2>2 hours to maintain oxygen saturations >90% (outside of the delivery room).
The clinical recommendations for treating SRCs include (a) no culture or no antibiotic treatment, (b) culture but no antibiotic treatment, (c) empirical antibiotic treatment and (d) strong consideration of empirical antibiotic treatment. The calculator also provides clinical recommendations about the frequency of vital sign monitoring and NICU admission. A baby could be recommended to be (1) monitored routinely, (2) monitored every 4 hours for 24 hours or (3) admitted to the NICU for continuous monitoring. For the calculator to be implemented to fit the local context, we considered the calculator recommendations (c) and (d) as a fulfilling determinant to initiate antibiotic treatment when calculating the specificity and negative predictive value.
The proposed modification
Once a baby is suspected to have EOS, they should be classified based on their clinical examination as (1) well-appearing, (2) equivocal and (3) ill-appearing. If the baby falls into the well-appearing category, the rest of the fields on the calculator website should be filled out. If the recommendation of the SRC was not to initiate antibiotic treatment, the baby should be monitored for at least 48 hours, observed frequently, and reassessed when needed. If the calculator recommendation was to give antibiotics, antibiotics should be immediately given. For babies who are classified as equivocal or ill-appearing, treatment should be based on physician decision or unit policy and the calculator recommendation should be disregarded. Nevertheless, it should be noted that the ultimate diagnosis of EONS depends on the culture result and that the outcomes of the calculator classification are not a diagnosis. This tool and the proposed modification in this study are only meant to guide initial antibiotic treatment, whereas the culture result, once available, is the standard to guide further antibiotic treatment.
Validation parameters
The neonates eligible for use by the SRC were classified into two groups. Well-appearing babies constituted group A (low risk), and both equivocal and ill-appearing neonates were combined to constitute group B (high risk). In group A, those whose recommendations by the calculator were not to initiate antibiotic treatment and who had a negative or positive culture were true negative and false negative, respectively. In contrast, those whose recommendations by the calculator were to initiate antibiotic treatment and had negative and positive cultures were false positives and true positives, respectively. Group B babies, by just being categorised by the SRC as high risk (equivocal or ill-appearing), were considered all positive regardless of the culture result because they are perceived clinically ill. If they received antibiotic treatment within the first 24 hours, they were considered true positive (TP), and if they did not, they were considered false positive. See figure 1.
Domains of the two-by-two table for calculating the specificity and the negative predictive value.
Statistical analysis
For descriptive statistics, mean±SD was used to represent continuous variables. Percentages were used to describe categorical variables. The chi-square test and independent t-test were used to examine an association between different variables. Specificity and negative predictive value were calculated using a standard 2×2 table after combining all of its domains23 (see figure 1). Expected antibiotic rate reduction was calculated as the difference between the overall current antibiotic exposure rate and antibiotic exposure rate if the SRC is to be implemented as proposed with the modification. Data management and analysis were carried out using the Statistical Software for Social Sciences (SPSS) V.25.24
The proposed method to use the sepsis risk calculator.(figure 2)
Result
Out of 69 836 deliveries during the study period, 2618 (3.7%) neonates were screened. Excluded subjects were due to gestational age <34 weeks (n=212), not investigated for EONS or with missing data (n=1540), and congenital anomalies (n=96). Eventually, 770 neonates were recruited to evaluate the modified use Kaiser Permanente early-onset neonatal sepsis risk calculator. The overall incidence of positive culture EONS was estimated to be 0.7/1000 live births. Excluding neonates who did not meet the calculator criteria, the definitive positive culture EONS incidence was 0.3/1000 live births.
Among 770 neonates, 403 (52.3%) were males, 722 (93.8%) were term and 367 (47.7%) were admitted to the NICU. The average birth weight of all babies was 3 kg±0.5. Most babies were delivered by normal spontaneous vaginal delivery (74.4%) and had a negative maternal GBS test. Six babies were diagnosed with positive culture EONS. Causative organisms were coagulase-negative staphylococcus (n=2), Escherichia coli (n=1), GBS (n=1), Haemophilus influenzae (n=1) and Staphylococcus haemolyticus (n=1). Most babies were exposed to antibiotics during the first 24 hours of life (76.4%). Two positive cultures EONS cases were well-appearing (H. influenzae and coagulase-negative staphylococcus), whereas the remaining four cases were equivocal (S. haemolyticus) and ill-appearing (coagulase-negative staphylococcus, Escherichia coli and GBS) (see table 1).
Characteristics of study subjects and their association with high-risk and low-risk groups
Babies were classified as well-appearing (low risk) (n=419) and equivocal and ill-appearing (high risk) (n=351). Late preterm neonates, babies who were admitted to the NICU and babies who were born by emergency caesarean section were more likely to be in the equivocal and ill-appearing group (p value <0.001). Mother who had prolonged rupture of membrane and positive GBS test were more likely to have their babies well-appearing (p value <0.001) (see table 1).
The specificity of the calculator was 99.28% (95 CI: 97.92% to 99.85%) and a negative predictive value of 99.5% (95 CI: 98.8% to 100%) with two false negative cases. If the calculator had been implemented, it would have resulted overall in a 40.4% reduction in antibiotic exposure (from 76.4% to 45.5%), and an 11.7% reduction in NICU admissions (from 47.7% to 42.1%) (see table 2). Overall, 588 babies (76.4%) received antibiotics within the first 24 hours of life, and 367 babies (47.7%) were admitted to the NICU. Whereas, the calculator recommended initiation of antibiotic treatment and NICU admission for 324 babies (42.1%). In low-risk and high-risk babies, 239 (57%) and 349 (99.4%), respectively, received antibiotic treatment. On the other hand, the calculator recommended antibiotic treatment initiation in one baby in the low-risk group and 323 babies (92%) in the high-risk group (see table 3).
Validation parameters
Calculator recommendations and actual actions taken correspondence based on culture result
Discussion
The SRC is currently employed for well-appearing, equivocal and ill-appearing babies. However, we conducted a feasibility test by using the tool’s recommendation solely for well-appearing babies, while disregarding it for equivocal or ill-appearing babies. This approach is aimed at optimising the use of the tool for well-appearing neonates. While this partial application is not anticipated to surpass the complete use of the tool theoretically, it is expected to be more beneficial than not employing any tool at all. Although the SRC was not validated as practiced by its developers, the expected decrease in unnecessary antibiotic use was very similar to previous studies in the literature that ranged approximately from 55% to 40%.25–29 We aimed to use the calculator to identify healthy babies and found that the outcomes of the specificity and the negative predictive value reflected applicable and very promising numbers. These numbers are based on our proposed method of using the SRC and they underscore the significance of risk stratification tools in enhancing neonatal care and resource utilisation. However, further prospective studies are needed to validate the effectiveness of the SRC across various clinical environments and its potential influence on patient outcomes, based on our proposed method to use the calculator.
The complete implementation of the tool remains a challenge, with many neonatologists expressing concerns that such a tool could overlook babies with positive-culture sepsis.30 Another obstacle to implementing such risk stratification tools is the lack of data and resources in low-and-middle-income countries (LMIC) settings.16 Therefore, the proposed partial implementation of the SRC, as described in this study, could potentially boost clinicians’ confidence, and encourage wider application of this tool. This may be expected as we were more conservative by disregarding the babies in the equivocal group and consequently had a lower threshold to miss unhealthy babies. Regardless, we would still monitor all suspected babies for at least 48 hours. See figure 2.2
While the primary objective of implementing risk stratification strategies is to minimise antibiotic exposure, the more important principle is not to overlook babies with EONS who need antibiotic treatment.10 The main concern driving the hesitation to implement these tools is the fear of overlooking culture-positive babies, as evidenced by the two false negative cases in this study.30 31 However, these cases were not clinically significant. The first baby, diagnosed with H. influenzae, did not show symptoms until the second day of life, at which point antibiotic treatment was initiated. The second baby, who had coagulase-negative staphylococci, remained asymptomatic throughout his NICU stay and was only treated when the culture result came back positive. Most well-appearing babies who received antibiotic treatment did so due to the absence of a clear unit policy. The decision to initiate antibiotic treatment in our centres is typically made by the physician assessing the baby, and it is often influenced by factors such as transit tachycardia, tachypnoea or signs of respiratory distress. However, we strictly followed the definition of the SRC classifications, and those who were defined as well-appearing were not necessarily asymptomatic. This is reasonable because transit physiological abnormalities in newborns can be due to the transition to extrauterine life and are not necessarily manifestations of clinical sepsis.26 Moreover, some neonates might have received antibiotic treatment due to being resuscitated with some respiratory assistance at birth or being born to a mother suspected of chorioamnionitis.
It is not novel that positive-culture EONS can be asymptomatic at birth, and further evaluation is promoted once the baby is symptomatic. In fact, out of 24 babies at baseline in the study where the tool was originally developed, 50% (n=12) of babies with positive culture EONS were not symptomatic at birth, of which, 8 babies were never symptomatic throughout their NICU stay.26 This suggests that excessive antibiotic exposure for any symptomatic baby would not eliminate the possibility of missing a baby with positive-culture sepsis, because there is still a chance that a baby has sepsis but is asymptomatic. Hence, the practice of overtreatment that we adopt, and maybe other centres as well, is not reasonable because it did not result eventually in zero cases of positive-culture EONS. Therefore, using a risk stratification strategy that potentially narrows the chance of missing a baby who has early-onset sepsis is very critical for neonatologists and institutional policymakers to adopt.32
The aim of our study was not only to validate this tool and to advocate for its use but also to encourage other centres and clinicians in our region to adopt strategies that work toward decreasing antibiotic exposure in late preterm and term infants. Hence, we did not adopt the calculator completely and modified its use. This is because babies who are classified as equivocal will be at least tachypnic, tachycardic or in respiratory distress for a minimum of 2 hours, and it is very difficult to have those babies not treated after being observed if the calculator recommended otherwise. Applying the calculator in such a way ensures even a narrower chance of missing a culture-positive EOS baby. However, it is critical to emphasise that calculator recommendations are not diagnostic, and the culture result for all babies should followed. To transition from a practice where many healthy babies do receive unnecessary antibiotic treatment, to a practice where antibiotic treatment is very optimised, it is important to navigate between the two practices with caution and persuasiveness. This is why we tested the calculator’s feasibility exclusively for well-appearing neonates. However, our findings are limited by the study’s design and randomised clinical trials are needed to affirm our hypothesis and reproduce similar or contrasting results. Also, the study’s findings cannot be generalised for other centres, since the number of positive EOS cases did not reach 100 cases, which is the minimum required number of cases to evaluate the effectiveness of a prediction model.33
Conclusion
This research presents a new method to use the Kaiser Permanente early-onset neonatal sepsis risk calculator by optimising its use solely for babies who are categorised as well-appearing based on the SRC classification. We found that this proposed method can predict healthy neonates and safely rule out sepsis, although further observation and monitoring should be warranted. The rate of the expected reduction in antibiotic exposure was more than desired, without an increase in mortality or morbidities. Adopting this approach is expected to decrease the incidence of false negative cases and may be more conservative when further evaluation of a suspected case of sepsis is warranted. Centres and institutions that lack the infrastructure for data retrieval and do not have available allocated resources to implement risk stratification strategies may evaluate this approach as a quality improvement project. Further prospective studies are needed to measure the direct impact and clinical implication of our recommended approach.
Data availability statement
Data are available upon reasonable request. Data collected and analyzed in this study will be available upon reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was carried out in compliance with the ethical standards established in the Declaration of Helsinki (as revised in 2013). The informed consent requirement for this study was waived owing to its retrospective nature. The study was approved by the Institutional Review Board (IRB) committee at King Abdullah International Medical Center (KAIMRC) (IRB/0922/22). Due to the nature of the study design of the study, the process of obtaining an informed consent was waived.
References
Footnotes
Contributors MYA and SAS conceived and designed the study, and revised the initial data collection sheet, and final manuscript. MHA and AA conducted the research, provided research materials, collected and organized data, and contributed to manuscript writing. AM and MAA interpreted the data and contributed to the initial manuscript. FGA collected the data, analysed the data, contributed to the manuscript writing, and revised the initial and final manuscript. All authors have critically reviewed and approved the final draft and are responsible for the content and similarity index of the manuscript. MYA is the guarantor who accepts full responsibility for the work and/or the conduct of the study, had access to the data, and controlled the decision to publish.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.