Article Text
PDF
Abstract
Background and objectives Early diagnosis of neonatal infection is essential to prevent serious complications and to avoid unnecessary use of antibiotics. The prevalence of healthcare-associated infections (HAIs) among very low birthweight (VLBW; <1500 g) infants is 20%; and the mortality in low-resource settings can be as high as 70%. This study aimed to develop an Infection Prediction Score to diagnose bacterial HAIs.
Methods A retrospective cohort of VLBW infants investigated for HAI was randomised into two unmatched cohorts. The first cohort was used for development of the score, and the second cohort was used for the internal validation thereof. Potential predictors included risk factors, clinical features, interventions, and laboratory data. The model was developed based on logistic regression analysis.
Results The study population of 655 VLBW infants with 1116 episodes of clinically suspected HAIs was used to develop the model. The model had five significant variables: capillary refill time >3 s, lethargy, abdominal distention, presence of a central venous catheter in the previous 48 hours and a C reactive protein ≥10 mg/L. The area below the receiver operating characteristic curve was 0.868. A score of ≥2 had a sensitivity of 54.2% and a specificity of 96.4%.
Conclusion A novel Infection Prediction Score for HAIs among VLBW infants may be an important tool for healthcare providers working in low-resource settings but external validation needs to be performed before widespread use can be recommended.
- neonatology
- epidemiology
Data availability statement
Data are available upon reasonable request. Anonymised data can be made available upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Healthcare-associated infections (HAIs) occur frequently in very low birthweight (<1500 g) infants; however, the diagnosis of these infections is difficult, especially in low-resource settings where access to laboratory facilities may be limited.
Diagnostic uncertainty contributes to increased use of empirical antimicrobial therapy.
WHAT THIS STUDY ADDS
A novel Infection Prediction Score developed in a low-resource setting that achieved similar diagnostic performance to existing scores developed in high-resource setting.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
Further research must include the external validation of the score, before the use of the score can be recommended outside of the research setting.
Prediction scores for HAIs can optimise antibiotic stewardship by reducing duration and exposure to empirical antimicrobial therapy.
Introduction
Healthcare-associated infection (HAI) is a frequent infectious complication in neonatal units worldwide, affecting approximately 20% of very low birthweight (VLBW; <1500 g) infants.1–3 In lower-middle income countries, for example, Brazil and Indonesia, the prevalence of HAI has been reported to be 51%–52%.4 5 In sub-Saharan Africa (SSA), HAI-associated mortality rates of 27%–72% have been observed6 7 with HAI survivors frequently suffering long-term complications of infection-related systemic inflammatory response, for example, cognitive impairment and cerebral palsy.8
Timely and accurate diagnosis of HAI is important, but the current diagnostic gold standard (blood culture), time consuming, prone to false negatives and positives and has a low yield of 5%–10%.9–12 The clinical symptoms of HAI in neonates are non-specific and overlap with non-infectious conditions such as respiratory distress syndrome,10 complicating HAI diagnosis in neonates. The diagnostic accuracy of haematological tests and acute inflammatory markers such as C-reactive protein (CRP) have been questioned.13 14 Newer tests, for example, CD64, procalcitonin, presepsin and interleukin-6 have been assessed for HAI diagnosis,15 but are expensive and not readily available in low-resource settings (LRSs), where neonatal HAIs occur commonly. Given the challenges and diagnostic uncertainty of neonatal HAI diagnosis, physicians in LRSs have a low threshold for prescribing antimicrobials on clinical suspicion of HAI. Inadequate, inappropriate or unnecessary empirical therapy contributes to the development of antimicrobial resistance and may be associated with long-term adverse outcomes16 17
Using an Infection Prediction Score (IPS) as a screening tool may facilitate earlier diagnosis and treatment of neonatal HAI, and assist to differentiate infectious from non-infectious causes of neonatal deterioration, potentially reducing empirical antimicrobial use.18 A meta-analysis of 12 existing IPSs concluded that they had limited diagnostic accuracy with sensitivities of 56%–98% and specificities of 18%–73% and should be considered as guidance only.18 In a performance comparison of IPSs, none of the scores performed well in a South African setting.19 Furthermore, many IPSs used clinical and laboratory variables that were not routinely available in LRSs, for example, continuous heart rate monitoring, blood gas analysis and blood pressure monitoring, hampering their application in LRSs.
The objective of this study was to develop and internally validate a neonatal IPS for the diagnosis of bacterial HAI in VLBW infants in LRSs.
Materials and methods
Study setting
Tygerberg Hospital, Cape Town, South Africa, is a 1384-bed tertiary hospital situated in the Western Cape, South Africa. The obstetric-neonatal service manages approximately 8000 high-risk deliveries (37% low birth weight; <2500 g) and 3000 neonatal admissions annually, of which 800–1000 are VLBW infants.20 The neonatal unit consists of 132 beds, including a 12-bed Neonatal Intensive Care Unit (NICU), 3 high-dependency wards and 1 kangaroo mother care ward.
Patients and data collection
A pre-existing dataset of HAI episodes in VLBW infants from Tygerberg Hospital was used for the IPS development. In the original study, electronic patient records (Tygerberg Hospital Enterprise Content Management system) and the National Health Laboratory Service Trakcare Results viewer were used to retrospectively collect clinical and laboratory data from VLBW infants admitted between 1 January 2016 and 31 December 2017 for >72 hours, using REDCap, a secure online electronic data capture tool.21,22
Study definitions
Neonatal HAI may be suspected when clinical signs of infection is present. These signs include, for example, tachypnoea, tachycardia, temperature and glucose instability and mottled skin. Based on these clinical signs, and at the discretion of the attending physician, a single blood culture is aseptically collected, as well as a CRP and a complete blood count, and empirical antimicrobials initiated.
HAI episodes occurring after 72 hours of admission to the neonatal unit were classified in two categories based on local institutional infection surveillance protocols.
Any HAI includes both the following:
Proven HAI: positive blood culture with clinical symptoms of infection. Organisms were classified using the United States Centres for Disease control list of pathogens and contaminants.23 Records of patients where coagulase-negative staphylococci (CoNS) were isolated were reviewed to distinguish between infection and contamination. The definition of CoNS infection was adapted from Stoll et al1 and simplified for our setting. CoNS infection was defined as (i) two positive blood cultures taken 24–48 hours apart, or (ii) a single positive blood culture combined with a serum CRP ≥10 mg/L with clinical features suggestive of infection. Contaminants were included in the group with no HAI.
Presumed HAI: clinical symptoms of infection in the presence of a CRP ≥10 mg/L and a negative blood culture, where antimicrobial treatment was continued for≥5 days.
No HAI: this included patients with short-lived symptoms but no objective findings of infection, with negative blood culture, CRP <10 mg/L, where antimicrobials were discontinued within 48–72 hours based on local treatment guidelines.
All VLBW infants who were born before arrival to hospital and deliveries at other facilities within the Tygerberg Hospital referral area were defined as delivery outside of a tertiary care facility. Apnoea was defined as a pause of breathing for more than 15–20 s, or accompanied by oxygen desaturation and bradycardia.24 Small for gestational age was defined as birth weight below the 10th percentile for gestational age.25 Lethargy was defined as mental and physical sluggishness: a degree of inactivity and unresponsiveness approaching or verging on the unconscious.26 Glucose instability was defined as a glucose <2.6 mmol/L27 or >7 mmol/L28 for which no iatrogenic cause, for example, non-functioning of intravenous catheter, was identified.
Statistical analysis, derivation and validation of the IPS
All statistical analysis was performed using IBM SPSS Statistics for MacIntosh, V.27.0. For normally distributed continuous variables, means and SDs were calculated. Medians and IQRs were used for non-normally distributed continuous data. The statistical software was used to randomise the study population into two unmatched groups: one for the development of the model IPS (derivation cohort), and one for internal validation of the IPS (validation cohort).
Phase I: model development (derivation cohort)
The IPS was developed in a stepwise manner. Thirty-six potential variables were identified from existing literature and clinical experience, and included maternal and neonatal characteristics, clinical features, interventions and laboratory data. Univariate analysis was performed to identify variables with p<0.1 using the χ2 test and Fisher’s exact test for dichotomous variables, and the student’s t-test was used for continuous variables. For the logistic regression analysis, only variables with p<0.1 were included. We also applied the classification tree approach to identify relationships between variables and its ability to predict future events. Thereafter, variables with a p<0.05 on logistic regression analysis were included in the final IPS model, using the Hosmer-Lemeshow goodness-of-fit test to assess how well the data fit the model. Bootstrapping was used to correct for possible overoptimistic results in the final model. Bootstrapping generates an estimate of how well the model might fit in a new study population.29 The total IPS for each episode of HAI was the sum of the variable values, and receiver operating characteristic (ROC) curve analysis and Youden index was then used to evaluate the performance of the prediction model.
Phase II: internal validation (validation cohort)
The IPS model was applied to the validation cohort. Positive predictive value, negative predictive value, positive likelihood ratio (PLR), negative likelihood ratio (NLR), sensitivity and specificity were evaluated for the score model at multiple numerical cut-off values.
All findings were reported in accordance with the STROBE-NI criteria.30 The Stellenbosch University Health Research Ethics Committee and the Tygerberg Hospital management reviewed and approved the study protocol (S20/11/325).
Patient and public involvement
Patents and/or the public were not involved in this research’s design, conduct, reporting or dissemination plans.
Results
Description of the study population
During the study period, 731 VLBW infants were investigated for 1694 separate episodes of clinically suspected HAI. Of these, 578 episodes were excluded based on incomplete records. The remaining 1116 clinically suspected HAI episodes occurred in 655 VLBW infants, in which 670 (60.0%) HAIs were considered excluded and 446 (40.0%) in which any HAI was diagnosed as per the study definitions (figure 1). Both cohorts had similar demographics based on the first episode of infection (table 1). There were 113 and 109 episodes of proven HAI (blood culture positive) in the derivation and validation cohorts, respectively. Gram negative organisms were responsible for the majority of blood culture positive HAI in both cohorts (46.9% (53/113) in derivation cohort, and 51.4% (56/109) in the validation cohort; p 0.696.
Flow diagram of neonatal HAI episodes in very low birthweight infants included in the analysis. 1Any HAI: this refers to proven and presumed HAI combined. HAI, healthcare-associated infection.
Comparison of baseline characteristics and pathogen distribution of the derivation and validation cohort
Development of the IPS
The derivation cohort included 564 episodes from 412 VLBW infants investigated for HAI, of which 333 (59.0%) had no HAI and 231 (41.0%) had any HAI. A total of 36 variables (7 maternal and labour risk factors, 4 neonatal characteristics, 14 clinical variables, 3 interventions and 8 laboratory variables—online supplemental table 1) were included in the initial univariate analysis comparing infants with no HAI to those with any HAI. The following 15 variables achieved a p<0.1 favouring any HAI: delivery outside of a tertiary facility, mottled skin, prolonged capillary refill time (CRT) >3 s, lethargy, glucose instability, abdominal distention, total parenteral nutrition (TPN), presence of a central venous catheter (CVC), a raised CRP, raised absolute neutrophil count, raised neutrophil percentage, lower platelet count, presence of toxic granulation and left shift (increased immature white cells) (online supplemental table 1). Increased respiratory rate and macroscopic blood in stool achieved p values of<0.1, favouring no HAI.
Supplemental material
On stepwise logistic regression analysis of the clinical characteristics (including the labour and interventional characteristics) and laboratory characteristics, the following variables achieved statistical significance: prolonged CRT >3 s, lethargy, glucose instability, abdominal distension, presence of a CVC and raised CRP. Platelet count also achieved statistical significance; however, this was associated with an inverse (negative) regression coefficient of −0.003 (table 2).
Logistic regression analysis of 15 variables with p<0.1 on univariable analysis
The classification tree analysis of clinical characteristics (including labour and interventional characteristics) included CVC, abdominal distention, lethargy, prolonged CRT >3 s and glucose instability, and had an overall accuracy of 70.5% (online supplemental figure 1). For laboratory characteristics, CRP and platelet count was included, with an overall accuracy of 93.6% (online supplemental figure 1).
Supplemental material
Based on the regression analysis, classification trees and face validity, five variables were included in the final Neonatal Healthcare-associated infectiOn Prediction (NeoHoP) score: CRT >3 s, lethargy, abdominal distention, presence of a CVC in the previous 48 hours and a raised CRP. Raised CRP (continuous variable) was substituted with CRP ≥10 mg/L as a categorical variable, based on the initial definition of HAI and to simplify the score to be used as a bed-side tool (table 3). Regression analysis was repeated with bootstrapping, and all variables retained significance.
The NeoHoP score and the diagnostic performance of the score
The NeoHoP score achieved an area under the ROC curve (ROC AUC) of 0.902 (95% CI: 0.876 to 0.928), and a NeoHoP score of ≥2 achieved a sensitivity of 55.2%, specificity of 98.2%, PLR of 30.8 and Youden index of 0.53 (figure 2; table 3).
Graph showing the ROC curve for the analysis of the diagnostic accuracy of the NeoHoP score in the derivation cohort (A), area under curve 0.902 and the validation cohort (B), area under curve 0.868. The reference line on both graphs represents a curve with no predictive value (area under curve=0.50). NeoHoP, Neonatal Healthcare-associated infectiOn Prediction; ROC, receiver operating characteristic.
Internal validation of the IPS
The validation cohort included 552 episodes from 406 VLBW infants investigated for HAI, of which 337 (61.1%) had no infection and 215 (38.9%) had any HAI (table 1).
The NeoHoP score was calculated for each episode, using a cut-off value of ≥2, based on its performance in the derivation cohort (table 3). The score achieved a ROC AUC of 0.868 (95% CI: 0.837 to 0.900), and a NeoHoP score ≥2 achieved a low sensitivity of 54.2%, high specificity of 96.4%, high PLR of 15.2, low NLR of 0.48 and Youden index of 0.51 (figure 2).
Discussion
We developed a novel IPS for evaluation of suspected neonatal HAI episodes using clinical, management and laboratory variables in a retrospective cohort of VLBW infants. The NeoHoP (Neonatal Healthcare-associated infectiOn Prediction) score consists of five variables, and a score of ≥2 achieved high specificity, high PLR of >15, a ROC AUC of 0.868 and an acceptable Youden index of 0.51. The score is easy to use and incorporates parameters that are readily available in LRSs.
The NeoHoP ≥2 is best positioned to be used as a rule-in test, owing to its high specificity, low sensitivity and high PLR. The NeoHoP score has good discriminative performance as shown by its high ROC AUC, comparing well to the NOSEP-1 score from Belgium (AUC 0.82),31 the Okascharoen score from Thailand (AUC 0.80)32 and the Bekhof score from the Netherlands (AUC 0.83).33
Many of the variables included in the analysis are commonly used to identify infections but are non-specific. For instance, temperature instability (fever and hypothermia) is considered an important clinical feature of infections. Mahieu et al found that temperature instability and hypothermia may be found in conditions other than infection, and that fever was an important feature of infection in preterm and term infants.31 We found that neither fever nor hypothermia reached statistical significance in our cohort. This may be partly related to the inconsistent availability of continuous transcutaneous temperature measurements in our unit and is likely to also be problematic to obtain routinely in other LRSs.
Similarly, increased respiratory rate was not found to be associated with HAI. This is in keeping with the findings of Rosenberg et al,34 who concluded that respiratory symptoms, which are highly prevalent in premature infants, are not specific enough to differentiate between infection and non-infectious respiratory pathology in neonates. Lethargy is commonly found in neonates of all gestations investigated for HAI,35 and was also found to be a significant predictor of HAI in our analysis.
Adnyana et al36 found that hypoglycaemia of <2.6 mmol/L and hyperglycaemia >8.1 mmol/L was associated with an increased incidence of mortality among neonates of all gestational ages with HAI. However, in our setting, glucose levels are often affected by other factors such as the availability of oral feeds and interruption of intravenous fluids, hence our decision to exclude it from the final model.
Mahieu et al31 described the contribution of CVC and TPN to increased infection risk, and these were included in their NOSEP score. We found that the presence of a CVC at the time of investigation, or in the previous 48 hours, was a significant risk factor for HAI and warranted inclusion in the NeoHoP score. This inclusion may be controversial, as in many LRSs access indwelling CVCs are limited. However, most neonatal units do make use of umbilical venous catheters, and therefore this variable was retained in the final model.
Haematological parameters were found to be significant predictors of HAI in the univariate analysis but lost its significance on the logistic regression analysis. This is supported by Hornik et al,13 who found that white cell count, absolute neutrophil count, immature-to-total neutrophil (I:T) ratio and platelet count lacked diagnostic accuracy in HAI, possibly as this parameter is affected by gestational age and birth weight. Rodwell et al37 devised a haematological scoring system for early identification of infection. Narasimha et al38 reviewed the score and found that an abnormal I:T ratio was particularly sensitive in identifying infants with infection. This parameter is not available in our unit and most other LRSs.
The use of CRP in the diagnosis of HAI remains controversial with the emergence of newer and more expensive biomarkers. Serum CRP concentrations rise within 10–12 hours in response to bacterial infections, and peak after 36–48 hours.39 Due to this delay in elevation, the appropriate time to perform a CRP needs careful consideration. In a recent meta-analysis it was concluded that CRP lacked diagnostic accuracy for HAI, however, it was analysed as a stand-alone test and not in combination with other tests or biomarkers.14 It has been argued that in LRSs, the CRP may still be used as a simple and easily available tool to identify at-risk infants.40 However, laboratory CRP may also not be universally available in LRSs, therefore future studies should explore the use of low-cost point-of-care test (POCT) CRP as an alternative.
A major strength of our study is the inclusion of VLBW infants in SSA. The study population reflects a category of patients with high morbidity and mortality related to HAI.1 41 Additionally, the inclusion of presumed HAI within the ‘any HAI’ group is aligned with clinical practice in LRSs where many healthcare providers rely on auxiliary tests such as the CRP to guide their antimicrobial decision making. In our unit, presumed HAI is equally as prevalent as proven HAI.22
The retrospective nature of the data used for the IPS development and validation is a study limitation. Ideally, this IPS should be validated prospectively in an external setting. We included only VLBW infants at a tertiary neonatal referral centre, which may limit its generalisability to other units. Positive cultures from other sterile sites, for example, cerebrospinal fluid and urine were not included in the analysis as the data were not available.
Conclusion
We developed a novel IPS (NeoHoP score) that performed well on internal validation as a rule-in screening test for HAI in VLBW infants. The NeoHoP score may be an important tool for healthcare providers working in LRSs. External validation of the score across all gestations is required to assess its generalisability in other LRSs neonatal units before its widespread use can be recommended. The availability and use of low-cost POCT CRP in LRSs should also be considered as a priority research area in future.
Data availability statement
Data are available upon reasonable request. Anonymised data can be made available upon reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
This study was approved by Stellenbosch University Health Research Ethics Committee S20/11/325As; the study was performed retrospectively, and the ethics committee approved a waiver of individual informed consent.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors LGL, AD, AB and MMvW conceptualised the study LGL collected and analysed the data and prepared the first draft. All authors read, edited and approved the final manuscript. LGL is the guarantor.
Funding LG Lloyd received funding for this study from FIND, Diagnosis for all (Award number not applicable)
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.