Infectious Diseases•

Systematic review and meta-analysis of the diagnostic value of four biomarkers in detecting neonatal sepsis in low- and middle-income countries

Abstract

Background Biomarkers may enhance diagnostic capability for common paediatric infections, especially in low- and middle-income countries (LMICs) where standard diagnostic modalities are frequently unavailable, but disease burden is high. A comprehensive understanding of the diagnostic capability of commonly available biomarkers for neonatal sepsis in LMICs is lacking. Our objective was to systematically review evidence on biomarkers to understand their diagnostic performance for neonatal sepsis in LMICs.

Methods We conducted a systematic review and meta-analysis of studies published in English, Spanish, French, German, Dutch, and Arabic reporting the diagnostic performance of C reactive protein (CRP), erythrocyte sedimentation rate (ESR), white blood cell count (WBC) and procalcitonin (PCT) for neonatal sepsis. We calculated pooled test characteristics and the area under the curve (AUC) for each biomarker compared with the reference standards blood culture or clinical sepsis defined by each article.

Results Of 6570 studies related to biomarkers in children, 134 met inclusion criteria and included 23 179 neonates. There were 80 (59.7%) studies conducted in LMICs. CRP of ≥60 mg/L (AUC 0.87, 95% CI 0.76 to 0.91) among 1339 neonates and PCT of ≥0.5 ng/mL (AUC 0.87, 95% CI 0.70 to 0.92) among 617 neonates demonstrated the greatest discriminatory value for the diagnosis of neonatal sepsis using blood culture as the reference standard in LMICs.

Conclusions PCT and CRP had good discriminatory value for neonatal sepsis in LMICs. ESR and WBC demonstrated poor discrimination for neonatal sepsis in LMICs. Future studies may incorporate biomarkers into clinical evaluation in LMICs to diagnose neonatal sepsis more accurately.

PROSPERO registration number CRD42020188680.

What is already known on this topic

  • Despite decreasing incidence over time, bacterial infections contribute significantly to neonatal morbidity and mortality worldwide, particularly in low- and middle-income countries (LMICs).

  • In many LMICs, reference standard diagnostics for bacterial infections such as blood cultures are often unavailable.

  • C reactive protein (CRP), erythrocyte sedimentation rate (ESR), white blood cell count (WBC), and procalcitonin (PCT) have been incorporated into clinical predictive algorithms in high-income countries but their discriminatory value in LMICs is less clear.

What this study adds

  • In a systematic review and meta-analysis including 134 studies and 23 179 neonates, none of the evaluated biomarkers had sufficient specificity or discriminatory value to be used in isolation to diagnose neonatal sepsis in LMICs.

  • CRP and PCT had good discriminatory value for neonatal sepsis in LMICs.

  • ESR and WBC had poor discriminatory value for neonatal sepsis in LMICs.

How this study might affect research, practice or policy

  • CRP, ESR, WBC, and PCT alone should not be used to differentiate neonates at risk for neonatal sepsis.

  • Future studies may incorporate biomarkers into clinical evaluation in LMICs to diagnose neonatal sepsis more accurately.

Introduction

Despite decreasing incidence over time, bacterial infections contribute significantly to childhood morbidity and mortality worldwide, particularly in low- and middle-income countries (LMICs).1 2 Neonatal sepsis is a common cause of neonatal morbidity and mortality in LMICs.1 Nonetheless, there is no unified criteria for the diagnosis of neonatal sepsis, which makes clinicians in resource-limited settings with scarce access to blood cultures rely on a clinical diagnosis.

In sub-Saharan Africa, as many as 22%–25% of determined causes of fever are bacterial in nature among children presenting for clinical care.3 4 However, in many LMICs, reference standard diagnostics for bacterial infections such as blood cultures, chest radiography, or polymerase chain reaction (PCR) are often unavailable.5–7 This may result in widespread overuse of antibiotics, or, conversely, under-recognition and undertreatment of bacterial infections.

Biomarkers (or biological markers) are objective measures that may be evaluated as indicators of pathological processes and have the potential to facilitate risk stratification for infectious diseases.8 C reactive protein (CRP), erythrocyte sedimentation rate (ESR), white blood cell count (WBC), and procalcitonin (PCT) have been incorporated into clinical predictive algorithms in high-income countries (HICs).9 10 However, biomarkers are not yet widely used in many LMICs where there is greater burden of bacterial disease and lower rates of immunisation. Consequently, an understanding of how biomarkers perform among children in these settings is lacking.

A reliable approach to identifying a child’s risk of infection may enhance the quality of clinical care, promote better resource utilisation, and allow for targeted and responsible antibiotic use in settings with limited access to reference standard diagnostics. A comprehensive understanding of the diagnostic capability of commonly available biomarkers for neonatal sepsis in LMICs is lacking but may allow for more accurate diagnoses among neonates and more judicious antibiotic use. Our objective was to systematically review existing evidence on the use of four biomarkers (CRP, ESR, WBC, and PCT) to understand their diagnostic performance against the reference standards of blood culture and clinical sepsis for neonatal sepsis, with a focus on studies conducted in LMICs. We focused our analysis on neonatal sepsis as it makes significant contributions to childhood morbidity and mortality globally.1 2

Methods

Study design

We conducted a systematic review and meta-analysis following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.11 We registered this study in PROSPERO, an international prospective register for systematic reviews (CRD42020188680).

We focused our review on CRP, ESR, WBC, and PCT to understand the potential use of these biomarkers in clinical settings in which reference standard diagnostic testing may be limited. Though other biomarkers, including proadrenomedullin and various serum interleukins, have also been used to assess the presence of bacterial illness against reference standards,12 13 these are not currently routinely accessible in many settings, both in HICs and particularly in LMICs, so were excluded from the analysis.

Patient and public involvement statement

The development of the research question was informed by the high disease burden of neonatal sepsis. Patients were not involved in the design, recruitment, or conduct of the study, nor were they advisers in this study. Results of this study have been made publicly available through publication.

Data sources

We searched the Medline, EMBASE, DARE, CINAHL, and Babelmesh databases on 12 February 2021 and conducted an updated search on 29 August 2022. We extracted articles that were included in each of these databases from their inception to 29 August 2022. The search terms used to identify studies that focused on the use of the four biomarkers of interest are included in online supplemental appendix 1. Our search was limited to articles published in English, Spanish, French, German, Dutch, and Arabic as members of our team were fluent in these languages.

Inclusion and exclusion criteria

We included studies that met the following criteria (1) were peer-reviewed, original research articles published from the inception of each database to 29 August 2022, (2) evaluated the use of one of the four biomarkers of interest in the diagnosis of an infectious disease, (3) included participants aged 0–18 years and (4) included a control group that did not test positive with a reference standard as a comparison for the diagnostic performance of the biomarkers evaluated. Initially, our search was not restricted to specific diseases. However, post hoc, we decided to focus our analysis on neonatal sepsis as there were at least 20 studies that met our inclusion criteria, and it contributes to a large burden of childhood morbidity and mortality globally. There were >20 studies that reported the test characteristics of the included biomarkers for pneumonia. However, these were not included in our manuscript because those studies did not differentiate viral from bacterial disease.

We excluded studies that met any of the following criteria: (1) articles that were not published in English, Spanish, French, German, Dutch, or Arabic, (2) abstracts without full text, (3) articles that only included highly medicalised populations, (4) articles reporting only mean or median values for biomarkers, (5) articles that did not evaluate children separately if adults aged >18 years were included, (6) articles that only assessed changes in biomarkers during treatment, and (7) case reports, editorials, study protocols, review articles, systematic reviews, and meta-analyses. We reviewed systematic reviews and meta-analyses for other articles reporting primary data our initial query did not capture. Any potential articles identified therein were included if they met inclusion criteria.

Definitions

We used the definitions used for our outcome of neonatal sepsis as reported in the included studies (ie, either positive blood culture or clinical sepsis).14 Study countries were defined as low- middle-income and high-income according to the World Bank definitions.15

Data extraction and risk of bias assessment

Using the results from our database query, we uploaded all articles into the platform Covidence (Melbourne, Australia) to screen article titles and abstracts for potential inclusion. Two reviewers independently screened articles in two rounds. Each reviewer was blinded to the other reviewer’s screening. The first round included a review of all abstracts for the presence of exclusion criteria. All article titles and abstracts that resulted in disagreement between two independent reviewers were reviewed by an arbiter (CAR) to assess inclusion or exclusion. The second round included a review of article full texts for those remaining after titles and abstracts were reviewed. The full text of articles in Spanish, French, German, Dutch, or Arabic were screened and reviewed by a team member who was fluent in the respective language.

We reviewed the full text of each article that was included after the initial phase of article title and abstract review. We extracted the following information from each included article: study location (eg, outpatient, emergency department, inpatient such as neonatal intensive care unit), study design, study country, included patient ages, disease studied, biomarker(s) evaluated, reference standard and study inclusion and exclusion criteria. Biomarkers were considered diagnostic if they were used to distinguish an infection in a child from healthy controls or children who had negative reference standard testing. We extracted the reported number of true negatives (TNs), true positives (TPs), false negatives (FNs), and false positives (FPs) based on reported biomarker cut points and reference standard testing. For studies that did not report these numbers, we extracted the reported sensitivity, specificity, positive and negative likelihood ratios wherever possible and emailed the corresponding author to request additional data. If there was no answer to an initial email request, a second email was sent 2 weeks later.

The risk of bias of the included studies was assessed using the Quality Assessment of Studies for Diagnostic Accuracy Included in Systematic Reviews-2 (QUADAS-2) tool, which is designed to assess bias and applicability concerns for diagnostic studies.16

Statistical analyses

If a study did not provide the TN, TP, FN, and FP but provided sensitivity, specificity, and the total population number, and corresponding authors did not respond to our request, we calculated the 2×2 table numbers rounded to the nearest integer. We reported the aggregate performance of each biomarker cut point with up to two reference standards in the same studies (eg, blood culture or clinical sepsis) for neonatal sepsis and alone in cases in which ≥3 studies reported the same cut point.

Many of the studies that met our inclusion criteria used different cut points for their respective biomarker. We evaluated each biomarker cut point used by ≥3 studies individually using a bivariate model created by Reitsma et al through the reitsma function in the R package Mada.17 18 The bivariate analysis method created by Reitsma et al produces summary estimates of sensitivity and specificity that include 95% CIs that account for heterogeneity. We also calculated Holling’s sample size adjusted measure for heterogeneity (I2) which was developed for use in bivariate meta-analyses of diagnostic accuracy.19 We calculated the sensitivity, specificity, and the area under the curve (AUC) along with their respective 95% CIs, for each disease and biomarker combination. 95% CIs for AUCs were calculated through bootstrapping with 2000 resamplings via the AUC boot function in the dmetatools R package created by Noma H.20 We calculated and reported the highest Youden’s index for each biomarker and disease combination. Based on published standards, we used the following scale to qualify the discriminatory value of each score: AUC ≥0.90 for ‘excellent discrimination’, AUC 0.80–0.89 for ‘good discrimination’, AUC 0.70–0.79 for ‘minimal discrimination’, and ‘poor discrimination’ for AUC <0.70.21–23 We subanalysed all results by study country income group according to the World Bank and reference standard if there were ≥3 studies using the same cut point within that subgroup. All statistical analyses were conducted using SAS V.9.4 and R V.4.0.3 (R Foundation for Statistical Computing, Vienna, Austria).

Results

Study selection and characteristics

There were 6570 studies identified through our search. After abstract screening, 1816 full-text articles were reviewed and 134 reported biomarker performance for neonatal sepsis and met our inclusion criteria (figure 1). In the 134 studies included in the pooled analysis, there were 23 179 total neonates. The 134 studies reported work conducted in 42 different countries, 80 (59.7%) in LMICs, and 54 (40.3%) in HICs.

Figure 1
Request permissionrequest permission icon
Figure 1

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram detailing the identification, screening and inclusion of studies for the use of biomarkers in the diagnosis of infections in children.

Included study characteristics are described in online supplemental table 1. Of the 134 included studies, 70 (52.2%) were deemed low risk for bias (QUADAS-2 of 1 or 2), 43 (32.1%) were deemed intermediate risk (QUADAS-2 of 3), and the remaining 21 (15.7%) had high risk of bias (QUADAS-2 of 4 or 5). Of the 80 studies conducted in LMICs, 36 (45.0%) were low risk for bias, 29 (36.3%) were intermediate risk, and the remaining 15 (18.8%) had high risk of bias. Following a similar distribution, of the 54 studies conducted in HICs, 34 (63.0%) were low risk for bias, 14 (25.9%) were intermediate risk, and the remaining 6 (11.1%) had high risk of bias.

Biomarker performance

Of the 134 studies that evaluated the performance of biomarkers for neonatal sepsis, 109 (81.3%) evaluated CRP, 3 (2.2%) evaluated ESR, 17 (12.7%) evaluated WBC, and 31 (23.1%) evaluated PCT. Of the 134 studies, 123 (91.8%) used blood culture as the reference standard and 11 (8.2%) used clinical sepsis.

The CRP cut point with the highest Youden’s index in the diagnosis of neonatal sepsis in LMICs was ≥60 mg/L using blood culture as the reference standard among 1339 neonates from nine studies (table 1). A CRP of ≥60 mg/L demonstrated good discriminatory value in differentiating neonates at risk of neonatal sepsis (AUC 0.87, 95% CI 0.76 to 0.91). Among studies conducted in all settings, the CRP cut point that demonstrated the highest Youden’s index was ≥2.5 mg/L with blood culture as the reference standard among 263 neonates and had good discriminatory value (AUC 0.83, 95% CI 0.70 to 0.93) among three studies.

Table 1
|
Test characteristics of C reactive protein (CRP) in the diagnosis of neonatal sepsis

ESR was evaluated less commonly than CRP for the diagnosis of neonatal sepsis in the included studies. Among 3 studies with 599 neonates in all country brackets using blood culture as the reference standard, an ESR of ≥15 mm/hour had a low Youden’s index (0.11) and poor discriminatory value (AUC 0.36, 95% CI 0.17 to 0.85) (online supplemental table 2). There were not enough studies to evaluate ESR for the diagnosis of neonatal sepsis in HICs or LMICs alone. WBC was the biomarker with the lowest sensitivity and specificity among biomarkers for the diagnosis of neonatal sepsis in all study settings (online supplemental table 3).

The PCT cut point with the highest sensitivity and specificity was ≥2.0 ng/mL using blood culture and clinical sepsis as the reference standard for neonatal sepsis among 728 neonates from eight studies conducted in LMICs (Youden’s index 0.55) (table 2). A PCT of ≥0.5 ng/mL demonstrated good discriminatory value in diagnosing neonatal sepsis (AUC 0.87, 95% CI 0.70 to 0.92). Among studies conducted in all settings, a PCT of ≥1.7 ng/mL demonstrated the highest sensitivity and specificity (Youden’s index 0.52) among 433 neonates from three studies and had good discriminatory value (AUC 0.83, 95% CI 0.71 to 0.88).

Table 2
|
Test characteristics of procalcitonin (PCT) in the diagnosis of neonatal sepsis

Discussion

In this systematic review and meta-analysis including 134 studies and over 23 000 neonates, the utility of CRP, ESR, WBC, and PCT demonstrated substantial heterogeneity in the diagnosis of neonatal sepsis. CRP and PCT had good discriminatory value for neonatal sepsis in LMICs. However, none of the evaluated biomarkers had sufficient specificity or discriminatory value to be used in isolation to diagnose neonatal sepsis in LMICs. Moreover, despite bearing most of the world’s childhood disease burden for neonatal sepsis, there was a relative paucity of data from LMICs.

Despite its high incidence and significant disease burden globally, there is no unified criteria for the diagnosis of neonatal sepsis. The WHO Guidelines for the Management of Common Childhood Illnesses include risk factors and antibiotic recommendations for neonatal sepsis, though diagnostic criteria are lacking.24 For purposes of standardisation, we evaluated the diagnostic performance of biomarkers using positive blood cultures as a reference standard. However, important challenges in the diagnosis of neonatal sepsis include potential false-negative culture results due to maternal antibiotic administration, insufficient blood volume obtained in blood draws, and low, or intermittent, levels of bacteraemia.25 26 There were 16 studies that used clinical sepsis as a reference standard, although clear and consistent definitions of this reference were lacking.

Prior systematic reviews have described the diagnostic utility of biomarkers in neonatal sepsis27–29; however, most reviews omit studies conducted in LMICs, where the disease burden for neonatal sepsis is highest.30 Our study found CRP for neonatal sepsis in LMICs demonstrated poor specificity at varying cut points, but good overall discriminatory value. WBC had little diagnostic value in the diagnosis of neonatal sepsis in our study. PCT had relatively low specificity but good discriminatory value in studies conducted in LMICs. Prior reviews highlight statistical heterogeneity between studies on PCT and neonatal sepsis.31

In practice in many resource-limited settings, elevations in biomarkers such as CRP and ESR are used to make decisions around the initiation of antibiotics for neonates. However, our study suggests that the sole reliance on a single biomarker to make such a decision may not have sufficient discriminatory value. The development and validation of clinical prediction models including historical findings, other risk factors, as well as biomarkers for neonatal sepsis in LMICs may enhance the diagnostic capabilities in such settings.9

Limitations

Most of the included studies did not assess all four biomarkers of interest, making unclear their comparative test characteristics in the same populations. It is possible that some studies included neonates that had been pretreated with antibiotics, which could affect the level of biomarkers.32 Most included studies did not differentiate early from late-onset neonatal sepsis. Biomarkers in early-onset sepsis may reflect maternal values. Additionally, populations of neonates who had malnutrition or were infected with, or exposed to, HIV were often excluded from the included studies, leaving unclear the diagnostic performance of biomarkers for the evaluation of infectious diseases in these vulnerable populations. Many studies reported the test characteristics of biomarkers for several infectious diseases in aggregate, which precluded our analysis from teasing out the test characteristics for individual infectious diseases. Lastly, though we attempted to review articles in as many languages as our team was capable to, several articles were excluded from our analysis because they were not published in English, Spanish, French, German, Dutch, or Arabic. This may have introduced some selection bias, potentially excluding more articles reporting research conducted in LMICs where these languages are not spoken.

Conclusions

CRP and PCT had good discriminatory value to diagnose neonatal sepsis in LMICs. However, none of the evaluated biomarkers had sufficient specificity or discriminatory value to be used in isolation to diagnose neonatal sepsis in LMICs. Future studies conducted in LMICs and should incorporate biomarkers into clinical prediction algorithms to achieve more optimal diagnostic and discriminatory ability for neonatal sepsis.

Overview

Article metrics
Altmetricopen-url
Dimensionsopen-url