Article Text
PDF
Abstract
Background There are no validated clinical decision aids to identify neonates and young children at risk of hospital readmission or postdischarge mortality in sub-Saharan Africa, leaving the decision to discharge a child to a clinician’s impression. Our objective was to determine the precision of clinician impression to identify neonates and young children at risk for readmission and postdischarge mortality.
Methods We conducted a survey study nested in a prospective observational cohort of neonates and children aged 1–59 months followed 60 days after hospital discharge from Muhimbili National Hospital in Dar es Salaam, Tanzania or John F. Kennedy Medical Center in Monrovia, Liberia. Clinicians who discharged each enrolled patient were surveyed to determine their perceived probability of the patient’s risk of 60-day hospital readmission or postdischarge mortality. We calculated the area under the precision-recall curve (AUPRC) to determine the precision of clinician impression for both outcomes.
Results Of 4247 discharged patients, 3896 (91.7%) had available clinician surveys and 3847 (98.7%) had 60-day outcomes available: 187 (4.8%) were readmitted and 120 (3.1%) died within 60 days of hospital discharge. Clinician impression had poor precision in identifying neonates and young children at risk of hospital readmission (AUPRC: 0.06, 95% CI: 0.04 to 0.08) and postdischarge mortality (AUPRC: 0.05, 95% CI: 0.03 to 0.08). Patients for whom clinicians attributed inability to pay for future medical treatment as the reason for risk for unplanned hospital readmission had 4.76 times the odds hospital readmission (95% CI: 1.31 to 17.25, p=0.02).
Conclusions Given the poor precision of clinician impression alone to identify neonates and young children at risk of hospital readmission and postdischarge mortality, validated clinical decision aids are needed to aid in the identification of young children at risk for these outcomes.
- mortality
- health services research
- neonatology
Data availability statement
Data are available upon reasonable request. Data Sharing Agreement: Data may be made available upon reasonable request to the corresponding author.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
What is already known on this topic?
In parts of sub-Saharan Africa, hospital readmissions and postdischarge mortality rates are estimated to range from 1% to 18% within months of hospital discharge.
There are no validated clinical decision aids to accurately identify neonates, infants and young children at risk of hospital readmission or postdischarge mortality in sub-Saharan Africa.
What this study adds?
Clinician impression alone had poor precision in identifying neonates, infants and children at risk of hospital readmission within 60 days of hospital discharge.
Clinician impression also did not accurately identify neonates, infants and children at risk of 60-day postdischarge mortality.
How this study might affect research, practice or policy?
Clinician impression alone is not sufficient to accurately identify neonates, infants and children at risk of hospital readmission or postdischarge mortality.
Validated and objective clinical decision aids are urgently needed to better identify neonates, infants and children at risk of hospital readmission and postdischarge mortality.
Introduction
Mortality rates among children aged <5 years in sub-Saharan Africa are 74 per 1000 live births, which is the highest in the world and is 14 times higher than rates in Europe and North America.1 The time after an inpatient hospital admission is particularly vulnerable in the life of a child in sub-Saharan Africa. Recent studies suggest that readmissions occur in 8%–18% of young children and as much as 1%–20% of young children die within 6 months after hospital discharge.2–5 Childhood mortality rates in the period immediately after hospitalisation for an illness (ie, the postdischarge period) approximate and may even outpace rates of mortality during hospitalisation.5 6
Although clinical prediction rules for all-cause hospital readmissions among children in settings like the USA have been developed that include variables such as prior healthcare utilisation and markers of illness severity,7 to our knowledge, there are currently no clinical prediction rules for hospital readmissions among children in sub-Saharan Africa, where readmissions are common. Differences in healthcare access and disease prevalence in the USA and sub-Saharan Africa necessitate the creation of clinical prediction rules catered to settings in sub-Saharan Africa. Clinical prediction rules have been developed to identify both young children and those aged <15 years at risk of postdischarge mortality in some settings in sub-Saharan Africa.8 9 However, these clinical prediction rules lack external validation and thus are not widely used in clinical practice. Given the absence of validated risk assessment tools to identify young children at risk of readmission and postdischarge mortality in sub-Saharan Africa, the decision to safely discharge a young child from a hospital is often driven by clinical judgement.
Clinician impression relies on a clinician’s ability to recognise patterns that may be associated with severe disease or an adverse outcome.10 However, the accuracy of clinician impression to predict outcomes, such as severe disease from infections, among children has varied in previous studies conducted in high-income settings.11–13 In a survey of 39 providers in Kenya, clinicians underestimated the overall incidence of postdischarge mortality among children.14 However, that study did not assess clinician impression of postdischarge mortality for individual patients and, to our knowledge, that has not been studied previously.
Given the absence of validated prognostic tools for hospital readmission and postdischarge mortality among children in sub-Saharan Africa, our primary objective was to determine the precision of treating clinicians’ clinical impression to identify neonates and young children at risk for hospital readmission and postdischarge mortality in Dar es Salaam, Tanzania and Monrovia, Liberia. Our secondary objective was to evaluate factors associated with accuracy of treating clinicians’ clinical impression to identify neonates and young children at risk for hospital readmission and postdischarge mortality.
Methods
Study design
We conducted a survey nested in a prospective observational cohort study of paediatric patients discharged from Muhimbili National Hospital in Dar es Salaam, Tanzania and John F. Kennedy Medical Center in Monrovia, Liberia from October 2019 to January 2022. Details of our study protocol have been published previously.15 Besides differences in estimated and actual enrolment, there were no deviations from that protocol. Neonates and young children aged 1–59 months were enrolled at discharge from the neonatal or paediatric wards at each facility. Follow-up consisted of caregivers receiving phone calls up to 60 days after hospital discharge. Caregivers provided written consent for participation in Tanzania and oral consent in Liberia because of cultural preference and low rates of caregiver literacy.
Patient and public involvement
The development of the research question was informed by the disease burden of readmission and postdischarge mortality among children in sub-Saharan Africa. Patients were not involved in the design, recruitment or conduct of the study, nor were they advisers in this study. Results of this study will be made publicly available through publication.
Study setting
This study was conducted at two large, national referral hospitals supported by each country’s Ministry of Health. They are in urban areas in their respective countries. Muhimbili National Hospital has a catchment of approximately 6 million people and John F. Kennedy Medical Center has a catchment of approximately 1.5 million people. Both hospitals are training hospitals for paediatric residents who are completing their specialty training.
Study populations
Neonates and young children discharged from the wards were consecutively enrolled. Neonates and young children who died during initial hospitalisation were excluded. Neonates and young children whose caregivers did not have telephones for follow-up or those who declined enrolment were excluded.
Surveyed clinicians included consultants/specialists, interns/residents or medical officers. Consultants/specialists were certified paediatricians or paediatric specialists who completed medical school, residency and subspecialty training (for specialists). Interns/residents had completed medical school and were completing residency training in paediatrics. Medical officers received 3 years of clinical training prior to providing clinical care to patients.
Study procedures
After obtaining informed consent from caregivers, trained research coordinators at each site approached the clinician who discharged each enrolled patient, obtained consent and asked them to complete a survey near the time of the patient’s hospital discharge. This survey modelled previous surveys that assessed clinician impression12 and was developed through an iterative process by the research team with multiple opportunities to each investigator to refine the content. The survey was also reviewed by an expert in survey design (ie, a survey methodologist) to ensure question clarity and appropriate response types (online supplemental appendix survey). This survey was designed to allow clinician respondents to describe their perceived probability that each patient would experience both outcomes. Responses were recorded on standardised, electronic case report forms in electronic tablets using SQL (Tanzania) and KoboToolbox (Liberia).
Supplemental material
Measurement and outcomes
The exposure variable was the impression of the discharging clinician of the patient’s risk of: (1) unplanned hospital readmission within 60 days of hospital discharge or (2) all-cause, 60-day postdischarge mortality. Aligned with previous studies,12 probabilities of perceived risk of readmission or postdischarge mortality included categorical options of 0%, 1%–5%, 6%–20%, 21%–40%, 41%–60%, 61%–80%, 81%–99% and 100%. This survey also assessed discharging clinicians’ perceptions of why readmission or postdischarge mortality were possible for those who were identified as at-risk for each outcome. Surveyed clinicians were familiar with the patients’ clinical history and laboratory results during hospital admission. To assess for outcomes, phone calls to patients’ caregivers were made by research staff at 7, 14, 30, 45 and 60 days after hospital discharge. Outcomes were determined as reported by caregivers to research staff.
Statistical analyses
The association of the discharging clinicians’ predicted probability of readmission or postdischarge mortality and proportion of patients at each clinician-estimated risk threshold (eg, 0%, 1%–5%, etc) who were readmitted or died was compared using χ2 or Fisher’s exact testing (p<0.05 for significance).12 We calculated sensitivity, specificity, positive predictive value and negative predictive value of treating clinician’s impression at each per cent risk threshold using caregiver-reported readmission or postdischarge mortality as the reference standards. We determined the precision of clinician impression for identifying patients at risk of readmission or postdischarge mortality by calculating the area under the precision-recall curve (AUPRC) which is useful for evaluating binary classifiers in imbalanced datasets.16 95% CI for the AUPRC were calculated through fivefold cross-validation. We compared AUPRC and corresponding 95% CIs to baseline chance of the outcome occurring in that group. A 95% CI higher than baseline chance indicated better precision than random chance.
We conducted subanalyses by the discharged patient’s age group (ie, neonate or young child), clinician experience level (ie, consultant/specialist, intern/resident or medical officer), site and time to outcome. We conducted binary logistic regression analyses to assess whether the perceived reason for risk for each outcome was associated with the patient’s likelihood of each outcome. Additionally, we conducted binary logistic regression analyses to assess whether the clinician probability for each outcome was associated with the patient’s likelihood of each outcome after adjusting for clinician type, patient age at discharge (months), whether the discharge diagnosis was infectious or not and duration of hospitalisation (days). Due to small sample sizes in the non-0% clinician probability categories, we reduced the categorisation of perceived risk to 0%, 1%–20%, 21%–60% and 61%–99%. The clinician cited reason for the outcome was also considered in the model but was removed due to collinearity as assessed by the variance inflation factor. All tests were two-sided tests and used a 0.05 significance level. AUPRC analyses were conducted through the precrec package in R.17 All analyses were performed in R V.4.1.3 (R Foundation for Statistical Computing, Vienna, Austria) and SAS V.9.4 (SAS Institute, Cary, North Carolina, USA).
Results
There were 4460 discharged patients, 4247 (95.2%) enrolled and 3896 (91.7%) had complete clinician surveys (figure 1). Enrolment was approximately equal between the two sites (Tanzania n=1997, 51.3%, Liberia n=1899, 48.7%) (table 1). There were 2173 (55.8%) neonates and 1723 (44.2%) young children who had clinician surveys available.
Flow diagram for included neonates and children aged 1–59 months discharged from the neonatal wards and paediatric wards in Dar es Salaam, Tanzania and Monrovia, Liberia.
Characteristics of neonates and young children included in the evaluation of clinician impression on predicting 60-day hospital readmission or postdischarge mortality
Sixty-day outcomes were available for 3847 (98.7%) enrolled patients. The median age of enrolled neonates was 2 days (IQR: 1–7) and 12 months (IQR: 5–24) for infants and children. The most common discharge diagnoses among neonates were sepsis (29.7%, n=609), prematurity (28.8%, n=591) and birth asphyxia (15.8%, n=323). Among infants and children, pneumonia (12.1%, n=223), diarrhoeal disease (10.1%, n=186), and malaria (7.2%, n=133) were most common.
There were 187 (4.8%) patients readmitted and 120 (3.1%) died within 60 days of discharge. There were 80 (3.6%) neonates who were readmitted and 61 (2.8%) died within 60 days of hospital discharge. Among infants and children, 107 (6.2%) were readmitted and 59 (3.4%) died after hospital discharge. The median time from hospital discharge to readmission was 30 days (IQR: 7–45). The median time from hospital discharge to mortality was 30 days (IQR 14–45).
Clinician impression and hospital readmission
Nearly three quarters of patients were perceived to have 0% risk of readmission within 60 days (table 2). Patients who were readmitted were more likely to have a perceived risk of readmission of 0% than those who were not readmitted (81% vs 72%, p<0.001; table 2). Among the 187 neonates and young children who were readmitted, 80.7% (n=151) were perceived to have 0% risk of readmission.
Association of discharging clinicians’ predicted probability and unplanned 60-day hospital readmission among neonates and young children
Overall, clinician impression had poor precision in identifying neonates and young children at risk of readmission (AUPRC: 0.06, 95% CI: 0.04 to 0.08) (table 3). Among medical officers, clinician impression had greater precision in identifying children at risk of readmission (AUPRC: 0.23, 95% CI: 0.17 to 0.34); this group was marginally better at identifying patients at risk of readmission than interns/residents and consultants/specialists.
Test characteristics for clinician predicted probability of unplanned, 60-day hospital readmission among all enrolled neonates and young children aged 1–59 months overall and by clinician type
By clinician type, medical officer clinician impression had poor precision in identifying neonates (online supplemental table 1) but greater precision when identifying infants and children at risk for readmission (online supplemental table 2). In site-specific analyses, clinician impression was imprecise when identifying neonates and young children at risk of readmission in Tanzania (AUPRC: 0.11, 95% CI: 0.07 to 0.15, chance: 0.07) and Liberia (AUPRC: 0.03, 95% CI: 0.02 to 0.04, chance: 0.02). Regardless of the time from hospital discharge to readmission, clinician impression had poor precision in identifying neonates and young children at risk for readmission (online supplemental table 3).
Supplemental material
Clinician impression and postdischarge mortality
Most (96.1%, n=3746) patients were assigned 0% risk of postdischarge mortality (table 4). Patients who died within 60 days of discharge were more likely to have a perceived risk of 0% than patients who survived (96% vs 90.8%, p=0.002; table 4). Among the 120 neonates and young children who died within 60 days of hospital discharge, 90.8% (n=109) were estimated to have a 0% probability of postdischarge mortality.
Association of discharging clinicians’ predicted probability and all-cause, 60-day postdischarge mortality among neonates and young children
Overall, clinician impression had poor precision in identifying neonates and young children at risk for postdischarge mortality (AUPRC: 0.05, 95% CI: 0.03 to 0.08) and did not vary substantially among interns/residents, specialist/consultants or medical officers (table 5). Clinician impression had poor precision in identifying postdischarge mortality among neonates (AUPRC: 0.04, 95% CI: 0.03 to 0.06, chance: 0.03) and infants and children (AUPRC: 0.06, 95 % CI: 0.03 to 0.10, chance: 0.03). When analysed by site, clinician impression in both Tanzania (AUPRC: 0.08, 95% CI: 0.03 to 0.13, chance: 0.03) and Liberia (AUPRC: 0.04, 95% CI: 0.03 to 0.06, chance: 0.04) had poor precision in identifying neonates and young children at risk of postdischarge mortality. Clinician impression had poor precision regardless of the time to postdischarge mortality (online supplemental table 4).
Test characteristics for clinician predicted probability of all-cause, 60-day postdischarge mortality among all enrolled neonates and young children aged 1–59 months by clinician type
Reason for perceived risk of hospital readmission and postdischarge mortality
Patients for whom clinicians attributed inability to pay for treatment as the reason for readmission had 4.76 times the odds readmission (95% CI: 1.31 to 17.25, p=0.02) compared with those with no perceived risk (table 6). Patients whose clinician cited ‘other’ reasons to be at risk had lower odds of readmission compared with those whose clinician did not believe they were at risk for readmission (OR: 0.24, 95% CI: 0.09 to 0.66, p=0.005). Patients for whom clinicians attributed inability to pay for treatment as the reason for potential postdischarge mortality had 5.53 times the odds of postdischarge mortality (95% CI 1.22 to 25.10, p=0.03).
Reasons for perceived risk of hospital readmission and postdischarge mortality
In multivariable analyses, patients whom clinicians estimated to be at moderate risk for hospital readmission (ie, 21%–60%) were at decreased odds of hospital readmission (adjusted OR: 0.45, 95% CI: 0.26 to 0.74, p=0.003) (table 7). No other factors were independently associated with either hospital readmission or postdischarge mortality.
Multivariable regression model for all-cause 60-day hospital readmission and postdischarge mortality among young children discharged in Dar es Salaam, Tanzania and Monrovia, Liberia
Discussion
Among nearly 3900 neonates and young children discharged from referral hospitals in Dar es Salaam, Tanzania and Monrovia, Liberia, clinician impression had poor precision for identifying those at risk of unplanned hospital readmissions and postdischarge mortality. Medical officer clinician impression at both sites had slightly greater precision in identifying young children at risk of readmission. Clinician perception of inability to pay for treatment was associated with readmission and postdischarge mortality.
The poor precision of clinician impression in identifying neonates and young children at risk of readmission and postdischarge mortality differs from findings in studies in high-income settings that assessed the diagnosis of acute coronary syndrome and sinusitis in adults,18 19 the presence of pneumonia20 or the development of severe pneumonia among children12 or the presence of appendicitis among children.21 This difference is likely multifactorial in nature and may consist of differences in available diagnostic and prognostic resources. Prior studies suggest that laboratory capabilities in resource-limited settings are inadequate,22–24 leading to dependence on clinical examination findings to make diagnoses and determine prognosis,25 which may hinder the accuracy of clinician impression to identify neonates and young children at risk of untoward postdischarge outcomes. Moreover, clinicians may not consider key factors in the home (eg, access to healthcare facilities and maternal health) that may contribute to postdischarge outcomes. Additionally, clinician impression had poor precision in identifying neonates and young children at risk for readmission or postdischarge mortality regardless of the time from discharge to either event. Prior studies of clinician impression assessed outcomes within hours or days12 18 19 and not up to 60 days, which may contribute to the difference in our results compared with prior studies assessing clinician impression in prognostication of outcomes.
Clinician impression among medical officers had fair precision in identifying young children at risk of readmission. This may be due to the combination of more clinical experience than interns/residents and more time spent with patients than consultants/specialists who often spend less time directly with patients and more time supervising clinical care. Prior studies conducted in high-income settings demonstrate that clinician impression of less experienced clinicians may have less discriminatory value than that of more experienced clinicians.12
linical prediction rules for postdischarge mortality have been developed among young children aged 6 months–5 years in Uganda and aged <15 years in Mozambique.8 9 These clinical prediction rules include variables such as clinical diagnoses and anthropometry to assign weighted points to included variables to assess an individual patient’s risk for postdischarge mortality up to 6 months after discharge. However, none of these have focused specifically on neonates and none have been externally validated, which is a necessary step to assess discriminatory value prior to clinical use. Thus, the current state of prognostic determination for young children after discharge in sub-Saharan Africa depends on clinician impression and, given its poor precision demonstrated in our study, validated clinical prediction rules to identify neonates and young children at risk of postdischarge mortality are urgently needed. Such clinical decision aids should include commonly collected variables and may include biomarkers to add precision to risk stratification to identify neonates and young children at risk of postdischarge morbidity and mortality.26–28
Our examination of reasons for estimated outcomes suggested that clinician perception of inability to pay for future treatment was associated with higher risk of readmission and postdischarge mortality. Young children from lower socioeconomic status have poorer overall health outcomes compared with young children from higher socioeconomic households in sub-Saharan Africa.29–31 This is particularly relevant in the postdischarge period during which the financial burden of care seeking may influence the ability for a family to seek additional clinical care after a potentially costly hospital admission.
Limitations
Clinician impression is multifactorial and depends on clinical training as well as available laboratory, radiological and clinical data that may not have been available to all discharging clinicians. We did not assess the availability of these in our analysis. We did not include nurses or clinical officers in our study, which is a limitation as these groups may have good insight into potential adverse outcomes after discharge. We could not account for the potential role that variations in the quality of clinical care provided to patients may have had in readmissions or postdischarge mortality.
Conclusions
Clinician impression had poor precision in identifying neonates and young children at risk of unplanned hospital readmission and postdischarge mortality at two referral hospitals in Dar es Salaam, Tanzania and Monrovia, Liberia. Validated and objective clinical decision aids to assist clinicians in the identification of young children at risk of readmission and postdischarge mortality may facilitate the identification of those at greatest risk.
Data availability statement
Data are available upon reasonable request. Data Sharing Agreement: Data may be made available upon reasonable request to the corresponding author.
Ethics statements
Patient consent for publication
Ethics approval
The study received ethical clearance from the Tanzania National Institute of Medical Research (#NIMR/HQ/R8a/Vol.IX/3494), the Muhimbili University of Health and Allied Sciences Research and Ethics Committee (#307/323/01), the John F. Kennedy Medical Center Institutional Review Board (#08062019), the Boston Children’s Hospital Institutional Review Board (#P00033242), and the use of deidentified data was exempted from review by the Emory University Institutional Review Board (no number provided for exempted studies). Participants gave informed consent to participate in the study before taking part.
Acknowledgments
We would like to thank the patients and their caregivers who enrolled in this study. We also thank the clinicians who volunteered their time to respond to the surveys.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
KPM and CPD are joint senior authors.
Contributors CAR, RK, RCI, AS, EG, HKM, CRS, MN, KM and CD conceptualised and designed the study. CAR, RCI, JK, AS, EG, CRS, KM and CD oversaw data collection and verified the underlying data. CAR and ALW verified the underlying data. ALW conducted the statistical analyses. CAR wrote the first draft of the manuscript. CAR, RK, RCI, JK, Y-JGC-N, AS, EG, HKM, CRS, ALW, MN, CRM, CGW, RFB, KM and CD interpreted the data, reviewed, and provided input to the final draft. CAR had final responsibility for the decision to submit for publication and is responsible for the overall content as the guarantor.
Funding The authors wish to acknowledge funding for this work from the National Institutes of Health (K24 DK104676 and P30 DK040561 to CPD), the Boston Children’s Hospital Global Health Program to CAR, the Palfrey Fund for Child Health Advocacy to CAR, and the Emory Pediatric Research Alliance Junior Faculty Focused Award to CAR.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.