Article Text
PDF
Abstract
Objectives There is little experience in implementing the WHO Standards for improving the quality of care (QOC) for children. We describe the use of 75 WHO-Standard based Quality Measures to assess paediatric QOC, using health workers (HWs) as data sources.
Design Cross-sectional study.
Setting 12 Italian hospitals.
Participants The minimum target of 75% of HWs was reached in all facilities; answers from 598 HWs were analysed.
Primary and secondary outcome measures 75 prioritised WHO Quality Measures were collected using a validated, and Italian-language questionnaire exploring views of HWs providing care to children. A QOC index was also calculated based on the assessed Quality Measures.
Results In both the domain of resources and work organisation, most Quality Measures showed a high overall frequency of reported ‘need for improvement’, with high variability across hospitals. Key needs for improvement included: availability of clear and complete protocols (eg, on paediatric emergencies: 44.6%; range 10.6%–92.6%); clear hospitalisation criteria for diarrhoea (50.5%; range 30.3%–71.7%); number of hand-washing stations (13.2%; range 3.4%–37.0%); equipped working rooms with computers for HWs (66.1%; range: 32.1%–97.0%); training (eg, on pain management: 43.5%; range 17.9%–76.7%), periodic discussion of clinical cases (43.5%; range 8.1%–83.7%) audits (48.8%; range 29.7%–76.7%); and all indicators related to system to improve QOC. Factors significantly associated with a lower QOC Index included HWs working in facilities in Southern Italy (p=0.001) and absence of a paediatric emergency department (p=0.011).
Conclusions The use of the 75 prioritised Quality Measures, specific to HWs provide valuable data on paediatric QOC, which can be used to drive a quality improvement process.
- Child Health
- Infant
Data availability statement
Data collected and analysed in the current study are not publicly available but are available from the corresponding author on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Previous studies conducted in the European region revealed gaps in the quality of care (QOC) for children, but the perspective of service providers has been rarely investigated.
WHAT THIS STUDY ADDS
The utilisation of 75 prioritised WHO-Standard based Quality Measures to assess service providers’ perspective on paediatric QOC across 12 Italian hospitals revealed large heterogeneity across facilities, with several key common gaps, in particular related to availability of resources and organisation of care.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
The prioritised Quality Measures allows for the identification of both general and facility-specific gaps in paediatric QOC. Quality improvement initiatives should consider service providers’ perspectives, and focus on both a core list of priority common across facilities, and specific needs at each facility level.
Background
Quality of care (QOC) has been recognised by the WHO as a key aspect of the child health agenda.1 Currently, the levels of QOC still remain substandard in many different paediatric settings, in both low-income and high-income.2–6 Specific to the high-income countries setting, several studies in Europe and the USA reported inappropriate hospitalisations, medical errors, drug overuse, inadequate pain management, unsatisfactory patient experience of care and lack of resources, such as shortage of staff.7–10 Poor QOC may lead to sicker patients, more disabilities, lower confidence in the healthcare system and higher global social costs.11
Gaps in QOC have been found in facilities of different types (eg, both public and private) and levels.12 Moreover, health workers’ (HWs’) perspective on QOC has been rarely investigated.13 Including the voice of healthcare staff in assessing QOC may help in identifying key aspects in QOC gaps, and in building local ownership and commitment to drive changes.2
Recently, WHO developed a list of ‘Standards to Improve the Quality of Care for Children and Young Adolescents at Facility Level’.14 The eight standards are organised in three main domains — provision of care, experience of care and health system resources. One of the first recommended step in implementing the WHO Standards is conducting a baseline assessment to analyse existing QOC.14
Given that WHO Standards are relatively recent,14 there is little experience in their application. Therefore, we established a multicentre project called CHOICE (Child HOspItal CarE). Data collection tools for the project were developed and validated.15 This paper is part of a series of papers reporting key results on the baseline assessment of the CHOICE study in Italy. Specifically, this paper focuses on data collected on QOC for children when using HWs as a source of data for a prioritised list of QOC measures.
Methods
Study design
This was a multicentre cross-sectional study, and it is reported according to the Strengthening the Reporting of Observational Studies in Epidemiology statement.16
Study population
The study was conducted in the 12 hospitals providing paediatric care and participating in the CHOICE study, distributed across the national territory (North, Centre and South) (online supplemental file 1). Intentionally, study sites were chosen to have different characteristics in terms of the organisational structure (ie, facility level and type) and volume of work (online supplemental file 2).
Supplemental material
HWs included in the study were paediatricians, residents in paediatrics, and nurses routinely assisting children in the paediatric wards or emergency departments (EDs) of participating centres. An HW coordinator was identified in each study site, and he/she directly and orally invited other HWs to answer an anonymous online survey by email or other channels (eg, WhatsApp) after providing their consent to study participation. HWs with the following characteristics were excluded: HWs not involved in clinical care, or working only on surgical or neuro/psychiatric conditions, or other less common conditions not considered as part of ‘general paediatrics’; prolonged absence from work (eg, maternity leave) and/or unavailability during the study period. The study had a predefined target to collect answers from at least 75% of staff, that is, paediatricians, residents in paediatrics and nurses routinely assisting children in the paediatric wards or EDs, in each participating facility.
Study variables and data collection tools
A validated questionnaire15 was used to collect data. The process of prioritisation of the WHO Quality Measures and of questionnaire validation has been reported elsewhere15 and is summarised in the online supplemental file 3.
Both health service users (children and their parents) and health providers (HWs at facility level) were involved in the CHOICE study in multiple stages. As a first step, in 2019–2020 they were involved in the prioritisation of Quality Measures, thus affecting the selection of research outcomes. Second, they were involved in the validation of data collection tools,15 which included collecting their opinion on the acceptability of the questionnaire which resulted in good values of validity, reliability, acceptability, perceived utility and internal consistency.15 Finally, their opinion on QOC was actively collected; more specifically, the opinion of service users was collected on 75 prioritised Quality Measures reported in a separate publication17, and the opinion of service providers was collected on another 75 prioritised Quality Measures, reported in this publication. In each facility, HWs were involved in the dissemination of study findings (year 2022–2023), and in planning quality improvement interventions. In the nearest future we plan to further involve the general public in data dissemination.
Among the 75 Quality Measures collected from service providers’ perspective, 40 were pertinent to physical resources, 25 to the organisation of care and 10 to the organisational changes related to the COVID-19 pandemic (online supplemental file 3). Each Quality Measure could be rated by respondents based on a qualitative three option scale about the adequacy of service, including the answers ‘Yes’, ‘No, needing some improvement’ and ‘No, needing substantial improvement’. Questionnaires were delivered in Italian, the same language as the validation study,15 between 30 April 2021 and 31 January 2022 (during the COVID-19 pandemic), using REDCap 8.5.21—2021 Vanderbilt University, via a centralised platform by self-completion. The questionnaire requested the HW to report on their views on QOC at the time of the questionnaire administration. For questions on education, the questionnaire referred to the education received in the year preceding the survey administration. Data on HWs characteristics were also collected.
Data analysis
Collected data were preprocessed removing cases that were missing information on ≥90% of all close-ended questions including those on sociodemographic variables. Cases with the same answers to all close-ended questions and the same date of questionnaire completion (where available) were identified as suspected duplicates and only the most recent entry was kept for analysis.
We conducted a descriptive analysis of respondents’ characteristics and Quality Measures, presenting data on the overall sample and by hospital, for each domain and subdomain of QOC explored (physical resources, work organisation and organisational changes related to the COVID-19 pandemic). In the primary analysis, we presented together the two answers ‘No, need for some improvement’ and ‘No, needing for substantial improvement’. To further assess findings, the frequency of ‘No, needing substantial improvement’ answers were also presented as sensitivity analyses.
A QOC index was calculated, based on the WHO framework domains,14 and based on similar index developed using the WHO Standards for improving the quality of maternal and newborn care.18–20 Online supplemental file 4 briefly summarises the scoring system. Only HWs providing all answers to the 75 WHO Quality Measures contributed to the QOC indexes. The QOC indexes were graphically presented with kernel density curves and described using median and IQRs. A comparison between the domain-specific QOC indexes was performed using the Kruskal-Wallis test.
Associations between the total QOC index and HWs characteristics (ie, department; gender; years of working experience in the paediatric field; years worked in the study hospital; year of questionnaire completion) adjusted for hospital characteristics (online supplemental file 1) were explored using a general linear model using Gaussian family with identity link function and robust standard errors. For the selection of the optimal model, automatic backward elimination method was applied. For factor variables, the category with the highest frequency was selected as a reference. Findings were presented with β coefficients with 95% CIs and p value.
A p value of <0.05 was taken as statistically significant. Stata/SE V.14.0 (Stata Corporation, College Station, TX, USA) and R V.4.1.1 was used for data analysis.
Results
Characteristics of the sample
In each facility the target of collecting answers from at least 75% of staff was reached. Of 627 HWs participating the survey, a total of 598 were analysed across the 12 hospitals and included 185 (30.9%) paediatricians, 138 (23.1%) residents in paediatrics and 227 (38.0%) nurses while 48 (8.0%) declared to have a professionality among those included in the study without disclosing it (figure 1). The majority of participants were female (74.9%), working both in paediatric clinic and emergency room (49.3%). Most participating HW had a working experience of 1–5 years or greater than 10 years in the paediatric area (41.5% and 36.3%, respectively) (online supplemental file 5).
Flow diagram. 1Missing information on all close-ended questions including sociodemographic questions; 2suspected duplicates identified as cases with same answers to close-ended questions including sociodemographic questions and, when available, same date of questionnaire completion.
Quality Measures
Quality Measures highly differed among hospitals in all subdomains explored, making it difficult to identify common area of strengths and weakness across facilities (tables 1–3, online supplemental files 6–8).
Physical resources: need for improvement
Work organisation: need for improvement
Reorganisational changes due to COVID-19: need for improvement
Specifically, in the domain of physical resources (table 1, online supplemental file 6), a small group of measures had low overall frequencies of reported ‘need for improvement’, with low variability across hospitals. These included: the availability of antibiotics/bronchodilators (2.8%; range: 0.0%–11.1%) and of oxygen (1.5%; range: 0.0%–7.4%). A second small group of Quality Measures had low overall frequencies of reported ‘need for improvement’, but with high variability among different hospitals. These included: the availability of a 24/7 working triage (5.5%, range: 0.0%–25.9%); of surgical supply and personal protective equipment (7.0%; range: 0.0%–33.3%). A third small group of Quality Measures had high overall frequencies of ‘need for improvement’, with low variability across hospitals. These included: system to review care provided to referred children (75.8% of HW reporting a need for improvement; range across hospitals: 67.9%–86.7%) and visible rehydration algorithm clearly exposed on the ward walls (79.3%; range: 63.6%–92.6%). All the remaining Quality Measures had high overall frequencies of need for improvement with high variability among different hospitals.
In the domain of work organisation (table 2, online supplemental file 7), there was only one Quality Measure with low overall frequency of reported ‘need for improvement’, with low variability across hospitals: the availability of calendar of HW shifts (14.4%, range: 3.6%–23.9%). All other Quality Measures had high overall frequencies of ‘need for improvement’ with high variability among different hospitals. This included, as example: the availability of a written leadership structure (46.0% of HWs reporting a need for improvement; range 24.1%–67.4%); several Quality Measures related to capacity development, such as training on common paediatric diseases (42.8%; range: 25.8%–60.0%), on pain management (43.5%; range 17.9%–76.7%), on respectful care (71.2%, range: 56.4%–81.1%), on communication/counselling (71.4%; range: 61.5%–78.4%), and onsite education and supervision (58.5%; range 42.9%–77.8%), periodic discussion of clinical cases (43.5%; range 8.1%–83.7%), audits (48.8%; range 29.7%–76.7%), and all indicators related to system to improve QOC.
In the organisational changes due to COVID-19 domain (table 3, online supplemental file 8) two measures showed low reported frequencies of need of improvement, and low variability: the possibility to perform nasal-pharyngeal swabs to all admitted patients (2.0%, range: 0.0%–14.8%) and to caregivers (2.2%, range: 0.0%–14.8%). Another two measures showed low reported frequencies of need of improvement, but high variability: access to hand-washing station (10.4%, range: 0.0%–30.2%) and availability of personal protective equipment (12.2%; range: 0.0%–37.0%). All other measures showed relevant frequencies of need of improvement, and high variability.
Sensitivity analyses
Findings of the sensitivity analyses were similar to findings of the primary analysis (online supplemental file 9) with all hospitals reporting need for substantial improvements in the domain of physical resources with the lowest performing Quality Measures being system to review care provided to referred children, and rehydration algorithm for diarrhoea clearly exposed on the ward wall. Education and QOC were confirmed as priority subdomains for work organisation domain, while relatively small gaps were reported for reorganisational changes due to COVID-19 domain.
QOC index and multivariate analysis
QOC indexes (domain-specific and total) differed significantly among hospitals (figure 2, online supplemental file 10) with the lower median values being in the domain of work organisation (60.0, IQR 46.0–78.0 vs physical resources: 81.2, IQR 72.5–88.8 vs organisational changes due to COVID-19: 95.0, IQR 85.0–100.0, p<0.001).
QOC index by hospital (n=514). QOC, quality of care.
Based on the multivariate analysis (table 4), both paediatricians and resident in paediatrics reported higher QOC index compared with nurses (p<0.001 for both comparison). HW with <1 year of work in the study hospital reported higher QOC index for compared with HW working 1–5 years (p=0.005). Hospital with higher numbers of children admitted in short stay-observation associated with higher total QOC index (β=1.29 for each hundred children, p<0.001).
Associations between relevant variables and QOC index, multivariate linear regression (n=454)
Hospital characteristics significantly associated with lower total QOC index were geographical location — with HW working in Southern Italy reporting lower QOC index compared with HW working in Northern Italy (p<0.001)—, absence of paediatric ED separated from adult ED (β=−16.43, p=0.011) and absolute number (hundreds) of hospitalised children (β=−2.11, p=0.002).
Discussion
This is the first report on the use of 75 prioritised Quality Measures derived from the WHO Standards for improving the QOC for children and young adolescents in health facilities14 to assess the health providers perspective. As a general lesson, the study suggests that the use of the 75 WHO-Standard based Quality Measures, as prioritised by the CHOICE project, can help to identify key gaps in QOC in paediatric facilities.
The study also generated new evidence on the QOC in paediatric hospital care in Italy, highlighting: (a) a large variability in QOC among participating hospitals; (b) areas where all hospitals had major gaps; (c) areas where all hospitals had good practices. Several of the key gaps highlighted in this study call for urgent action. Clearly, essential resources, such as computers and working rooms, are needed to improve performance.
In regard to staff training, key gaps observed (training on communication, respectful care, ethics) should be addressed. Quality improvement training programmes, in particular during residency, allows trainees to develop necessary skills to deliver high-quality patient care.21–25 Some hospitals also reported the absence of clear and complete information on medical records. Inadequate documentation and poor communication may result in significant harm to children. Several studies suggest that the use of electronic patient records may increase efficiency and even substitute for some in-person healthcare visits, even in paediatric and neonatal settings.26 27
The organisational changes due to the COVID-19 pandemic did not significantly impact on QOC from the HWs perspective, showing less gaps than the other explored domains. This represents an important result suggesting that all hospitals may have responded to COVID-19 pandemic with similar patterns, thus HWs from different hospitals showed similar perspectives. Moreover, questionnaires were administered just during the pandemic period, and since the COVID-19 has enormously impacted HW’s clinical assistance, it was important to retain HWs perspective on this specific topic. The lack of information on COVID-19 situation represented the only problematic aspect emerged in some hospitals, as this may be due to the absence of specific protocols/procedures at the beginning of the pandemic. On the other hand, even if most HWs declared the presence of an adequate number of hand-washing dispenser, a great variability was noted. Alcohol-based hand rub dispensers at the point-of-care are strongly recommended by the WHO, especially in an ED setting.28
Some hospitals’ and HWs characteristics significantly affected QOC index. Medical staff reported higher QOC index in comparison to nurses. This may be explained by both knowledge among staff (eg, on existence of specific resources and procedures) and both by subjective judgement, which may be affected by many factors including previously described perception among nurses of a high work overload and lack of autonomy when compared with a low salary,29 thus generating fatigue and depression.30–32 Future initiatives should therefore aim at collecting views of staff with different profiles, triangulating them with other data sources (eg, direct assessment) and discussing findings in a participatory manner with the whole staff involved in providing care.
The finding of HWs working in Southern Italy reporting lower QOC is aligned with other existing evidence.33 Geographical disparity in QOC in Italy has been previously reported,34 with more than one-fifth of hospitalised children from southern regions being on average treated in hospitals in Northern or Central Italy, suggesting a lack of adequate paediatric services in the south.33 34 The significant association between absence of paediatric ED and a lower number of children hospitalised/admitted in short stay-observation with lower QOC index can also be explained with these variables being a proxy for facilities not fully specialised in paediatric care. Notably, existing guidelines recommend that every hospital ED should have the appropriate resources and staff to provide effective emergency care for children, with available separate spaces for paediatric patients.35 In previous studies, hospitals with higher paediatric patients’ volumes of work associated with greater adherence to established quality indicators.36 37
Limitations of this study include the relatively small sample of facilities; however, the CHOICE study did not aim at collecting large quantity of data, but rather at getting lessons on the implementation WHO Standards. Methods developed with this study could be easily translated elsewhere, and capitalised to collect large samples.
Findings of this study may have been affected by several response biases. However, this usually happens when participants are asked to self-report on subjective variables rather than on objective items (such as existence of specific equipment or procedure in place).38 Some Quality Measures may reflect knowledge of staff on specific procedures, and this is a relevant aspect of QOC. Data collected with the survey can be triangulated with other data sources (direct assessment, interview with hospital directors) to further increase comprehensiveness of data.
Length of the questionnaire may also have affected results. However, we reached the target of collecting the questionnaire from 75% of staff in each facility. The data collection tool was validated before use,15 and this should have increased data validity.
Conclusion
This study showed that the 75 WHO-Standard based Quality Measures may be useful to identify key gaps in QOC in paediatric hospitals, related to both hospital physical structure or to HWs specific roles. The high heterogeneity of perspectives across regions may also suggest that QOC in paediatric settings in Italy need further attention to achieve equitable outcomes.
Data availability statement
Data collected and analysed in the current study are not publicly available but are available from the corresponding author on reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and approval for data collection was obtained by the Ethical Committee of the Friuli Venezia Giulia Region for the coordinating centre (Study ID: 2976, RC 15/2019 Prot. 0035348 3 December 2019) and, by ethical committees of all participating hospitals. Anonymity in data collection was ensured by not collecting any information that could disclose participants’ identity. Participants gave informed consent to participate in the study before taking part.
Acknowledgments
CHOICE project was supported by the Ministry of Health, Rome—Italy, in collaboration with the Institute for Maternal and Child Health IRCCS Burlo Garofolo, Trieste—Italy. We would like to thank all CHOICE partners and volunteer who helped in the development of the questionnaire and all health workers who took the time to respond to this survey despite the burden of the COVID-19 pandemic. Special thanks to the CHOICE Study Group for their contribution to the development of this project and support for this manuscript. We are grateful to all health workers involved in data collection: Kevin Valentino from Alessandria; Lucia Grazia Tricarico, Mariateresa De Sario, Alessandra Pisani from Bari; Gian Luca Trobia, Vita Antonella Di Stefano from Catania; Riccardo Lubrano, Vanessa Martucci, Mariateresa Sanseviero, Silvia Bloise, Alessia Marcellino from Latina; Silvia Sordelli from Mantova; Francesca Tirelli from Padova; Roberta Parrino, Giuseppina De Rosa from Palermo; Dal Bo Sara, Troisi Angela, Federico Marchetti, Francesco Oppido, Giulia Sansovini from Ravenna; Marcella Massarotto from Treviso; Marco Greco from Firenze; Elia Balestra, Benmario Castaldo, Marta Magnolato from Trieste; Maristella Toniutti, Michela Pandullo, Sara Rivellini from Udine.
References
Footnotes
Collaborators CHOICE Study Group: Alessandria: Marta Gagliardi, Kevin Valentino, Chiara Grisaffi, Enrico Felici; Bari: Fabio Cardinale, Annunziata Lucarelli, Lucia Grazia Tricarico, Mariateresa De Sario, Alessandra Pisani; Catania: Maria Carla Finocchiaro, Laura Portale, Marta Arrabito, Gian Luca Trobia, Francesca Patanè, Vita Antonella Di Stefano; Firenze: Stefano Masi, Marco Greco, Emiliano Talanti, Andrea Iuorio, Anna Madera, Paola Stillo, Rosa Santangelo, Nicolò Chiti, Idanna Sforzi; Latina: Riccardo Lubrano, Vanessa Martucci, Mariateresa Sanseviero, Silvia Bloise, Alessia Marcellino; Mantova: Silvia Fasoli, Silvia Sordelli, Silvia Fasoli, Maria Luisa Casciana, Mariangela Labruzzo; Padova: Silvia Bressan, Silvia Galiazzo, Francesca Tirelli; Palermo: Massimo Lo Verde, Domenico Cipolla, Sarah Contorno, Sarah Contorno, Roberta Parrino, Giuseppina De Rosa; Ravenna: Dal Bo Sara, Troisi Angela, Federico Marchetti, Alessandra Iacono, Vanna Graziani, Carlotta Farneti, Francesco Oppido, Giulia Sansovini; Treviso: Stefano Martelossi, Chiara Stefani, Marcella Massarotto, Paola Crotti, Giada Sartor, Benedetta Ferro, Riccardo Pavanello, Marta Minute, Paola Moras; Trieste: Marzia Lazzerini, Egidio Barbi, Paolo Dalena, Ilaria Mariani, Elia Balestra, Benmario Castaldo, Marta Magnolato, Michele Maiola, Cozzi Giorgio, Amaddeo Alessandro, Alice Del Colle, Massimo Dagnelut; Udine: Ilaria Liguoro, Maristella Toniutti, Michela Pandullo, Sara Rivellini, Chiara Pilotto, Paola Cogo.
Contributors ML conceived the study and is the study guarantor. IL, IM and ML wrote the first draft which was revised by all authors. IM analysed data. All authors read and approved the final manuscript.
Funding This article is part of a supplement entitled Lesson learned with the implementation of the WHO Standards for improving the quality of care for children and young adolescents in health facilities (CHOICE study) in Italy published with support from WHO Collaborating Centre for Maternal and Child Health—Institute for Maternal and Child Health IRCCS Burlo Garofolo, Ospedale Ca’ Foncello’s Hospital—Pediatric Unit and Università Degli Studi di Padova—Dipartimento di Salute della Donna e del Bambino. The research was supported by the Ministry of Health, Rome—Italy, in collaboration with the Institute for Maternal and Child Health IRCCS Burlo Garofolo, Trieste—Italy (CHOICE Study-RC 15/19).
Competing interests No, there are no competing interests.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.