Article Text
PDF
Abstract
Objectives There is little experience in the use of the WHO Standards for improving the quality of care (QOC) for children at the facility level. We describe the use of 75 WHO Standard based Quality Measures to assess paediatric QOC, using service users as a source of data, in Italy.
Study design In a cross-sectional study including 12 hospitals, parents/caregivers of admitted children completed a validated questionnaire including 75 Quality Measures: 40 pertinent to the domain of experience of care; 25 to physical/structural resources; 10 to COVID-19 reorganisational changes. Univariate and multivariate analyses were conducted.
Results Answers from 1482 service users were analysed. Physical resources was the domain with the higher frequency of reported gaps in QOC, with key gaps (higher rates of responders reporting need for improvement and low variability across centres) being: (1) quality of meals (48.1%; range across facilities: 35.3%–61.7%); (2) presence of cooking areas (50.9%; range: 34.6%–70.0%); (3) spaces for family/friends (51.3%; range: 31.8%–77.4%). For experience of care, the most critical gap was the information on the rights of the child (76.6%; range: 59.9%–90.4%), with most other Quality Measures showing an overall frequency of reported need for improvement ranging between 5% and 35%. For reorganisational changes due to COVID-19 an improvement was felt necessary by <25% of responders in all Quality Measures, with low variability across centres. At the multivariate analyses, factors significantly associated with the QOC Index largely varied by QOC domain.
Conclusions The use of the 75 prioritised Quality Measures, specific to service users’ perspective, enabled the identification of both general and facility-specific gaps in QOC. Based on these findings, quality improvement initiatives shall focus on a core list of selected Quality Measures common to all facilities, plus on an additional list of Quality Measures as more relevant in each facility.
- Caregivers
- Qualitative research
Data availability statement
Data are available upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Although previous studies have shown gaps in the quality of care (QOC) for children, few have investigated QOC from the service users’ perspectives, and very limited is the experience in assessing QOC from the service users’ perspectives implementing the recently published WHO Standards for improving QOC for children.
WHAT THIS STUDY ADDS
The utilisation of WHO Standard based Quality Measures to assess service users’ perspective on paediatric QOC across multiple hospitals allowed for the identification of both general and facility-specific gaps, thus effectively contributing to benchmarking from a multidimensional standpoint and providing guidance for quality improvement initiatives.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
Successful implementation of the WHO Standard based Quality Measures based on service users’ perspective, as shown in our study, will help establish a common framework to analyse and compare QOC across multiple institutions and geographical regions, allowing for effective large scale monitoring and quality improvement initiatives to improve QOC for children globally.
Background
Access to high-quality health services is a fundamental aspect of human rights; yet evidence shows that this right is not equally guaranteed in all settings, with studies showing that patient characteristics, including lower sociodemographic and economic status, significantly associate with a lower access to high-quality care.1–3 Patients’ satisfaction with care received, which is often used in hospital surveys, is a very distal measure of quality of care (QOC), and previous literature suggests that it may correlate more with the final health outcome than with the actual QOC received.4 5 According to existing literature, several other domains of QOC, relevant to the experience of care (eg, perceived quality in the communication from health professionals), and the availability of physical resources (eg, quality in the hospital rooms, meals, etc),6 appear to be highly relevant to patients.6 7
Despite perceived QOC is considered of critical importance to achieve good health outcomes,8 9 so far there have been very few comprehensive assessments on the quality of paediatric care as perceived by service users. Available evidence mainly comes from studies reporting the views of adult patients.10–16 Furthermore, for many European countries, including Italy, still there is not an accreditation system in place, based on standards and quantifiable measures of QOC, aiming at standardising paediatric case management across facilities and at effectively ensuring that high QOC is delivered. A better understanding of the perceived QOC for children across different facilities could help develop more inclusive monitoring system, and tailor quality improvement initiatives to the needs of service users, as well as to the needs of individual facilities.11 17
In 2018, the WHO developed a list of ‘Standards to Improve the Quality of Care for Children and Young Adolescents at Facility Level’.18 The WHO recommends implementing these standards in healthcare facilities following the ‘Plan Do Study Act’ cycle. This approach implies, as a first step, a baseline assessment using prioritised Quality Measures as more relevant to the local context.18 However, there is lack of experience in using the WHO Standards,18 due to their recent publication.
In 2019, in collaboration with the WHO, we established a multi-country study named CHOICE (Child HOspItal CarE), aiming at documenting the implementation of the WHO Standards in high- and middle-income countries.18 Methods used to prioritise Quality Measures and the validation of data collection tools have been previously reported.19
This paper is part of a journal collection reporting key findings of the CHOICE study, related to lessons learnt from the first steps of the implementation of the WHO Standards19 in Italy. Specifically, the primary objective of the present study was to measure the quality of paediatric care delivered at the facility level based on the assessments and perspectives of service users. As a secondary objective we set out to determine factors associated with worse ratings in perceived QOC. The other papers of the collection focus on the assessment of QOC based on other sources of data (specifically, electronic medical records20–22 and opinions of hospital staff23), for a total of 175 WHO Standard-based Quality Measures reported (details in online supplemental table 1).
Supplemental material
Methods
Study design and setting
We conducted a multi-centre cross-sectional study at 12 public paediatric hospital facilities in Italy, distributed across the national territory and with different characteristics in terms of organisational structure (ie, facility level and type) and work volume (detailed characteristics are provided in online supplemental table 2). We reported our results according to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement24 (the STROBE checklist is provided as online supplemental table 3).
Study population
We surveyed service users, that is, parents and caregivers of paediatric patients younger than 15 years of age, admitted to the emergency departments observation units or paediatric wards (ie, excluding the neonatal unit) of the 12 participating sites between 30 April 2021 and 22 March 2022. Participation of children in completing the survey was encouraged where applicable, based on family assessment, for children older than 6 years of age. However, no strict age cut-off was applied. Decision about coparticipation of the child in answering the questionnaire was eventually made by the caregiver.
We excluded parents/caregivers if: (1) younger than 18 years of age; (2) with serious psychiatric disorders; (3) the child was admitted to the neonatal or paediatric intensive care at the time or in the case of a deceased child; (4) the child was hospitalised in neuropsychiatric or surgery departments; (5) the child was admitted to the day-hospital service. Caregivers of patients admitted to the settings reported in (3), (4) and (5) were not included as these services were not present across all participating sites. Patients admitted to the paediatric wards could be approached to participate in the study at any time during their hospitalisation. The study had a predefined target to collect answers for at least 100 caregivers in each participating facility.
Data collection tools and procedures
Parents/caregivers were asked to complete a validated questionnaire on QOC either online or in paper format.19 Decision on the administration modality was made by individual sites considering their specific context, with the purpose of maximising users’ response rates. Caregivers were asked to complete the questionnaire on the day it was administered. The questionnaire was developed ad-hoc for the CHOICE project by the research team. The process of prioritisation of the WHO Quality Measures and of the questionnaire validation has been detailed elsewhere19 and is briefly summarised in online supplemental table 1. Briefly, after the initial categorisation and prioritisation of WHO Quality Measures, the tool was translated into the Italian language and optimised through a Delphi process involving end-users and an international team of experts. During formal validation, the final version of the questionnaire, including 75 Quality measures, showed good validity, reliability, acceptability and perceived utility measures, as well as internal consistency.24 Of the 75 Quality Measures, 40 were pertinent to the domain of experience of care, 25 to physical/structural resources for the family (eg, beds, toilets, etc) and 10 to reorganisational changes related to the COVID-19 pandemic. Each quality measure could be rated by responders based on a qualitative three option scale about adequacy of service, including the answers ‘Yes’, ‘No, needing some improvement’ and ‘No, needing substantial improvement’. Few additional questions collected sociodemographic data of responders.
Parents/caregivers were approached during the clinical shifts by research staff who explained about the purpose of the study and about the questionnaire sections. Staff remained available for any questions that might have arisen during the completion of the questionnaire. Data were collected using REDCap (Research Electronic Data Capture) 8.5.21 Venderbilt University via a centralised platform either directly from responders or by entering data from the paper-based version of the questionnaires into REDCap by research staff.
Data analysis
The minimum sample size for inclusion for each hospital of 100 cases was reached. Data were preprocessed removing cases with missing information on key variables or on more than 90% of all close-ended questions including sociodemographic questions variables, in line with previous studies.25 Cases with same answers to sociodemographic questions and other close-ended questions and, when available, same date of questionnaire completion were identified as suspected duplicates and removed.
We conducted a descriptive analysis of respondents’ characteristics and of the results of the Quality Measures assessed. Data were presented as absolute frequencies and percentages, by each facility and on the overall sample. For each Quality Measure the percentages of responses merging the two answers ‘No, needing some improvement’ and ‘No, needing major improvement’ were graphically represented by participating facility and overall. For the domain of experience of care and patients’ rights (Section B of the Questionnaire), the subsection on the Rights of the Child, differently from the other quality measures, included only ‘Yes’ or ‘No’ response options, with the latter being graphically represented. To further assess findings, the frequency of ‘No, needing major improvement’ answers were also presented as sensitivity analyses.
Key gaps in overall QOC were identified with Quality Measures presenting the following two criteria: (1) poor ratings (defined as a frequency >40% of all responders reporting an improvement was needed) (2) and low variability (defined as a range in the frequencies across participating centres of <50%). Areas of good QOC were identified with Quality Measures presenting the following two criteria: (1) good ratings (defined as a frequency <20% of all responders rating that an improvement was needed) and (2) low variability (defined as a range in the frequencies across participating centres of <50%).
Based on the WHO framework domains,18 and on previous experience in other studies on the WHO Standards,25 26 a QOC Index was calculated using predefined criteria.19 This scoring system was intended as a complementary quantitative synthetic measure of the overall QOC, to be interpreted only in conjunction with other indicators. The sum of all scores in a domain constituted the QOC Index in that domain and could range from 0 to 100 with higher QOC Index values representing higher adherence to WHO Standards. The total QOC Index was calculated as the sum of all QOC Index by domain and could range from 0 to 300. The QOC Indexes were described using medians and IQRs. A comparison between the QOC Indexes in the three domains of care was performed using the Kruskal-Wallis test.
We performed a multivariate analysis with a general linear model using gaussian family with identity link function to assess the association between the QOC Indexes for each domain and characteristics of each facility adjusted for caregivers’ characteristics and patients’ demographic and clinical characteristics. To convert the skewed distribution of the outcomes to normal, the Box-Cox transformation was applied to the data.27 For the selection of the optimal model, automatic Backward Elimination method was applied, based on Akaike information criteria value. Findings were presented with β coefficients (transformed back to the original scale by applying the inverse of the Box-Cox transformation) with p value of significance. A p value of <0.05 was taken as statistically significant. Stata/SE V.14.0 (Stata Corporation, College Station, TX, USA) and R V.4.1.1 was used for data analysis.
Results
Characteristics of the sample
Out of 1563 parents/caregivers accessing the online questionnaire and providing consent to participate, 1482 (94.8%) were included in the analysis after exclusion of cases missing information on >90% of key variables (figure 1).
Flow chart of selection of valid completed surveys for analysis. 1Missing information on all close-ended questions including sociodemographic questions; 2suspected duplicates identified as cases with same answers to sociodemographic questions and other close-ended questions and, when available, same date of questionnaire completion.
Characteristics of respondents and their children are summarised in online supplemental table 4 (A and B, respectively). Most of the responders (83.4%) were mothers and the median age was 38 years. In more than 80% of cases, both parents were Italian and 34.6% of respondents had a university degree or higher. In 33.1% of cases, children actively contributed to answering the questions along with their parent(s)/caregiver(s). Children were mostly younger than 6 years of age (53.3%) with both sexes equally represented. Approximately half (52.5%) were admitted to the emergency department. Overall, the majority (40.6%) were hospitalised for a period of time between 3 and 7 days.
Quality Measures
Figure 2 reports the findings of each of the 75 key Quality Measures by domain, by participating facilities and overall. Most Quality Measures assessments had large variations across centres.
Percentages of responses on the Quality Measures about physical resources for children and parents (A), experience of care and patients’ rights (B) and COVID-19 emergency (C) rated as needing improvement, by facility and overall.
The domain of physical resources for children and parents (Section A of the Questionnaire), including 25 Quality Measures, received the worst ratings, with overall percentages of responses stating an improvement was required ranging between 8.7% and 52.8% across the assessed Quality Measures (figure 2A).
Key gaps (higher rates of responders reporting a need for improvement and low variability across centres) were: (1) adequacy of the parents’ bed (52.8%; range 18%–71.2%); (2) quality of meals (48.1%; range: 35.3%–61.7%); (3) presence of cooking areas (50.9%; range: 34.6%–70.0%); 4) spaces for family/friends (51.3%; range: 31.8%–77.4%, online supplemental table 5A).
Identified areas of good QOC (lower rates of responders reporting a need for improvement and low variability across centres were: (1) the adequacy of children’s bed (13.3%; range: 2.9%–46.1%); (2) room lighting (18.0%; range: 5.3%–53.9%); (3) quality measures related to the subsections on ‘staff’, such as an adequate number of doctors (14.9%; range: 1.9%–45.1%), staff adequately qualified (11.3%; range: 4.9%–32.4%), staff motivation (16.1%; range: 5.8%–48.1%) and ‘supplies’ (diagnostic procedure availability and drug availability).
Results for the domain of experience of care and patients’ rights (Section B of the Questionnaire) are reported in figure 2B. The following Quality Measures related to the information on the rights of the child highlighted key gaps in the QOC (ie, higher rates of responders reporting a need for improvement and low variability across centres): convention on the rights of the child displayed in the hospital (76.6%; range: 59.9%–90.4%); information received about children’s rights (45.7%; range: 18.7%–62.7%); posters/brochures on formula feeding (75.3%; range: 64.5%–86.5%).
For the other subsections of this domain, overall ratings were better than for the domain of physical resources, with a perceived need for improvement ranging between 5% and 35% for all Quality Measures assessed, and variability across centres <50% for all the Quality Measures. Several Quality Measures identified good QOC (frequency <20% of all responders rating that an improvement was needed and low variability) (online supplemental table 5B). More specifically, the Quality Measures receiving the best ratings (<10% of overall responses stating an improvement was needed) were: respect of dignity of the child and parents (5.7%; range: 0.0%–22.6%), parents allowed to be always present (5.8%; range: 2.1%–20.6%), staff providing information and clarification (9.1%; range: 4.0%–26.5%), effective staff communication with the child (7.9%; range: 3.9%–31.4%), effective staff communication (9.9%; range: 4.7%–26.5%) and payment for treatment received (6.9%; range: 3.3%–19.6%).
With respect to the domain of organisational changes due to COVID-19 (Section C of the Questionnaire), all Quality Measures identified good QOC (figure 2C, online supplemental table 5C). The last question (item 10 in figure 3C) about the perceived fear that the parents/caregivers or their children could contract COVID-19 at the hospital (which had a different type of responses) was rated as ‘a lot’ or ‘very much’ in slightly more than 25% of cases overall, however with a wide range (range: 18.1%–61.7%).
QOC Index by hospital. QOC, quality of care.
Sensitivity analyses
Findings of the sensitivity analyses were similar to the findings of the primary analysis (online supplemental table 6). The Quality Measures with the highest frequency of responses rated as ‘need a major improvement’ were found in the domain of physical resources: adequacy of a parent’s bed (52.8%; range: 18%–75.5%); adequacy of restrooms with respect to patient and parents’ needs and care (48%; range: 24.1%–77.5%); the quality of meals (48.1%; range: 35.3%–61.7%); the presence of cooking areas (50.9%; range: 34.6%–70%) and spaces for family/friends (51.3%; range: 31.8%–77.4%). ‘Staff’ and ‘supplies’ were again the sub-domains with the best ratings for physical resources. Adequacy of structures for parents and play areas were confirmed as priority aspects for the domain of experience of care, while small gaps were reported for the COVID-19 domain.
QOC Index and multivariate analysis
QOC Indexes differed among hospitals (figure 3, online supplemental figure 1) with overall lower median values being in the domain of physical resources (86.0 IQR 72.0–94.0 vs experience of care and patients’ rights: 86.3 IQR 77.5–91.3 vs reorganisational changes due to COVID-19: 100.0 IQR 90.0–100.0, p<0.001).
At the multivariate analyses (online supplemental table 7), factors significantly associated with the QOC Index varied in each domain. For the domain of physical resources, factors significantly associated with higher scores of the QOC Index were: patient’s country of birth other than Italy (β coeff=29.94, p value=0.04), non-Italian origin of both parents (β coeff=32.11, p value<0.01), patient admission to general paediatric wards (compared with the emergency department) (β coeff=18.89, p value=0.04), completion of the survey by caregivers other than the mothers (β coeff=41.89, p value=0.02), admission to tertiary care level facilities (β coeff=49.75, p value=0.02), admission to larger facilities (in terms of higher number of paediatric beds (β coeff=24.43, p value<0.01)), higher number of hospitalised children (β coeff=18.89, p value=0.01) and higher number of paediatricians (β coeff=7.19, p value=0.02). Geographical location of the facility in the South of Italy (β coeff=−39.79, p value<0.01) and facilities with a higher number of short-stay observation admissions in the emergency department (β coeff=−26.84, p value<0.01) resulted as negatively associated with the QOC Index.
For the domain of experience of care and patients’ rights factors significantly associated with higher scores of QOC Index were: admission to larger emergency departments (in terms of higher number of annual paediatric visits (β coeff=4.87, p value=0.02) and short-stay unit admissions (β coeff=13.26, p value<0.01). Completion of the survey without participation of the child (in responding to the questions) (β coeff=−29.53, p value<0.01), admission to university hospitals (β coeff=−31.47, p value<0.01) and admission to hospitals with higher annual paediatric hospitalisations (β coeff=−10.84, p value<0.01) were negatively associated with the overall QOC Index.
For organisational changes due to COVID-19, factors associated with a higher QOC Index were: completion of the survey by fathers (β coeff=74.13, p value<0.01), admission to tertiary care level facilities (β coeff=88.70, p value<0.01), and admission to larger facilities (in terms of higher number of hospitalised children (β coeff=64.31, p value<0.01), and higher number of paediatricians (β coeff=52.21, p value<0.01)). Several factors significantly associated with lower QOC Index scores: full time employment status of the caregiver (β coeff=−70.22, p value=0.02), admission longer than a week (β coeff=−69.17, p value=0.04), geographical location of the facility in the South of Italy (β coeff=−81.75, p value<0.01), completion of the survey without participation of the child (in responding to the questions) (β coeff=−69.26, p value<0.01), and number of short-stay unit admissions (β coeff=−71.16, p value<0.01).
Findings for the total QOC Index were similar to the previous ones: higher QOC Indexes were associated with tertiary care level and larger facilities (in terms of total beds in the emergency department (β coeff=167.78, p value<0.01) and number of hospitalised children (β coeff=150.56, p value<0.01), paediatricians and residents (β coeff=96.28, p value<0.01)); full time employment status of the caregiver (β coeff=−154.04, p value<0.01), geographical location of the facility in the South of Italy (β coeff=−213.33, p value<0.01), completion of the survey without participation of the child (β coeff=−162.08, p value<0.01), number of short-stay unit and total admissions (β coeff=−179.36, p value<0.01) were associated with lower indexes.
Discussion
This is the first study, to our knowledge, on the use of 75 prioritised Quality Measures derived from the WHO Standards,18 to assess QOC for children, as perceived by service users. The study suggests that the 75 WHO Standard based Quality Measures, as prioritised by the CHOICE project, can effectively be used to identify key gaps and areas of success in QOC for children across settings. The study also generated new evidence on QOC for children. While this is specific to the context of the study, study methods allow easy replications in other settings. The use of a validated questionnaire19 and a standardised methodology allows comparison of quantitative data across facilities and over time. Collection of services users’ views is critical to improve QOC,9 and its institutionalisation may favour real improvements, with benefits for children and their families.
Based on the findings of this study, the domain of physical resources for children and parents and the domain of experience of care were the ones perceived as most in need of improvement. The large variation in perceived QOC observed across centres has been confirmed by all other CHOICE studies, and underscores the need for actions to better standardise QOC for children in Italy. Study findings suggest that national quality improvement initiatives may focus on a common core list of indicators, plus a list of indicators as locally relevant in each individual facility. This new evidence is being used to drive a quality improvement process across participating facilities, which will be reported separately.
Respondent/patient level characteristics significantly associated with the perceived QOC, as expressed by the QOC Index, varied across the different QOC domains, with some exceptions. Some facility-level factors—admission to tertiary care and to larger facilities—similarly influenced in a positive manner service user’s perceptions of the two domains of physical resources and COVID-19 reorganisation changes, while geographical location of the facility in the South of Italy and higher number of short-stay unit admissions in the emergency department negatively associated with these two domains. On the other side, care at large emergency departments—as determined by their annual census and number of short-stay unit admissions—was positively associated with better perceived QOC in the experience of care domain. These results highlighted the complexity of factors that may affect the perceived QOC, and underscore that while some domains of QOC may be interlinked, others may be completely separate, thus suggesting the need to collect multiple indicators of QOC to allow a comprehensive assessment.
Only limited research has investigated QOC received by paediatric inpatients from a service users’ perspective,10 11 17 and before the WHO Standards were released.18 Toomey et al10 developed and tested the Consumer Assessment of Healthcare Providers and Systems Hospital Survey-Child Version, which is a publicly available standardised survey of inpatient experience of care, commissioned by the Agency for Healthcare Research and Quality, which has been used across 69 hospitals in the USA11 and in Canada.17 This tool includes 62 items, but it is focused mostly on communication. Similarly, the sole domain of communication has been assessed by Lee et al7 who surveyed caregivers of paediatric patients on their perception and satisfaction with physician communication. Their study found that communication expectation affects caregiver satisfaction and instruction retention and can thus impact quality of the healthcare experience. When compared with this literature our study appears to add valuable information.
Two recent large multicentre studies from China,2 28 although investigating satisfaction of solely or mostly adult patients, found that the hospital environment domain received lower satisfaction scores compared with the other domains assessed and that items such as hospital ward infrastructure, quietness, cleanliness, quality of meals, facilities to prevent falling, all needed improvement. Nevertheless, studies assessing overall patient satisfaction for either adult patient or paediatric patient care across different settings1 2 28 found inconsistent results with respect to the influence that hospital environment features seem to have on patient satisfaction. Physical and structural limitations of our inpatient public facilities are deeply rooted in our historical/cultural context. Hospitals, in some instances, over a century old, were originally located in the heart of towns, progressively expanded to accommodate new buildings, as modern medicine developed and population healthcare needs increased.
We acknowledge the limitations of this study. First, the study was presented to service users by clinical staff, and this may have resulted in selection bias. However, we cannot predict in which direction this may have affected results, considering that the survey was anonymous, local staff could not access answers, and both families perceiving a high QOC and those perceiving a low QOC may have had an interest in participating. Future studies should evaluate other methods for patient enrolment (eg, dedicated independent staff). However, if wishing to incorporate the assessment of QOC in routine practice, sustainability needs to be carefully considered. Second, online data collection may have selected responses from caregivers more familiar with electronic tools. Third, the high percentage of respondents with a university degree, not reflecting the national average population, is a selection bias, although it is difficult to understand how this has affected results.
Fourth, we acknowledge that addition of in-depth interviews with caregivers and focused group discussions, alongside the questionnaire results, would have led to a more comprehensive understanding of the QOC delivered as well as influencing factors, and opportunities for improvement. Finally, the survey was only available in Italian, thus limiting the participation of caregivers who were not sufficiently proficient in the Italian language. Non-Italian origin parents and immigrant parents29 may have different perceptions of QOC. For the future, we already planned to make the survey available in multiple languages.
Data availability statement
Data are available upon reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants. Approval for data collection was obtained by the Ethical Committee of the Friuli Venezia Giulia Region for the coordinating centre (Study ID: 2976, RC 15/2019 Prot. 0035348 3 December 2019) and by ethical committees of all individual participating hospitals. Anonymity in data collection was ensured by not collecting any information that could disclose participants’ identity. Participants gave informed consent to participate in the study before taking part.
Acknowledgments
CHOICE project was supported by the Ministry of Health, Rome, Italy, in collaboration with the Institute for Maternal and Child Health IRCCS Burlo Garofolo, Trieste, Italy. We would like to thank all CHOICE partners and volunteer who helped in the development of the questionnaire and all health workers who took the time to respond to this survey despite the burden of the COVID-19 pandemic. Special thanks to the CHOICE Study Group for their contribution to the development of this project and support for this manuscript. We are grateful to all health workers involved in data collection: Marta Magnolato, Michele Maiola, Alice del Colle from Trieste; Anna Madera Antonio Scarcella Nicolò Chiti from Firenze; Dal Bo Sara Troisi Angela, Alessandra Iacono, Vanna Graziani, Carlotta Farneti, Federico Marchetti from Ravenna; Gian Luca Trobia, Maria Carla Finocchiaro, Vita Antonella Di Stefano from Catania; Francesca Tirelli, Maria Auciello, Chiara Marchegiani Rizzoli from Padova; Kevin Valentino from Alessandria; R Lubrano, V Martucci, MT Sanseviero, S Bloise, A Marcellino from Latina; Lucia Grazia Tricarico, Mariateresa De Sario, Alessandra Pisani from Bari; Silvia Fasoli, Maria Luisa Casciana from Mantova; Ilaria Liguoro, Maristella Toniutti, Michela Pandullo, Sara Rivellini, Chiara Pilotto, Paola Cogo from Udine; Paola Crotti from Treviso; Roberta Parrino, Giuseppina De Rosa from Palermo.
Footnotes
Collaborators CHOICE Study Group: Alessandria: Marta Gagliardi, Kevin Valentino, Chiara Grisaffi, Enrico Felici; Bari: Fabio Cardinale, Annunziata Lucarelli, Lucia Grazia Tricarico, Mariateresa De Sario, Alessandra Pisani; Catania: Maria Carla Finocchiaro, Laura Portale, Marta Arrabito, Gian Luca Trobia, Francesca Patanè, Vita Antonella Di Stefano; Firenze: Stefano Masi, Marco Greco, Emiliano Talanti, Andrea Iuorio, Anna Madera, Paola Stillo, Rosa Santangelo, Nicolò Chiti, Idanna Sforzi; Latina: Riccardo Lubrano, Vanessa Martucci, Mariateresa Sanseviero, Silvia Bloise, Alessia Marcellino; Mantova: Silvia Fasoli, Silvia Sordelli, Maria Luisa Casciana, Mariangela Labruzzo; Padova: Silvia Bressan, Silvia Galiazzo, Francesca Tirelli; Palermo: Massimo Lo Verde, Domenico Cipolla, Sarah Contorno, Roberta Parrino, Giuseppina De Rosa; Ravenna: Sara Dal Bo, Angela Troisi, Federico Marchetti, Alessandra Iacono, Vanna Graziani, Carlotta Farneti, Francesco Oppido, Giulia Sansovini; Treviso: Stefano Martelossi, Chiara Stefani, Marcella Massarotto, Paola Crotti, Giada Sartor, Benedetta Ferro, Riccardo Pavanello, Marta Minute, Paola Moras; Trieste: Marzia Lazzerini, Egidio Barbi, Paolo Dalena, Ilaria Mariani, Elia Balestra, Benmario Castaldo, Marta Magnolato, Michele Maiola, Giorgio Cozzi, Alessandro Amaddeo, Alice Del Colle, Massimo Dagnelut; Udine: Ilaria Liguoro, Maristella Toniutti, Michela Pandullo, Sara Rivellini, Chiara Pilotto, Paola Cogo.
Contributors ML is the guarantor and conceived the study. SB, GS, PD and ML wrote the first draft which was revised by all authors. PD analysed data, with inputs from ML and all other authors. SB, GS, PD, EB, AMadera, FM, MCF, FT, EF, AMarcellino, SF, PC, RP, BC, RS, MT, GDR, VB, ML read and approved the final manuscript.
Funding This article is part of a supplement entitled Lesson learned with the implementation of the WHO Standards for improving the quality of care for children and young adolescents in health facilities (CHOICHE study) in Italy published with support from WHO Collaborating Centre for Maternal and Child Health, Institute for Maternal and Child Health IRCCS Burlo Garofolo. The research was supported by the Ministry of Health, Rome, Italy, in collaboration with the Institute for Maternal and Child Health IRCCS Burlo Garofolo, Trieste, Italy (CHOICE Study-RC 15/19).
Disclaimer Valentina Baltag is a staff member of the World Health Organization. The author alone is responsible for the views expressed in this article; they do not represent the decisions or policies of the World Health Organization.
Competing interests None declared.
Patient and public involvement Both health service users (children and their parents) and health providers (health workers at facility level) were involved in the CHOICE (Child HOspItal CarE) study in multiple stages. As a first step, in 2019-2020 they were involved in the prioritization of Quality Measures, thus affecting the selection of research outcomes. Secondly, they were involved in the validation of data collection tools, which included collecting their opinion on the acceptability of the questionnaire. Lastly, their opinion on quality of care was actively collected; more specifically, the option of service users was collected on 75 prioritized Quality Measure, and the opinion of service providers was collected on another 75 prioritized Quality Measure.26 In each facility health workers were involved in the dissemination of study findings (year 2022-2023), and in planning quality improvement interventions. In the nearest future we plan to further involve the general public in data dissemination.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.