BACKGROUND AND OBJECTIVES

Children living with medical complexity (CMC) experience frequent hospital admissions. Validated patient-reported experience measures may inform care improvements in this cohort. Our objectives were to examine the comprehensive inpatient experience of CMC by using a validated patient-reported experience measure and compare the results with all other respondents at 2 academic pediatric hospitals in a western Canadian province.

METHODS

Parents completed the Child Hospital Consumer Assessment of Healthcare Providers and Systems survey. Surveys were linked with inpatient records, and an accepted case definition was used to extract records pertaining to CMC. Results were reported as percent in “top box,” represented by the most positive answer choice to each measure. Odds of reporting a top box response were calculated while controlling for demographic and clinical features.

RESULTS

From October 2015 to March 2019, 4197 surveys (1515 CMC; 2682 non-CMC) were collected. Among CMC, the highest-rated measures pertained to being kept informed while in the emergency department, a willingness to recommend the hospital, and parents having a clear understanding of their role in their child’s care. The lowest-rated measures pertained to preventing mistakes and reporting concerns and the quietness of the hospital room at night. Compared with others, parents of CMC reported lower raw results on 20 of the 28 measures. They also reported lower a odds of reporting a top box score on 2 measures and higher odds on 1.

CONCLUSIONS

Parents of CMC revealed many perceived gaps. These findings can be used to inform strategies to improve care among CMC and policies to support the care of CMC and their families.

Children living with medical complexity (CMC) have health conditions that are associated with significant functional limitations.1  Previous studies have revealed that although CMC comprise <1% of the general population, they have a disproportionately higher use of the health care system, particularly acute inpatient care.15  It has been estimated that CMC account for ∼20% of all hospitalizations6  and one-third of health care spending for all children (ie, $100 billion).7  Given this, it is extremely important to understand the experiences with care that parents and caregivers of CMC report. In the context of the Institute for Healthcare Improvement Triple Aim Framework, this may lead to improved outcomes and lower health care costs.8  In short, a better understanding of how care is provided to CMC in the eyes of parents and caregivers can serve as a valuable source of data to inform quality improvement efforts.

Over the last decade, the Child Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey has emerged as a valid and reliable patient-reported experience measure to capture feedback from the parents and/or guardians of children who have been hospitalized.911  The Child HCAHPS brings the voice of parents and caregivers to quality improvement and performance measurement in the care for pediatric patients. Ongoing collection of Child HCAHPS data and linkage of these to other administrative data sources can provide an in-depth look at specific clinical populations and demographics. Preliminary results from its use across our jurisdiction12  as well as a correlational analysis to examine which questions contributed most to the overall rating of care question13  which is often used as a high-level indicator of hospital experience, have been reported. To date, however, in no studies have researchers looked at data or results from the Child HCAHPS as they apply to CMC. This is extremely important because CMC may have specific care needs that can be missed when looking at Child HCAHPS data from all patients in an aggregated fashion. Extracting data and insights from CMC may assist hospital administrators and policy makers in highlighting areas where care may be improved in a tailored fashion.14 

With this study, we sought to better understand the experiences with care that CMC received and compare these with the experiences of other children receiving care at the same health care facilities, with the aim to inform quality improvement strategies at the provincial level. Specifically, the primary objective of this study was to describe the comprehensive experience of CMC at 2 major academic pediatric provincial children’s hospitals by using the Child HCAHPS. The secondary objective was to compare these results with those obtained from all other patients who completed Child HCAHPS at these sites.

This quantitative study includes the collection of experiences of parents and guardians with the quality of care received by their children at 2 large academic pediatric hospitals in a western Canadian province. Provincial survey data were linked to administrative records, as provided by the Discharge Abstract Database (DAD)15  by using the patient’s personal health number, the hospital code (5-digit identifier), and the hospital discharge date.

Ethics approval for this study was granted by our university research ethics board. All survey and administrative data were provided to the research team as part of a research agreement with the provincial health authority.

The Child HCAHPS survey has been continuously administered across our province since October 2015. Respondents completed a modified 65-question version of the survey. This included 39 experience questions and 10 screening questions with the same wording and response options from the standard Child HCAHPS.9  An additional 10 experience and 6 screening questions were developed by our jurisdiction’s health authority by using parent interviews and focus groups. These additional items are included as part of the modified survey and have been described elsewhere.12  Surveys were completed within 6 weeks of the child’s discharge from 1 of 14 hospitals (2 stand-alone pediatric sites and 12 adult sites providing pediatric care) across the province. For the purposes of this study, we have only included discharges from the 2 stand-alone academic pediatric hospitals in the province because these sites provide the majority of care for CMC. Survey eligibility criteria included live discharges of children <18 years of age at the time of hospital discharge, with a minimum hospital stay of 24 hours (eg, overnight). Cases in which care occurred exclusively in an emergency department as well as those due to mental health concerns were excluded from sampling.

Surveys were administered by using a standard script, a list of approved prompts, and answers to frequently asked questions (eg, how telephone numbers were obtained, the purpose of the survey, what will be done with the data, etc). Randomly selected parents and guardians of eligible children were called from 9 am to 9 pm on weekdays and 10 am to 3 pm on Saturdays. To increase the potential for participation, eligible telephone numbers were dialed up to 9 times on varying days and times. During the most recent fiscal year of data (April 2018 to March 2019), the overall survey response rate was 67% (figure provided by the health authority). All respondents provided their verbal consent and were also asked to verify that the hospital discharge associated with the call did indeed take place. Each survey took 13 to 24 minutes to complete (median: 16 minutes).

Surveys were linked with corresponding inpatient records, from the DAD15  by using the patient’s personal health number, the hospital code (5-digit identifier), and the hospital discharge date. The DAD is a record of all hospital discharges and is coded according to standards established by the Canadian Institute for Health Information. Each record includes up to 25 diagnoses, as classified according to the Canadian version of the International Classification of Diseases, 10th Revision.16 

To determine which surveys were completed by parents and guardians of CMC, we used an administrative definition, as defined by Cohen et al.1  Any inpatient record with a most responsible diagnosis according to this definition was determined to be from a parent or guardian of a CMC. From this administrative definition, the list of eligible diagnosis codes may also be classified into 3 mutually exclusive subgroups (neurologic impairment, complex chronic conditions, medical technology assistance).1  To eliminate any potential confounding as a result of interhospital transfers, only surveys from cases in which children were discharged from the hospital (DAD discharge disposition of “04” or “05”) were included. The data collection and linkage processes are shown in Fig 1.

In addition to the survey responses, we examined a variety of demographic and clinical characteristics, as provided by the survey and inpatient data sets. Patient demographics included the child’s sex, age group, and health status (as reported by the parent or guardian). Respondent demographics included age group, education level, relationship to the child, and portion of time spent at the hospital with the child. Clinical characteristics included the hospital where care was received, admission type, total length of hospital stay, and discharge disposition. Given that some respondents may have previously completed a Child HCAHPS survey, a variable for first-time survey (yes or no) was created.

Demographic and clinical characteristics of the sample were reported by using descriptive statistics. Differences between CMC and other patients were assessed by using the Student's t test or χ2 test, as appropriate. Because the Child HCAHPS was developed to be used and interpreted at the measure-level, we classified survey questions into composite measures, as in previous works.912  A detailed description of the 18 standard Child HCAHPS measures and their calculation methodology is provided by the Agency for Healthcare Research and Quality.17  Additional survey items developed by our health authority were analyzed and presented as stand-alone items. For all measures (standard Child HCAHPS measures and additional organization-specific items), we reported the percent reporting “top box” scores.18 

After this, logistic regression was used to calculate the odds of reporting a top box response for each measure and/or question, according to CMC or non-CMC classification. Non-CMC composed the reference group, while adjusting for the demographic and clinical covariates outlined above. Corresponding 95% confidence intervals (CIs) were also calculated. Any odds ratio not containing the value of 1.0 was deemed statistically significant. For all logistic regression analyses, the c-statistic was calculated to report the amount of variance explained by the statistical model. Analyses were performed by using SAS Network Version 9.3 for Windows (SAS Institute, Inc, Cary, NC).

From October 2015 to March 2019, a total of 4197 surveys were completed across the 2 academic provincial children’s hospitals. This included 1515 surveys from parents and/or guardians of CMC (36.1% of all surveys). The demographic and clinical characteristics of patients and respondents are shown in Table 1. Just over one-half of CMC were male (51.5%), ∼3 of every 5 (60.9%) were <5 years of age, and less than one-half (47.3%) had an overall health rating of excellent or very good, as indicated by their parent or guardian at the time of survey administration. Among CMC, most respondents were the child’s mother (82.1%) and spent all or most of the time (95.8%) at the hospital with their child. When compared with other pediatric patients, CMC were significantly younger (mean: 4.9 vs 5.5 years; P < .01), had a poorer parent and/or guardian-reported health status, were admitted to hospital more on an elective basis (43.2% vs 23.5%l P <.01), had a longer length of hospital stay (mean: 9.6 vs 4.1 days; P <.01), and were discharged without support services less frequently (89.3% vs 98.1%; P <.01).

In Table 2, we present the results from all survey measures and questions, for parents and guardians of CMC and non-CMC. Among CMC, the highest-rated measures from the standard Child HCAHPS pertained to parents being kept informed while their child was in the emergency department (96.2%), a willingness to recommend the hospital (92.7%), preparation for leaving the hospital (81.7%), communication between parents and/or guardians and doctors (81.5%), and communication between parents and/or guardians and nurses (78.4%). In contrast, the lowest-rated measures from the standard Child HCAHPS pertained to preventing mistakes and helping to report concerns (43.1%), how often the child’s hospital room was quiet at night (46.9%), and how well doctors communicated with children (63.3%). Results from the 10 additional questions developed by the health authority ranged from 89.4% (parents having a clear understanding of their role in caring for the child) to 55.9% (providers having an understanding of the child’s condition). When compared with the non-CMC cohort, a lower percentage of parents and guardians of CMC reported top box scores on 13 of the 18 standard Child HCAHPS measures. The largest differences observed were for quietness of hospital room at night (46.9% vs 53.0%), responsiveness to the call button (66.7% vs 71.7%), staff paying attention to the child’s pain (76.9% vs 82.1%), and communication between parents and/or guardians and nurses (78.4% vs 82.9% reporting “always”). Additionally, CMC had lower results on 7 of 10 stand-alone organization-specific items. The largest differences observed were for providers having a clear understanding of the child’s condition (55.9% vs 64.3%), the overall rating of care from nurses (73.2% vs 78.0%), and providers doing everything they could to help with pain (76.2% vs 79.4%).

In Table 2, we also present the adjusted odds of parents of CMC reporting a top box score for each of the survey domains and questions, when compared with non-CMC. Parents of CMC reported lower odds of reporting a top box score for communication between nurses and parents and guardians (adjusted odds ratio [aOR] 0.85; 95% CI: 0.73–0.99) and providers paying attention to the child’s pain (aOR: 0.77; 95% CI: 0.62–0.96). Conversely, parents of CMC reported higher odds of reporting a top box rating for preventing mistakes and helping to report concerns (aOR: 1.25; 95% CI: 1.02–1.52). The c-statistic for the logistic regression models ranged from 0.57 to 0.66.

CMC experience hospitalization at a much higher rate than other children. With this descriptive study, we are the first to specifically examine the in-hospital experience of CMC using the Child HCAHPS survey. Our findings revealed that the overall experience of CMC in the pediatric hospitals studied was positive, with a high top box rating of overall care (74.2% rating 9 or 10 out of 10 [best]). Additionally, 92.7% of parents and guardians of CMC stated that they would “definitely recommend” the hospital to family members and/or friends. Through their responses, participants also highlighted potential areas for improvement. From the standard Child HCAHPS measures, these included topics such as quietness of the hospital room, preventing mistakes and helping parents and guardians to report concerns, and doctor communication with the child. From the additional organization-specific items, parents and guardians of CMC highlighted potential improvements, including providers’ understanding of the child’s condition, asking about the child’s usual behaviors, and in following-up on parents’ concerns/observations. When compared with other respondents, parents and guardians of CMC reported lower top box percentages on 20 of the 28 measures, which were examined. The greatest disparities reported were for questions pertaining to providers having a good understanding of the child’s condition and medical history, the quietness of the child’s hospital room at night, and responsiveness to the call button. Parents and guardians of CMC also reported significantly lower scores for nurse communication as well as in their overall rating of nursing care. These differences extended to the odds of reporting a top box score, while controlling for demographic and clinical features. Parents of CMC reported lower odds on 2 of the 28 measures examined (communication between nurses and parents and guardians and providers paying attention to the child’s pain) and higher odds on 1 measure (preventing mistakes and helping to report concerns).

Because this study is the first of its kind, we cannot make direct comparisons with others who have used the Child HCAHPS to examine the comprehensive hospital experience of parents and guardians of CMC. However, our findings echo those of previous qualitative reports in which researchers have highlighted gaps in care for CMC. It is important to note that because of the high risk and susceptibility of CMC, hospital admissions may not be entirely avoidable.19  Despite this, there are processes in care that, when improved, may have positive impacts on patients and families. As in our study, coordination of care, information sharing, and interactions between providers and children have been cited as areas in which care for CMC may be improved.2023  In our study, close to 90% of parents identified that they “definitely” understood their role in caring for their child. This aligns with results from a qualitative study of 17 parents who expected to continue providing care to their child.24  Additionally, 56% of parents of CMC felt health care providers had a good understanding of their child’s health condition, a lower proportion than parents of children without MC. A key theme from the previously cited qualitative study was “we know our child best,”24  and, in a further qualitative study, researchers identified that “parents needed skilled staff who could relieve them of medical responsibility, but to be involved in the care and in the decisions taken.”25  Our results may reflect this more in-depth perspective of parents and, perhaps, provide a potential explanation of this finding.

A previous study by Hirschfield et al26  revealed that prolonged hospital stay among CMC was a source of dysfunctional health care team collaboration and poor communication between parents and the care team. This finding was echoed in this study. The finding of the disparity in communication between nurses and parents of CMC versus non-CMC is an interesting one, given that nurses play a key role in care for patients. Although we did not have access to staffing information in our study, this may be a product of more nurses and staff who care for CMC (because of longer length of stay and care complexity).

There are some limitations to this study that warrant discussion. For instance, as in the work of Cohen et al,1  we defined the cohort of CMC from hospitalization data alone and did not include other sources such as primary care or emergency department visits. Thus, it is plausible that we underestimated the number of cases of CMC in our analyses because we may have missed cases that could be deemed complex in the eyes of providers or parents. It is possible that certain elements of care that may be important to CMC may have gone unasked. For example, missing from this quantitative study were elements to better understand the need for parent staff partnership with ongoing communication and negotiation of caregiving roles during hospitalization,24  support, and relief needs as well as parental trust in hospital staff.25 

Future work in which researchers integrate personal narratives to better understand the individual experience within the context of person-centered care (PCC) may be warranted in these areas.2729  Other limitations of our study pertained to the survey mode and protocol. In our jurisdiction, the Child HCAHPS survey is administered exclusively by telephone, in English only. Thus, our results may not be generalizable to a mail-out format or among parents or caregivers who do not speak English. These are important points because, in previous research among adult patients, researchers have identified HCAHPS score variations across different modalities (eg, phone, mail) and languages and shown higher scores with a telephone format, partly because of the possibility for social desirability.30  Lastly, although the bulk of pediatric care among CMC occurs at the 2 pediatric sites that we studied, there are some instances in which care is provided at other sites across our province. These were not included in this study because of the small number of cases with an accompanying survey. In light of this, future work by our team will examine care provided for CMC at these sites, ones that provide predominantly adult care.

In the current study, we explored the comprehensive hospital experience of parents and guardians in the care for their child living with medical complexity. With the data, we have highlighted areas for improvement, in addition to a gap that exists between the reports of experience for CMC and other children receiving care at the 2 pediatric hospitals studied. These are important findings because parents of CMC are not only experts in their child’s health, but, through their many experiences with the health care system, they provide valuable, informed feedback on how services may be improved. Their contributions are essential in the provision of patient and family-centered care31  as well as for learning health systems seeking to improve under the lens of the Triple Aim framework.8  Our findings are also useful, given that many of the items highlighted above have revealed a strong correlation with the overall rating of care and overall experience. If these items, particularly those that focus on communication between providers as well as between providers and patients and families were to be improved, one may expect the overall experience ratings to be bolstered.32  In future research, researchers may also explore the need for a survey that expands on the existing Child HCAHPS, to reflect the unique needs of CMC as well as their parents and caregivers.

We thank the parents and guardians who took the time to complete the survey. We also thank Ms Rose Petrovic and Ms Ting (Grace) Wang, Alberta Health Services for their assistance with data acquisition and linkage.

FUNDING: No external funding.

1.
Cohen
E
,
Berry
JG
,
Camacho
X
,
Anderson
G
,
Wodchis
W
,
Guttmann
A
.
Patterns and costs of health care use of children with medical complexity
.
Pediatrics
.
2012
;
130
(
6
):
e1463
e1470
2.
Berry
JG
,
Hall
DE
,
Kuo
DZ
, et al
.
Hospital utilization and characteristics of patients experiencing recurrent readmissions within children’s hospitals
.
JAMA
.
2011
;
305
(
7
):
682
690
3.
Burns
KH
,
Casey
PH
,
Lyle
RE
,
Bird
TM
,
Fussell
JJ
,
Robbins
JM
.
Increasing prevalence of medically complex children in US hospitals
.
Pediatrics
.
2010
;
126
(
4
):
638
646
4.
Simon
TD
,
Berry
J
,
Feudtner
C
, et al
.
Children with complex chronic conditions in inpatient hospital settings in the United States
.
Pediatrics
.
2010
;
126
(
4
):
647
655
5.
Bucholz
EM
,
Toomey
SL
,
Schuster
MA
.
Trends in pediatric hospitalizations and readmissions: 2010-2016
.
Pediatrics
.
2019
;
143
(
2
):
e20181958
6.
Kuo
DZ
,
Cohen
E
,
Agrawal
R
,
Berry
JG
,
Casey
PH
.
A national profile of caregiver challenges among more medically complex children with special health care needs
.
Arch Pediatr Adolesc Med
.
2011
;
165
(
11
):
1020
1026
7.
Berry
JG
,
Hall
M
,
Neff
J
, et al
.
Children with medical complexity and Medicaid: spending and cost savings
.
Health Aff (Millwood)
.
2014
;
33
(
12
):
2199
2206
8.
Berwick
DM
,
Nolan
TW
,
Whittington
J
.
The triple aim: care, health, and cost
.
Health Aff (Millwood)
.
2008
;
27
(
3
):
759
769
9.
Toomey
SL
,
Zaslavsky
AM
,
Elliott
MN
, et al
.
The development of a pediatric inpatient experience of care measure: Child HCAHPS
.
Pediatrics
.
2015
;
136
(
2
):
360
369
10.
Toomey
SL
,
Elliott
MN
,
Zaslavsky
AM
, et al
.
Variation in family experience of pediatric inpatient care as measured by child HCAHPS
.
Pediatrics
.
2017
;
139
(
4
):
e20163372
11.
Bruyneel
L
,
Coeckelberghs
E
,
Buyse
G
, et al
.
Validation of the Child HCAHPS survey to measure pediatric inpatient experience of care in Flanders
.
Eur J Pediatr
.
2017
;
176
(
7
):
935
945
12.
Kemp
KA
,
Ahmed
S
,
Quan
H
,
Johnson
D
,
Santana
MJ
.
Family experiences of pediatric inpatient care in Alberta, Canada: results from the Child HCAHPS
.
Hosp Pediatr
.
2018
;
8
(
6
):
338
344
13.
Ahmed
S
,
Kemp
K
,
Johnson
D
,
Quan
H
,
Santana
MJ
.
Identifying areas for improvement in paediatric inpatient care using the Child HCAHPS survey
.
Paediatr Child Health
.
2019
;
25
(
6
):
365
371
14.
Dewan
T
,
Cohen
E
.
Children with medical complexity in Canada
.
Paediatr Child Health
.
2013
;
18
(
10
):
518
522
15.
Canadian Institute for Health Information
.
DAD data elements 2018-2019
.
16.
Canadian Institute for Health Information
.
DAD data elements 2021-2022
.
17.
Agency for Healthcare Research and Quality
.
CAHPS child hospital survey measures
.
18.
Centers for Medicare and Medicaid Services
.
A note about HCAHPS “boxes”
.
Available at: www.hcahpsonline.org/SummaryAnalyses.aspx. Accessed November 10, 2020
19.
Nelson
BB
,
Coller
RJ
,
Saenz
AA
, et al
.
How avoidable are hospitalizations for children with medical complexity? Understanding parent perspectives
.
Acad Pediatr
.
2016
;
16
(
6
):
579
586
20.
Canadian Paediatric Society
.
Experiences in health care as ‘complex medical patients’
.
Paediatr Child Health
.
2016
;
21
(
5
):
235
21.
Shilling
V
,
Edwards
V
,
Rogers
M
,
Morris
C
.
The experience of disabled children as inpatients: a structured review and synthesis of qualitative studies reporting the views of children, parents and professionals
.
Child Care Health Dev
.
2012
;
38
(
6
):
778
788
22.
Smith
J
,
Cheater
F
,
Bekker
H
.
Parents’ experiences of living with a child with a long-term condition: a rapid structured review of the literature
.
Health Expect
.
2015
;
18
(
4
):
452
474
23.
Leary
JC
,
Krcmar
R
,
Yoon
GH
,
Freund
KM
,
LeClair
AM
.
Parent perspectives during hospital readmissions for children with medical complexity: a qualitative study
.
Hosp Pediatr
.
2020
;
10
(
3
):
222
229
24.
Rennick
JE
,
St-Sauveur
I
,
Knox
AM
,
Ruddy
M
.
Exploring the experiences of parent caregivers of children with chronic medical complexity during pediatric intensive care unit hospitalization: an interpretive descriptive study
.
BMC Pediatr
.
2019
;
19
(
1
):
272
25.
Hagvall
M
,
Ehnfors
M
,
Anderzén-Carlsson
A
.
Experiences of parenting a child with medical complexity in need of acute hospital care
.
J Child Health Care
.
2016
;
20
(
1
):
68
76
26.
Hirschfeld
RS
,
Barone
S
,
Johnson
E
,
Boss
RD
.
Pediatric chronic illness: gaps in inpatient intrateam communication
.
Pediatr Crit Care Med
.
2019
;
20
(
12
):
e546
e555
27.
Greenhalgh
T
,
Hurwitz
B
.
Narrative based medicine: why study narrative?
BMJ
.
1999
;
318
(
7175
):
48
50
28.
Locock
L
,
Robert
G
,
Boaz
A
, et al
.
Using a national archive of patient experience narratives to promote local patient-centered quality improvement: an ethnographic process evaluation of ‘accelerated’ experience-based co-design
.
J Health Serv Res Policy
.
2014
;
19
(
4
):
200
207
29.
Guney
S
,
Santoro
D
.
The place for patient narratives in healthcare
.
30.
Elliott
MN
,
Zaslavsky
AM
,
Goldstein
E
, et al
.
Effects of survey mode, patient mix, and nonresponse on CAHPS hospital survey scores
.
Health Serv Res
.
2009
;
44
(
2
,
pt 1
):
501
518
31.
Committee on Quality of Health Care in America, Institute of Medicine
.
Crossing the Quality Chasm: A New Health System for the 21st Century
.
Washington, DC
:
National Academy Press
;
2001
32.
Gleeson
H
,
Calderon
A
,
Swami
V
,
Deighton
J
,
Wolpert
M
,
Edbrooke-Childs
J
.
Systematic review of approaches to using patient experience data for quality improvement in healthcare settings
.
BMJ Open
.
2016
;
6
(
8
):
e011907

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.