Family-Rated Pediatric Health Status Is Associated With Unplanned Health Services Use

Eligible pediatric patients (age <18) were residents of Alberta, Canada, discharged from inpatient services within the last 2 to 42 days. Patients with a <24-hour hospital stay, deceased patients, and patients presenting for psychiatric treatment (with a psychiatric unit or psychiatrist on their medical record) were excluded from sampling. The methods of sampling have previously been described.^28,29  This sample was drawn from 16 hospitals (2 pediatric) across the province, over the course of October 2015 to March 2019 (P.F., B.J.S., K.A.K., M.J.S., unpublished data, 2021). As previously described (P.F., B.J.S., K.A.K., M.J.S., unpublished data, 2021), the provincial health authority also administers the Child-HCAHPS to parents or guardians of newborn admissions (ie, children born during these hospital stays); these cases were removed by our team before analyses.

The Child-HCAHPS was designed to assess inpatient experience in children.¹⁸  Alberta administers a modified Child-HCAHPS (additional measures include questions specific to family and patient centered care).²⁸  For this project, variables drawn from Alberta Child-HCAHPS included the FRH (fair, poor, good, very good, excellent), number of hospitalizations within the last year (1, this time only; 2; 3 –5; and >5), respondent relationship to the child (mother, father, other [relative or guardian other than parent]), respondent age (<25, 25–34, 35–44, ≥45 years), and respondent education (less than high school or some high school; high school, some college, or 2-year degree; 4-year degree or more than 4-year college degree). Facility information was also provided by Alberta Health Services (AHS) Health Services ([acronym redacted]), and hospital type was coded as pediatric and general. AHS only administers the Child-HCAHPS by telephone. As previously reported, AHS reports a high response rate for the Child-HCAHPS in the province (67% for the 2019–2020 fiscal year).

The Discharge Abstract Database is a national health records system in Canada, comprising data from eligible hospital discharges, excluding Quebec.³⁰  Age at discharge (years, categorized according to case-mix categories: <1, 1–4, 5–8, 9–12, and 13–17), length of stay (LOS) (hospital: days), and number of comorbidities were extracted from the Discharge Abstract Database.

This study was approved by the University of Calgary Research Ethics Board (REB17-0769).

Initial cleaning was performed by AHS, who provided these data. Additional cleaning was performed by using SAS Network Version 9.3 for Windows (SAS Institute, Inc, Cary, NC) and R version 3.6.3.³¹  All analyses in this study were performed by using R. All cases with incomplete data for the variables of interest were removed from the analysis. Hospitals with <50 cases were removed from the analysis to ensure an adequate number of samples from each facility.

Descriptive statistics were reported for eligible surveys. All statistical tests were run at the significance level of 0.05.

Two-sided Spearman rank correlation was performed between FRH and number of previous hospitalizations, resource intensity weight (RIW; estimates of expected resource use³² ), LOS (days), and number of comorbidities present at discharge. Because exact P values cannot be given when there are ties, P values were approximate. We hypothesized that FRH may have some association with these administrative indicators.

To account for differences in respondent characteristics that might influence ratings, we ran a second set of correlations between the same administrative variables and an FRH score adjusted for covariates known to influence proxy reports of pediatric patient experience.^33,34  FRH were converted to the 100-point scale (excellent, 5 = 100, poor, 1 = 0). A linear model was constructed with the 100-point health rating as the dependent variable and the respondent age, relationship to child, and respondent level of education entered as numeric predictors.

Ordinal regression was planned, with FRH as the dependent variable. However, after establishing that the proportional odds assumption was violated, the authors chose to dichotomize FRH into positive health (excellent, very good, and good) and negative (or “less than good”) health (fair, poor).³⁵  Positive health served as the reference category. Two models were created: 1 with patient factors and 1 with both child (patient) and respondent factors. Comparisons were made between models by using the likelihood-ratio test. Given the relevance of respondent-level factors in proxy-reported health status, it was expected that the model with both child and respondent factors would have the best fit.

Factors associated with unplanned health services use vary across clinical settings and populations, but patient demographics and clinical characteristics are both associated with readmissions.^36–38  For this study, we used the set available from the linked experience survey and discharge abstract for 9 case features: hospital stays within the last year, LOS (days), admission category (urgent or elective), RIW, number of comorbidities, age at discharge (entered as a continuous variable), child sex (male or female), facility type (general or pediatric), and FRH. Outcomes were emergency department visits and unplanned readmissions within 90 days of discharge. Cases in which <90 days had elapsed since discharge were removed from these analyses. We hypothesized that if FRH provides comparable information to SRH, it may aid in the selection of models for unplanned health services use.

We chose to employ the R package “glmulti” to construct the logistic regression model (an R package is an extension to the R software that adds additional functions; they are open source and documentation is available online).³⁹  Unlike stepwise regression, the glmulti compares all possible models on the basis of an information criterion, which is a metric for evaluating the fit of a model. Models in our analyses were selected on the basis of lowest possible Akaike information criterion (AIC; lower numbers indicate better performance). For more details on glmulti, refer to the online documentation.^39,40  We hypothesized that FRH would be included in the models and would be significantly associated with unplanned health system use. However, we cannot make strong inferences about how FRH predicts health services use. Our models are built with a constrained set of features for a general sample of patients (eg, not investigating single conditions and not including variables that may be more relevant for specific conditions), and this analysis did not use more robust causal modeling. Odds ratios for the selected models and concordance statistics (C-statistics) were reported for the models.

The sample had 7951 cases. After removing newborn admissions from the sample (n = 1362 cases), cases from hospitals with <50 surveys fielded (n = 129), and incomplete cases (n = 224), we were left with a sample of 6236 linked cases from 13 facilities (2 pediatric). Descriptive statistics are reported in Table 1.

Results of the Spearman correlations (unadjusted and adjusted) are presented in Table 2. Approximating the results to determine estimated significance using the cor.test function in R, we found that FRH had small but significant correlations with previous hospital stays (ρ = 0.37), RIW (ρ = 0.09), LOS (ρ = 0.14), and number of comorbidities (ρ = 0.20). The magnitude of these relationships changed slightly after adjusting for respondent age, education, and relationship to the child.

The 2 models were compared for fit. The model with child and respondent variables had a lower AIC (4436.6) than the model with only the child variables (4550.4). As expected, the model with both child and respondent factors fit better than the model with only child factors according to the likelihood-ratio test (P < .001). The results of the multivariate analysis are presented in Table 3.

If a child had visited the hospital multiple times within the last year, they were much more likely to be rated with negative FRH. Compared with children with only a single admission to hospital, the odds ratio was largest for children with >5 admissions (adjusted odds ratio [aOR] = 10.99, 95% confidence interval [CI]: 8.72–13.86). In a bivariate model, LOS in days from the most recent hospital admission had a significant but negligible increased odd ratio. This did not remain significant in the multivariate model. The number of comorbidities at discharge was significantly associated with negative FRH (aOR = 1.12, 95% CI: 1.09–1.15).

Compared with the reference category of patients age of <1, all of the older age categories at discharge was significantly associated with increased odds of negative FRH for all age categories. Respondent age and level of education was not significant in the multivariate model.

Compared with mothers, fathers had lower odds of providing negative FRH (aOR = 0.69, 95% CI: 0.53–0.89), and other respondents (eg, grandparents, aunts and uncles, other guardians) had higher odds of providing negative FRH (aOR = 1.96, 95% CI: 1.36–2.80).

The models selected by glmulti for both emergency department visits and readmissions included FRH in the list of selected variables. Models with selected variables and the adjusted odds ratios for FRH are reported in Table 4. The first section (best-fit formulae) includes the list of variables included in the best-fitting models as selected by the glmulti package on the basis of AIC. With “excellent” as the refence category, children with “poor” ratings had increased odds of emergency department visits (aOR: 2.15, 95% CI: 1.62–2.85) and readmissions (aOR: 2.48, 95% CI: 1.74–3.53). The model C-statistics were 0.641 and 0.824 for emergency department visits and readmissions, respectively. A summary of adjusted odds ratios for both models are reported in Supplemental Table 6.

Self-reported patient outcomes may be rare in routine pediatric clinical care, but family-reported outcomes and experiences are widely used and increasingly becoming adopted within inpatient care.^19,41  In our study, we provided evidence to suggest that family ratings of health status have some comparability to self-rated health and can potentially serve as a useful family-reported outcome measure of child health. FRH is associated with administrative indicators, impacted by both respondent and patient factors, and associated with unplanned health services use within 90 days of discharge. To identify patients who may need additional care and to possibly prevent unplanned health services use, our results suggest that providers should ask parents or guardians about a child’s overall general health.

In their recent report on factors associated with an overall experience measure, Feng et al used a sample of >17 000 surveys collected across 34 states.⁴¹  The increasing adoption of the Child-HCAHPS provides new opportunities to study pediatric patient experiences stays within the contexts of specific clinical populations. Previous research has suggested that proxy ratings of pediatric health status are useful clinical indicators that can predict health care use.²⁴  However, FRH from the Child-HCAHPS has not been studied for linkage with health records on a population level. In this article, we perform a descriptive analysis of how this data source may provide an easily accessible proxy for self-rated health. In the Adult HCAHPS, proxy responses are never allowed. Reporting by proxy introduces a confound in that we are not directly measuring patient experience, but by capturing additional data about respondent and patient identity, models may be able to better use FRH as a low-burden measure of health status in pediatric inpatient care. There is a need to incorporate patient and family perspectives while maintaining efficient care.⁴²  Simple proxy-reported outcomes such as FRH provide opportunities for low-burden integration to gain useful data for managing pediatric care at both the clinical and systems level.

Proxy ratings of health status by parents and guardians were weakly associated with administrative indicators of health. Of note, the largest associations were with the number of comorbidities present at discharge and previous health services use. This is consistent with previous research in the SRH literature.^6,43  We found that a crude adjustment of FRH on the basis of respondent characteristics slightly reduced the strength of the associations. SRH literature suggests that the construct is heavily influenced by social and cultural factors,⁴⁴  and we might infer that family ratings of health are similarly influenced. Although the associations with specific administrative indicators may not be strong, this does not undermine the potential utility of family reports as a meaningful indicator of health status. Additional clinical data (such as condition-specific variables, comorbidity indexes⁴⁵ ) and more detailed respondent characteristics (such as socioeconomic status, ethnicity, and respondent health status) could provide additional insight on the relationship between ratings of health status and other administrative indicators of health.

In our model assessing the relationship between patient and respondent characteristics and reported FRH, 2 clinical health factors were associated with increased odds of having negative FRH. The association between multiple hospital stays and negative FRH is consistent with some of the literature from health care use. One of the factors most strongly associated with hospital admission is previous admission, including in pediatrics.^36,46  Increased age (compared with age <1 year) was also associated with increased likelihood of reporting negative health status. We would posit that this may be because the child has longer to develop, display symptoms of, or receive treatment of health issues. Furthermore, the significant impact of the number of comorbidities present at discharge suggests that increasing health burden may contribute to perceptions of negative health status. LOS from the most recent hospitalization was not associated with a significant coefficient in the multivariate model. Even as the respondent answered the survey on the basis of the child’s most recent hospitalization, this result suggests that these assessments of health do not rely on LOS to inform perceptions of health status. We might expect different results if data were collected during the inpatient stay,⁴⁷  but FRH may also be a more stable indicator of health, comparable to self-rated health.^11,17,48  Data collection during hospital stays and longitudinal study with repeated measures are needed to investigate this further. These results provide some evidence that proxy-rated health from the Child-HCAHPS may be a valid marker of health status.

Respondent characteristics are known to impact ratings on the Child-HCAHPS and are used to adjust survey scores when comparing between hospitals (P.F., B.J.S., K.A.K., M.J.S., unpublished data, 2021).^33,34  In this article, we found that the respondent’s relationship to the child was the only significant respondent factor in the multivariate logistic regression model for the likelihood of reporting negative health status. Education level and age were not significant. We have previously reported that fathers were significantly more likely than mothers to report most-positive ratings of pediatric inpatient experience on many domains of care on this same survey (P.F., B.J.S., K.A.K., M.J.S., unpublished data, 2021). We saw the same difference in positivity here, with fathers significantly more likely to report positive health status. Interestingly, respondents within the “other” category (family or guardians outside of parents) were more likely to report that the child’s health status was negative. These differences between respondents may be subject to confounding introduced by patient mix distribution and a lack of additional, relevant social and demographic information. This reinforces the need for further investigation with more detailed respondent-level variables to understand the social, economic, and cultural factors impacting different family members’ assessments of their child’s health.

It is difficult to interpret the accuracy of family-reported health status compared with what might be expected from a self-rated health by the patient. Proxy responses on CAHPS surveys have been demonstrably more negative than patients’ on average,²¹  but parents may rate children’s health more positively than children do.²⁷  In a systematic review on the agreement between parent and child ratings of health-related quality of life, the authors note that for most studies, absolute differences in ratings may be small and agreement is typically good.⁴⁹  They also note that these ratings (and whether parental proxies report more positively or negatively than the child) depend on the specific domains being assessed, the child’s health conditions, and respondent identity features. Beyond the characteristics measured in the current study, there is evidence that parent health status may play a role in the perception of pediatric health status.^27,49,50  Parental proxies, although imperfect measures of individual experience, can provide utility in the measurement of health status. Further research on the concordance between parent and children on single-item self-rated health is needed in pediatrics.

The automated model selection provided evidence that FRH is associated with unplanned health services use. We found that almost every increasingly negative level of FRH (compared with excellent) had increased odds of unplanned health services use within 90 days. These results provide evidence to suggest that FRH could provide accessible and useful information in models predicting health services use, similar to SRH.^9,51  However, a lack of sociodemographic data in our sample prevents stronger comparison with SRH.^52,53  As previously noted, these models are not causal, but the associations we observed in these models suggest that the predictive modeling could be improved with the inclusion of FRH.

Despite the general nature of these analyses, with this article we provide direction for more focused study. The utility of FRH in specific patient populations should be studied further, where condition-specific risk factors can be included in the models. Condition-specific patient-reported experience (PREMs) and patient-reported outcome measures (PROMs) are already incorporated in predictive modeling,^54–56  but the adoption of generalized parent and family reports could provide opportunities for systems-level study and comparisons. With the increased availability of linkable Child-HCAHPS data in hospitals and health systems, researchers and clinicians working on health services research could consider the inclusion of FRH to potentially improve predictive modeling.

The results of these analyses suggest that by asking parents or guardians about the child’s overall health, health care providers working with children may be able to identify patients with increased care needs or who might require additional planning to prevent unplanned health services use. There are some limitations to this research. Although this research provides evidence that the FRH collected from the Child-HCAHPS has some utility, it should be done with caution. AHS Child-HCAHPS respondents complete the survey between 2 and 42 days postdischarge. Respondent perceptions may be influenced by the passage of time after discharge (which we were unable to account for in our models), the current status of the child’s health (rather than a global assessment of the child’s health), and other, unmeasured respondent level factors such as race, income, health status, and experience with the health care system. Insurance status (both coverage type and provider) may also play a role in the likelihood of a child being rated as having positive or negative health.^57,58  Furthermore, economic status has been implicated in proxy reports of the health status but was unavailable for these analyses.¹⁶  The proxy-rated health status of pediatric inpatients may have different utility for different classes of patients (eg, for medical complexity, specific diagnoses, ages of children) and should be examined. Those interested in using proxy-rated health status should do so in combination with additional respondent, sociodemographic, and clinical factors.

In this article, we report on FRH as a proxy for self-rated health in pediatric inpatient settings. We found that FRH is associated with administrative indicators of health status, impacted by both respondent and child features, and associated with unplanned health services use within 90 days of discharge. We provide evidence to suggest that FRH provides useful information in predicting health services use. As health systems increasingly adopt patient experience measures, there are increasingly opportunities to use survey instruments in related research and practice. We stress the importance of the information that parents and guardians can provide about a child’s health. Physicians and health care providers should ask family members about a child’s overall health to identify patients who may need more attention or follow-up.

We thank Alberta Health Services Primary Data Support for collecting and providing this information.