Using Geomarkers and Sociodemographics to Inform Assessment of Caregiver Adversity and Resilience

We conducted a cohort study using data from a previous study of caregivers of hospitalized children recruited from the “Hospital-to-Home Outcomes” studies (H2O I and H2O II).^19,20  The H2O trials and this study were approved by our institutional review board. All caregivers provided written informed consent.

Details of H2O trial recruitment and design have been described previously.^19,20  Children were eligible for inclusion in either study if they were admitted to our institution’s general Hospital Medicine or the Hospital Medicine Complex Care Services; for H2O I, children hospitalized on the Neurology and Neurosurgery services were also eligible.^19,20  Patients were excluded if they were discharged to a residential facility, lived outside a home health care nurse service area, were eligible for skilled home health care services, or if the participating caregiver was non-English speaking.^19,20  In both H2O trials, face-to-face and paper-based questionnaires (H2O survey) inclusive of measures relevant to the present analyses were completed by caregivers during the index hospitalization. Here, we focused on a subset of caregivers, ≥18 years of age, whose children were enrolled in either arm of the H2O trials between August 2015 and October 2016 and had completed ACE, Brief Resilience Scale (BRS), and H2O survey, and had geocoded addresses.

Our primary outcomes were caregiver ACEs and resilience. The ACE questionnaire addresses abuse, neglect, and household dysfunction in the first 18 years of life.¹  It is 10 questions, each with a yes/no response.¹  We defined caregivers as low (ACE 0–3) or high (ACE ≥4) risk a priori as previous literature has described poor outcomes in adults with ≥4 ACEs.²¹  The ACE questionnaire was completed independently by the caregiver and returned to the research assistant in a sealed envelope.

Caregiver resilience was measured by using the BRS. The BRS is 6 items, each on a 5-point Likert scale. Responses were averaged, providing a total score of 1 to 5; higher scores are representative of higher resilience.²²  When used as a dichotomized outcome, we defined a BRS score <3 as low resilience and a BRS score ≥3 as high resilience.²³  BRS has been used in clinical settings; it has revealed a positive correlation with social support and a negative correlation with fatigue.²²  Caregivers answered BRS questions during the index pediatric hospitalization face-to-face H2O survey.

Child variables, obtainable from the electronic health record (EHR), included race, ethnicity, age, and payer. Race categories were White/Caucasian, Black/African American, American Indian or Alaskan Native, Asian or Pacific Islander, and other; ethnicity categories were Hispanic/Latino, non-Hispanic/Latino, and unknown. Given the low number of children reported to be Hispanic/Latino, we combined race and ethnicity into a single variable. We then categorized individuals as White, Black, and multiracial/Hispanic/other. Payer was categorized as private or public/self-pay.

Household variables, collected via the H2O survey administered by research assistants in a face-to-face survey, included caregiver race, education attainment, household income, employment, and financial and social strain. Caregivers reported their own race using the same categories as listed above for patient race. Educational attainment was analyzed in 2 categories: high school or less versus more than high school. Caregivers reported their annual household income in the following categories: <$15 000; $15 000–$29 999; $30 000–$44 999; $45 000–$59 999; $60 000–$89 999; $90 000–$119 999; ≥$120 000. Employment was dichotomized as not employed/student versus any employment. Financial and social strain were assessed by using a series of 9 previously described hardship questions.²⁴  These questions assessed, via respondent report, a family’s (or household’s) ability to make ends meet, pay rent/mortgage or utilities, need to move in with others because of financial reasons, and ability to borrow money if needed, as well as home ownership and caregiver marital status.^24,25  These strain questions were all dichotomous (yes/no, single/not single). For regression models, we used a composite strain variable, constructed a priori, by categorizing those reporting 0, 1–2, 3–4, and ≥5 strain items.²⁶ 

We assigned area-level measures to each child based on the geocoded address that was present within the EHR at the time of the hospital encounter. Residential addresses were geocoded to a latitude and longitude coordinate by using our custom geocoder based on census street-range files.^27,28  Once geocoded, we spatially joined patients (and their addresses) to the census tract in which they were located.

Geomarkers included area-based measures of socioeconomic deprivation (area deprivation index), healthy food access, and greenspace, each linked to a geocoded participant’s identified home census tract. Our team hypothesized that these geomarkers were representative of factors that may be associated with ACEs or resilience. The area deprivation index is a validated deprivation index,²⁹  which is a census tract-level composite of 6 variables related to material deprivation and socioeconomic status (eg, poverty, vacant housing, health insurance coverage). To construct the deprivation index, variables were extracted from the 2015 American Community Survey and used in a principal components analysis to generate a single index ranging from 0 to 1, with higher values being indicative of higher deprivation. Healthy food access was captured by using the modified food environment index (mRFEI), which is calculated as the percentage of total food retailers within a census tract that are considered “healthy.”³⁰  Lower mRFEI values indicate more convenience stores or fast-food restaurants compared with the number of healthy food retailers (eg, supermarkets). Greenspace was quantified by using the enhanced vegetation index (EVI), a satellite-based measure of greenness that ranges from -0.2 to 1, with higher values corresponding to more vegetation. A cloud-free composite raster at a resolution of 250 by 250 m was created by assembling individual images generated by moderate resolution imaging spectroradiometers aboard the National Aeronautics and Space Administration’s Earth Observation System satellites.

We compared sociodemographic characteristics of the low- versus high-ACE risk groups by using the Wilcoxon rank test for continuous measures and the Pearson’s χ² test for categorical variables. For ordered categorical variables, we used the proportional odds likelihood ratio test to account for trends among the ordered levels. The ACE and resilience outcomes were predicted by using a staged approach with new independent variables added at each level, termed models 1, 2, and 3 (Table 1). We used this graded approach to identify the minimum level of data that may inform the identification of caregiver ACEs or resilience to identify the most effective and efficient screening. Model 1 independent variables included only the sociodemographic information available in the EHR (child age, sex, insurance status, and race/ethnicity). In addition to Model 1 independent variables, Model 2 included geomarker information derived from a patient’s residential address (deprivation index, mRFEI, EVI). In addition to Models 1 and 2 independent variables, Model 3 also included questionnaire items collected as a part of the H2O study (caregiver educational attainment, household income, caregiver employment, strain). To help our comparison of model discriminatory performance, we fit a “Model 0,” or null model, for each outcome using no predictors except for an intercept term.

We used logistic regression modeling to reverse predict the dichotomous ACE and resilience variables. The predictive ability of each model for each outcome was characterized by creating receiver operator characteristic curves (ROCs) and calculating the area under the curve (AUC) with 95% confidence intervals (CI) based on 10-fold cross-validation repeated 10 times. The optimal threshold for classification was determined as that which simultaneously maximized sensitivity and specificity, and the resulting test characteristics were calculated by using this threshold. To test if predictive accuracy differed among the 3 models, we compared the estimated mean AUC values from each model using a 1-way analysis of variance (ANOVA). When the results of the ANOVA were significant (P <.05), we followed up with a Tukey-adjusted pairwise t test to determine which models had significantly different AUCs (Model 1 vs Model 2, Model 2 vs Model 3, and Model 1 vs Model 3).

As a sensitivity analysis, we pursued linear regression modeling using the ACE and resilience variables on a continuous scale. In addition, we explored the usage of classification and regression trees (CART) to discriminate both continuous and dichotomous ACE and resilience variables. The components of the deprivation index and the detailed H2O questionnaire items were included as predictors for these models because collinearity does not present a problem within the CART framework. Statistical computing was conducted in R (version 3.6.1).

A total of 1787 caregiver–child dyads were enrolled in H2O during the study period; 1320 (74%) completed the ACE questionnaire,⁵  and 1272 caregivers (71%) had information on all sociodemographic variables and completed all additional questionnaires (BRS, H2O survey).⁵  We were able to geocode and estimate geomarkers for all 1272 participants with complete data. Caregivers were primarily female and unemployed, with more than a high school education (Table 2), consistent with previous studies. A total of 499 (39%) of included caregivers had high ACEs, and 107 (8%) had low resilience; 63 (5%) caregivers had both high ACEs and low resilience. The high ACE group had differences in income, educational attainment, strain, and the deprivation index relative to the low ACE group.

For the outcome of high ACEs, Model 1 (sociodemographics only) had an AUC of 0.57 (95% CI 0.56–0.58) with low sensitivity (0.08) and high specificity (0.94) (Fig 1). Model 2 (sociodemographics plus geomarkers) had an AUC of 0.57 (95% CI 0.56–0.58). Sensitivity was low (0.12), and specificity was high (0.92). Model 3 (sociodemographics plus geomarkers plus H2O survey data) had a similar AUC (0.58; 95% CI 0.56–0.59), with a sensitivity of 0.22 and a specificity of 0.86. For each model, the optimal AUC corresponded to a sensitivity of 0 and a specificity of 1. All models performed better than the null; however, there was no significant difference in the AUC between Models 1, 2, and 3 (P = .08).

For the outcome of low resilience, results are demonstrated in Figs 1 and 2. Again, all models performed better than the null, but there was no significant difference between Models 1, 2, and 3 (P = .5). The ROC curves for each outcome are displayed in Fig 2.

Finally, we assessed our combined outcome of both high ACE and low resilience; results are demonstrated in Figs 1 and 2. There was a significant difference between the AUCs comparing Models 1, 2, and 3 (ANOVA P = .01). A Tukey-adjusted pairwise t test revealed that Model 1 was different from both Models 2 and 3, but that Models 2 and 3 were not significantly different from each other.

Finally, we pursued sensitivity analyses varying ACE and BRS cutoffs and treating both as continuous measures. This did not significantly change model results (data not shown). The purpose of the sensitivity analyses was to ensure that our ACE and BRS cutoffs were not affecting the models. We also employed CART as an alternative approach, but this did not reveal a change in the prediction accuracies of our outcomes of interest (data not shown).

There are many factors that contribute to a child’s risk of unanticipated health care reutilization, such as a caregiver’s history of ACEs⁵  or additional social needs. Our data reveal that caregivers with high ACEs were more likely to be publicly insured, live in communities with more socioeconomic deprivation indices, have lower incomes and educational achievement, and have a higher degree of strain, suggesting that ACEs and social needs may often coincide. Being able to identify and respond to such risks would be a valuable addition to care practices during and after a hospitalization. Yet, our models reveal that caregivers and child sociodemographic characteristics, geomarkers, and household variables did not meaningfully predict caregivers’ ACEs or resilience. Although the models we created using these variables performed better than the null models, the low AUCs indicate low clinical utility.

Our data suggest that the prediction of either high ACEs or low resilience is difficult, even with sociodemographic, geomarkers, and household factors. Although these results should be replicated to confirm the lack of predictive capacity, it may be more effective to pursue screening caregivers directly for this information. Although previous uses of demographic factors or geomarkers have been successful at identifying populations at risk for poor health outcomes or certain health-related social needs, they may have limited utility in predicting caregiver ACEs or resilience. ACEs are more common in minority or low-income populations, but they are not limited to those populations; they are prevalent across demographic and socioeconomic groups. In fact, the original ACE study by Felitti et al included a cohort of privately insured, White individuals.^1,3,31 

Previous literature also reveals that most individuals consider themselves to be resilient.⁶  Consistent with the previous literature, only 8% of caregivers in our cohort defined themselves as having low resilience. In addition, only 5% of caregivers in our sample reported themselves as having both high ACE and low resilience. We anticipate that efforts poised to identify caregivers within this high-ACE, low-resilience subgroup could be challenged by this limited prevalence. Another consideration is that these data were gathered via self-report, which may underestimate the true prevalence of caregiver adversity. To mitigate this bias, the ACE questionnaire was returned to caregivers in a sealed envelope to help yield more accurate responses.

To our knowledge, this is the first study to use a broad dataset, including readily available EHR data, and both geomarkers and household characteristics not routinely available to the medical team, to identify if there are EHR-based or linked factors that could predict the presence of caregiver ACEs and the degree of caregiver resilience. We did not find a meaningful combination of easily obtainable patient, household, or community data capable of reliably predicting caregivers with high ACEs, low resilience, or both. Using EHR and geomarkers in combination has previously revealed mixed predictive capacity in the assessment of a variety of health outcomes. A previous study from Bhavsar et al revealed that a combination of EHR-derived variables and geomarkers capturing area-level socioeconomic status did not improve the modeling of their health outcomes of interest, including hospitalizations.¹⁷  However, Fiscella et al demonstrated that adding geomarkers to the Framingham study improved the prediction of disease.³² 

Given our findings revealing the limited predictive ability of such data on ACEs and resilience, we suggest that one method to identify these caregivers may be to ask them about their history of past and current exposures, and how they cope with stressful events such as a child’s hospitalization. We recognize that caregivers may be reluctant to disclose this information, even with mechanisms to ensure anonymity. This may result in missed opportunities to identify these caregivers. Thus, providing universal education about the impact of ACEs on caregiver wellbeing and children’s health during hospitalization, paired with coping resources may benefit caregivers.

Our study should be considered in the context of several limitations. First, our included sample was limited to English-speaking caregivers already enrolled in a randomized control trial conducted at a single pediatric hospital. These factors limit the generalizability of our findings. For example, non-English speakers may be exposed to additional or different adversities (eg, language barriers) not recorded in our study. Secondly, we did not collect all geomarkers potentially associated with our outcomes of interest. Although we did have a wide array of sociodemographic and household-level data, there may be different factors that would have more effectively predicted our outcomes. Third, our outcomes of low resilience and the combination of high ACEs and low resilience were not as prevalent, which may make it difficult for our models to accurately reverse predict our outcome of interest.

Our study revealed that there is limited utility of patient-level EHR data, geomarkers, and household-level characteristics in the prediction of caregiver adversity in a hospitalized setting. Although future studies should consider the replicability of our findings, these results reveal that using existing data to predict such lived experience is likely insufficient, supporting integrating risk assessments and responses more directly into clinical care processes.