A Social Care System Implemented in Pediatric Primary Care: A Cluster RCT

We conducted a type 1 hybrid effectiveness-implementation cluster RCT in CHCs located within a 3-mile radius in Boston, Massachusetts from September 2015 to March 2020. CHCs were randomized 1:1 to either the augmented WE CARE or control group. Group assignment was concealed until a practice agreed to participate in the study. Each CHC provided care to ∼90 to 250 newborns annually (mean = 180).

All parent-child dyads who presented for a newborn visit to participating CHCs between September 2015 and March 2017 were eligible. Since our study population included healthy low-income children insured by Medicaid at baseline, families were excluded if the newborn had a history of prematurity (ie, born before 32 weeks), had commercial insurance, or was born with a chronic disease (eg, cardiac disease) or with Neonatal Abstinence Syndrome. Newborns living with foster parents were also excluded.

The augmented WE CARE system was implemented in the experimental arm CHCs as standard of care at well-child visits from birth to age 3. Research staff trained the clinical staff during a 1-hour session and with five 30-minute booster sessions over the study period. The augmented intervention consisted of 3 core components: (1) WE CARE screening instrument; (2) community resource information sheets made available from the electronic health records (EHR); and (3) a peer patient navigator.

CHC staff were trained to provide each family who came in for a well-child visit until at least age 3 with a WE CARE screening instrument at check-in. Overall, the instrument consisted of 14 questions designed to identify 7 social risks and determine families’ desire for assistance with: child care, education (parent not completing high school), employment, food security, household heat, housing, and language (eg, need for ESL classes) (Supplemental Appendix). The screener was written at third-grade level, made available in English, Spanish, and Vietnamese, and took <5 minutes to complete. Of note, the original screening instrument that this tool was adapted from has a test-retest reliability of 0.92.⁸  The WE CARE screener was consistent with the prior tool except that language need was added based on clinicians’ and CHC staffs’ recommendations.

Pediatric health care team members were able to access resource sheets initially on an external website embedded in Centricity EHR. These sheets listed a maximum of 4 community resources and included program name, contact information, program hours, and eligibility criteria for each specific need. A telephone number was listed at the bottom of each sheet that families could call to contact a patient navigator for assistance. The resource sheets were updated every 3 months.

Approximately 9 to 18 months after the study began, all the CHCs switched to Epic© EHR. Smart phrases were created in Epic© to disseminate resource information. Health care team members were trained to type smart phrases into the After Visit Summary, which auto populated community resource information, which they then provided to parents with needs. The resource information contained in the original resource information sheets and the After Visit Summary were written at, or below, fifth-grade reading level and available in English or Spanish.

A peer patient navigator was made available to meet with families, inquire about needs, and assist families with completing applications. The navigator was on site at least 1 day per week per site and received referrals from staff and clinicians via warm handoffs or messaging within the EHR. She contacted families to support and guide them in receiving community-based resources and entered notes into the EHR regarding families’ connecting with and accessing resources.

Of note, the funding mechanism (RFA-MD-12-003) called for integration of new technology and peer navigation; thus, the augmented WE CARE system differed from the original tested WE CARE system⁹  by offering resource sheets from the EHR (versus a Family Resource Book) and by including a peer patient navigator (versus research staff).

Participants at control CHCs received usual care. In order to compare referrals as well as for ethical reasons, the same resource information sheet referral system was embedded into control CHCs’ EHR (first in Centricity and then in Epic©). The pediatric team was notified of this by the research team at the study’s start and during the EHR transition.

Data were extracted post hoc through EHR review for visits during the study period (September 2015 to March 2020). Research staff reviewed each eligible participant’s chart for all visits from the newborn visit to the 3-year-old well-child visit. Staff collected sociodemographic data and data on documentation of resource information sheet printouts (consistent with our prior study, we use the term “referral” moving forward), child maltreatment, child protective services reports (CPS), and health care utilization outcomes. In the experimental arm, data on WE CARE screeners scanned into the EHR were recorded for each visit. Families referred to the patient navigator were logged into an Excel spreadsheet and crosschecked with the child’s EHR. For 10% of the charts, we had another research staff review the data; overall, there was strong concordance with an error rate of 0.12%. All discrepancies were resolved with the first author (A.G.). Data were censored at control sites if they began implementing a social needs and risk screener or at WE CARE sites if they began using a different screening instrument.

The primary study outcomes were measures of health care utilization, including emergency department (ED) visits, hospitalizations, and adherence to the AAP’s well-child visit and immunizations schedules. Secondary outcomes included referrals to resources and/or the patient navigator. We also examined EHR-documented reports to CPS and of children with suspected maltreatment. To evaluate experimental sites’ implementation of WE CARE, we calculated the percentage of visits with a WE CARE screener scanned into the EHR, as well as percentage of appropriate referrals (defined as information sheet printed out for parent indicating they want help on the screener).

Before conducting the study, we estimated a total of 8 CHCs would participate. We modeled our participation rate based on prior pediatric CHC data. Assuming an intraclass correlation of 0.02, we expected to have over 80% power to detect a 5% difference in our health care utilization, maltreatment, and referral outcomes.

Descriptive statistics were conducted on sociodemographic characteristics, screener administration, referrals, and outcome variables cumulatively over the 3-year time period as well as per year. Consistent with Abdus’ and Selden’s methodology,^10,11  we created well-child visit and immunization adherence ratios by dividing the number of visits attended and immunizations received by the overall numbers of well-child visits and immunizations recommended by the AAP for ages 0 to 1 year, >1 to 2 years, and >2 to 3 years of age.

Since 1 control site began implementing a similar social care system within the first year of the study, we considered them contaminated and excluded them from our primary analyses. For cumulative effects that were statistically significant, we also used an intention-to-treat approach and included the contaminated control site to compare augmented WE CARE and control groups. Generalized mixed-effects models were used to estimate the differences in sociodemographics between groups as well as test direct intervention effects on outcomes adjusting for time, CHC intraclass correlations, and sociodemographics. A random intercept for each site was included in the model to account for within-site correlation.

All statistical analyses were performed using the SAS software version 9.4 (SAS Institute, Inc, Cary, NC). Adjusted odds ratios (aORs), 95% confidence intervals (CIs), and P values were calculated for each model. Statistical significance was defined as P < .05.

The Boston University Medical Center Institutional Review Board approved this study. The study was registered on clinicaltrials.gov (NCT02451059) before its onset.

Eight CHCs in the Boston HealthNet network were initially approached to participate in the study; 2 CHCs declined because of lack of interest and anticipated burden, resulting in a total of 6 participating sites (n = 1193 patients). Although the initial data pull of potentially eligible records included 1485 newborns, 292 were excluded upon further review because of insurance status, chronic disease status, and duplication of records. Our final cohort included 3 sites in the WE CARE arm (n = 524 patients) and 3 in the control arm (n = 669 patients). Within a few months of implementation, as noted above, a control site (n = 315 patients) began their own social care screening and referral system and was therefore contaminated (Fig 1).

At baseline each CHC already had social history probing questions (eg, child care, housing, school) embedded in their well-child visit templates. Also, all CHCs had social workers, case managers, and/or volunteers to assist families with basic needs.

Among our cohort of children, a slight majority were female, almost half were non-Hispanic Black, and over a quarter were Latino/a/e. The most common household language was English (67.0%), followed by Spanish (12.6%), and Vietnamese (8.2%) (Table 1).

In the experimental arm, 28.9% of visits had a WE CARE screener scanned into the EHR. WE CARE screening ranged between 21.0% and 43.8% across the 3 sites. Over a third of parents who indicated wanting help with a need received an appropriate referral (38.7%, range 27.7% to 49.3%).

WE CARE families received significantly more resource referrals than control families at well-child visits (43.1% vs 1.9%; aOR 4.6; 95% CI, 2.0–5.6). The most common referral was for child care (25.8% of total referrals), followed by utilities (14.4%), and employment (14.1%). Over 20% of WE CARE families were referred to the patient navigator; the navigator connected with 73.4% of families via telephone, text message, or email.

Although WE CARE families had higher well-child visit adherence ratios, this did not achieve statistical significance. WE CARE children had significantly higher immunization ratios across all 3 years (Table 2). WE CARE children also had significantly higher ED visit rates than control children for all ages (Table 3). Although there were no significant differences in hospitalizations in the first 2 years, WE CARE children had significantly higher cumulative hospitalization rates by age 3 (14.1% vs 10.4%, aOR 1.3, 95% CI: 1.03–1.7). Results were similar in the intention to treat models as well.

There were no statistically significant differences in reported child maltreatment cases or CPS reports between the groups in the first 3 years (Table 4).

We found that a social care screening and referral system implemented in pediatric primary care was not only challenging to implement and maintain, as evidenced by infrequent screening and appropriately made referrals, but also had varied impact on immunization, health care utilization, and maltreatment outcomes. Although WE CARE children had higher immunization adherence ratios (and trends for higher well-child visit adherence), paradoxically they also had higher ED visits and hospitalization rates by age 3. To our knowledge, our study is the first clinical trial assessing the implementation and effectiveness of social care systems implemented in real-world primary care conditions, and thus our results carry important clinical and policy implications.

This 5-year study was a pragmatic trial conducted during a backdrop of unanticipated seismic changes in social care policies and recommendations. Begun 1 year before the AAP’s policy statement on Child Poverty recommending pediatricians screen for social factors and ending days before the coronavirus disease 2019 pandemic started, multiple social care initiatives including the Accountable Health Communities Model,³  Medicaid policies,^2,12  and professional recommendations, including from the National Academy of Sciences, Engineering, and Medicine,⁴  were enacted that accelerated the practice of screening and referring for unmet basic needs into pediatrics as standard of care (including in 1 of our control CHCs early in 2016 and many of our CHCs by the end of the study). Yet, despite this rush to implementation, there is limited evidence about the impact that these models have on children’s health and health care, along with its unintended consequences.⁷  Our results, which will be harder to replicate in pediatric practices given the current social care landscape, should give policy makers, health system leaders, and clinicians pause in their rush to implement and maintain screening for social drivers.

With our focus on the effectiveness component of this type 1 hybrid effectiveness-implementation study,¹³  we purposefully did not provide feedback regarding implementation metrics to practices in real-time, which in hindsight was a critical mistake. Compared to our efficacy study,⁹  our referral rate of 43% over a course of multiple visits for children in the first 3 years of life was much lower than our rate of 70% found at 1 index well-child visit, despite both studies being conducted in the same and/or similar CHCs. Qualitative work with staff and clinicians found strong mission alignment with the augmented WE CARE system, belief that it benefited their patients, and was overall easy to implement.¹⁴  However, the low rates of documented screening and appropriate referrals indicate poor implementation and highlight the importance of identifying and addressing organizational and administrative barriers before rolling out social care systems. Having clinical champions, piloting workflows, getting buy-in from all involved staff, sharing success stories, and monitoring and sharing back implementation data using quality improvement techniques and evolving workflows are likely critical in fully integrating social care into pediatric primary care.¹⁵ 

Beyond poor implementation, there are likely other factors that led to our unexpected results. Because of funding, we only had 1 patient navigator serving 3 clinics, which limited her on-site presence. We previously reported that clinicians were more likely to refer families if the navigator was on site.¹⁶  Although we found trends with well-child visit adherence and directionality for lower maltreatment outcomes for WE CARE children, none of these results achieved statistical significance. Prior investigators testing the SEEK model, which includes addressing poverty-related needs, found it led to lower rates of maltreatment under controlled conditions.¹⁷  Our findings may have also resulted from a loss of power because of contamination.

We are perplexed with our ED and hospitalization findings. We had expected WE CARE children to have better health outcomes based on the OASIS model, which suggests that the augmented WE CARE system can improve health care outcomes by reducing competing demands and stress for parents.¹⁸  In addition, clinical efficacy trials from Gottlieb et al have found improved parental report of child health and a reduction in hospitalizations from their similar social care intervention.^19–21  So why did we find the opposite? One possible explanation is that there was unmeasured confounding related to CHC-specific risk factors that were related to going to the hospital. Another possibility and/or contributing factor could be because of the negative ramifications of poor implementation, particularly related to referrals. Less than 40% of appropriate referrals were made, suggesting that the majority of time when a family indicated they wanted help with a basic need like food, there was no action taken by the health care team. This concept of double loss (ie, disclosing needs without having them addressed) can erode parents’ trust and lead to frustration²² ; perhaps it led some families to go to the ED instead of their child’s clinic when their child was ill. Alternatively, these results may be attributable to the intervention helping foster trust with the health care system; families who received WE CARE may have been more likely to go to the ED when they were sick or accept admission to the hospital. Further qualitative studies with parents and caregivers are needed.

There were limitations to the study. The study may have limited generalizability since it was conducted at CHCs. Study sites had robust support staff (eg, social workers, case managers) which may have contributed to our mixed results. Future studies should consider evaluating social care models in practices without such existing support. As previously noted, over the 5-year study period there were many pragmatic challenges, including EHR platform changes, staff turnover, and contamination. Finally, we relied primarily on the EHR documentation for our data, which may have undercounted our implementation screening outcome and conversations around social needs that might have occurred between clinicians and parents.

Overall, we found it was challenging to implement a social care screening and referral system with patient navigation in pediatric practice. Although the augmented WE CARE system led to increased referrals for families’ needs and higher immunization rates, it also led to more ED visits and hospitalizations. Further work is needed to build upon our study. Until then, caution and likely tempered expectations are necessary when developing and implementing social care policies and clinical interventions to identify and address unmet needs.

Our study found poor implementation and mixed benefits for a screening and referral social care system integrated into pediatric primary care on immunization, health care utilization, and maltreatment outcomes in early childhood.

We thank Michelle Pellicer, MPH and Marie Joseph, BA for their efforts and contributions to the study; and the patients and staff at Codman Square Health Center, DotHouse Health, Greater Roslindale Medical and Dental Center, Mattapan Community Health Center, South End Community Health Center, and Upham’s Corner Health Center for their participation.