In this interventional study, we addressed the selection and application of clinical interventions on pediatric patients identified as at risk by a predictive model for readmissions.
A predictive model for readmissions was implemented, and a team of providers expanded corresponding clinical interventions for at-risk patients at a freestanding children’s hospital. Interventions encompassed social determinants of health, outpatient care, medication reconciliation, inpatient and discharge planning, and postdischarge calls and/or follow-up. Statistical process control charts were used to compare readmission rates for the 3-year period preceding adoption of the model and clinical interventions with those for the 2-year period after adoption of the model and clinical interventions. Potential financial savings were estimated by using national estimates of the cost of pediatric inpatient readmissions.
The 30-day all-cause readmission rates during the periods before and after predictive modeling (and corresponding 95% confidence intervals [CI]) were 12.5% (95% CI: 12.2%–12.8%) and 11.1% (95% CI: 10.8%–11.5%), respectively. More modest but similar improvements were observed for 7-day readmissions. Statistical process control charts indicated nonrandom reductions in readmissions after predictive model adoption. The national estimate of the cost of pediatric readmissions indicates an associated health care savings due to reduced 30-day readmission during the 2-year predictive modeling period at $2 673 264 (95% CI: $2 612 431–$2 735 364).
A combination of predictive modeling and targeted clinical interventions to improve the management of pediatric patients at high risk for readmission was successful in reducing the rate of readmission and reducing overall health care costs. The continued prioritization of patients with potentially modifiable outcomes is key to improving patient outcomes.
In numerous studies, authors have investigated risk factors for hospital readmission,1–4 characteristics of potentially preventable readmissions,5–7 variability in readmission rates,8,9 and corresponding predictive models,10–14 mostly in adult patient populations. Investigations of these issues in pediatrics are lacking. Furthermore, the limited number of studies published to date have not extensively addressed the ability of predictive modeling and corresponding clinical interventions to decrease the risk of readmission. The predictive models for hospital readmission in previous studies depend on unmodifiable risk factors, such as previous health care resource use variables, which complicates the development of corresponding intervention protocols.10,14–16 For example, most studies have indicated that length of stay, emergent admission, previous emergency department visits, previous hospitalizations, previous readmissions, complex chronic medical conditions, and social determinants of health (SDoH) are among the strongest predictors of readmission.10,14–16 A common property of these risk factors is that they are unmodifiable, making well-defined prebuilt intervention protocols difficult to implement successfully. But the development of a comprehensive and powerful predictive model for readmission should include both these unmodifiable risk factors and modifiable variables, such as childhood malnutrition.1 Thorough investigations of unmodifiable risk factors in relation to previous health care use may reveal underlying modifiable causal factors.1 In addition to these considerations, the success of intervention protocols lies in targeting not only patients most at risk for readmission but also patients who are likely to have modifiable outcomes.5,6,17–19
In previous studies, researchers have attempted interventions aimed at reducing readmissions with varying levels of success or failure.20–23 In this study, a predictive model for readmission10 was adopted and implemented in the electronic health record (EHR) system of a tertiary pediatric hospital, and corresponding intervention protocols were developed and applied on the basis of evidence in the literature. The purpose of this interventional study is to describe changes in readmissions over a 2-year period after implementation and adoption of a predictive model of readmission coupled with new and existing intervention protocols. Intervention efforts had not yielded significant changes in readmission rates in the years leading to the use of predictive models to identify and target the most-at-risk patients.
Methods
This study was conducted at the main hospital of a large tertiary pediatric health system in Southern California. The hospital is a state-of-the-art 334-bed facility that includes neonatal, pediatric, and cardiovascular ICUs. An emergency department adjoins the hospital building. Collectively, these facilities serve as the hub of a pediatric health system that includes many primary and specialty care clinics across the community. The institution houses 4 centers of excellence addressing cardiovascular, neurologic, orthopedic, and oncologic care.
The predictive model for pediatric readmissions used in this study was published by Ehwerhemuepha et al10 in 2018. The model consisted of several classes of variables, such as demographics, proxies for social determinants of health, current and previous health care resource use variables, proxies for severity of illness, and patients’ diagnoses.10 The model was modified to consider other risk factors found to be associated with readmissions, such as certain medications and history of previous readmission.10,14,24
A multidisciplinary team consisting of physicians, nurses, care coordinators, social service specialists, data scientists and/or statisticians, and hospital administrators was created. The team reviewed the model, and a decision was made to implement it in the EHR, with hourly updates of predictions from admission to discharge. The EHR system of the hospital is the Cerner Corporation EHR. Implementation in the Cerner EHR may be broken into several stages. Clinical data required by the model were retrieved from Cerner into an on-premises enterprise data warehouse (EDW) that was initially developed for daily business intelligence applications. An R Statistical Programming Language process connects to the EDW and queries relevant Structured Query Language tables for new data at the 50th minute of the hour every hour. The timing is to ensure that there are no clashes between EDW database updates and the time the readmission algorithm runs. The readmission algorithm runs and makes predictions, using the data retrieved, on each patient currently admitted. The results of the prediction are written as a flat file unto a directory from which Infor Cloverleaf Interface Engine reads. Cloverleaf uses the results read to generate a Health Level Seven message. This Health Level Seven message is used within Cerner Millennium to write the results into a custom Millennium table. Refer to Supplemental Fig 3 for a data flow diagram. The predictions are then displayed in a custom built MPage summary (which is different from the Cerner standard patient summary) for each patient (Fig 1). A real-time report was also built within the EHR to provide data on all patients within a defined period.
Screenshots of the EHR readmission risk interface revealing randomly selected patients with high, moderate, and low risk of readmission as seen by providers logging into the chart of the patients. A, High readmission risk. B, Moderate readmission risk. C, Low readmission risk. Dx, diagnosis; LOS, length of stay.
Screenshots of the EHR readmission risk interface revealing randomly selected patients with high, moderate, and low risk of readmission as seen by providers logging into the chart of the patients. A, High readmission risk. B, Moderate readmission risk. C, Low readmission risk. Dx, diagnosis; LOS, length of stay.
The clinical team was provided appropriate training on interpreting the model and its real-time output, with a consensus on implementation strategies. Readmission risk strata consisting of high, moderate, and low risk were adopted with the predicted probability thresholds proposed by Ehwerhemuepha et al.10 To improve the model’s ease of use, a color-coded icon signifying risk for readmission was developed and approved for use on the patient summary page (MPage) of the EHR (Fig 1). Use of this “R” icon ensured that individualized readmission risk was visible to the entire clinical team in real time. Additionally, an EHR list of all actively hospitalized patients and their risk strata and risk factors was created for members of the hospital’s readmission team and other providers working to reduce readmissions.
Intervention Protocols
The clinical intervention team adopted several standard interventions aimed at mitigating readmission risk. These attempts were a combination of predictive modeling and targeted clinical interventions to improve the management of patients at higher risk of readmission. Some interventions (listed in Supplemental Table 4) were applied to all admitted patients regardless of probability of readmission. Table 1 reveals the targeted interventions for moderate- or high-risk patients. The specific interventions used and the intensity with which these measures were implemented were determined by risk strata and patient-specific considerations, such as the psychosocial and clinical needs of the patient and family members. Providers reviewed the EHR charts of patients with moderate or high risk of readmission to personalize the delivery of interventions.
Model-Driven Interventions
Time . | Interventions . | Low Risk . | Moderate Risk . | High Risk . |
---|---|---|---|---|
Discharge planning on admission | Dedicated DNN20,a,b: RN who focuses on health literacy and education, including assessments and teach-back, before discharge | — | +++ | +++ |
24-h postdischarge handoff(s) | PCCc or insurance care manager | +++ | — | — |
Outpatient specialty CMd | — | +++ | +++ | |
Outpatient specialty SWe | — | +++ | +++ | |
5–10 d post discharge | Telehealth appointment with CM and/or SWf: verifies proper use of DME, medications, and follow-up appointment, including authorizations | — | +++ | +++ |
12–30 d postdischarge point of contact | PCC or CMAg: staff returns phone calls and follows-up on outstanding issues | + | ++ | +++ |
CM or SWh: licensed professional who returns phone calls and follows-up on outstanding issues, including alignment with community resources | + | ++ | +++ |
Time . | Interventions . | Low Risk . | Moderate Risk . | High Risk . |
---|---|---|---|---|
Discharge planning on admission | Dedicated DNN20,a,b: RN who focuses on health literacy and education, including assessments and teach-back, before discharge | — | +++ | +++ |
24-h postdischarge handoff(s) | PCCc or insurance care manager | +++ | — | — |
Outpatient specialty CMd | — | +++ | +++ | |
Outpatient specialty SWe | — | +++ | +++ | |
5–10 d post discharge | Telehealth appointment with CM and/or SWf: verifies proper use of DME, medications, and follow-up appointment, including authorizations | — | +++ | +++ |
12–30 d postdischarge point of contact | PCC or CMAg: staff returns phone calls and follows-up on outstanding issues | + | ++ | +++ |
CM or SWh: licensed professional who returns phone calls and follows-up on outstanding issues, including alignment with community resources | + | ++ | +++ |
CM, care manager; CMA, care manager assistant; DME, durable medical equipment; LVN, licensed vocational nurse; PCC, patient care coordinator; RN, registered nurse; SMA, spinal muscular atrophy; SW, social worker; +, optional or referral based; ++, mostly (staff prioritized highest risk first and, if there was the capacity, would then focus on moderate risk); +++, always; —, not applicable.
Dedicated discharge navigator is a dedicated RN who focuses on health literacy, education, and discharge planning. Initially started April 8, 2018, with rounding on each patient on the medical floor. This transitioned with the advent of the readmissions risk tool to rounding on moderate- and high-risk patents only, starting December 2, 2018.
Malec A, Mørk A, Hoffman, R, Carlson E. The care team visit: approaching interdisciplinary rounds with renewed focus. J Nurs Care Qual. 2018;33(2):135–142.
PCC or insurance care manager (LVN care coordinators): leveraging external care coordinators embedded in outpatient clinics or at payer level to assist with postdischarge handoff (long-standing policy).
CM embedded into specialty clinics for the most high-risk patients, for example, CM embedded into neurology clinic for patients with SMA who helps coordinate the care between the inpatient stay and follow-up in the specialty clinic or other needed community services (long-standing service).
SW embedded in specialty clinics for the most high-risk patients (long-standing service).
Telehealth appointment with CM and/or SW to verify appropriate follow-up, medication compliance, and acquisition of DME (started May 2019).
CMA: nonclinical staff member who returns phone calls and follows-up on outstanding issues and confirms appointment, including primary care provider and CM or SW telehealth appointment (long-standing service).
Long-standing service.
The intervention protocols implemented in this study were developed on the basis of 5 broad categories of data: SDoH2,3,25–31 ; medication reconciliation and related interventions13,24,32–38 ; other inpatient or discharge planning23,33,35,39–41 ; primary, specialty, and other outpatient care appointments25,39,42–48 ; and postdischarge calls and/or follow-up.28,33,38,45,49–54 A summary of interventions employed and rationale for use is provided in Table 2. The intervention time line is shown in Supplemental Fig 4.
Summary of Interventions and Rationale (With Literature Cited in Text by Intervention Category)
Intervention Category . | Examples . | Evidence and Rationale . |
---|---|---|
SDoH surveys and interventions | Screening for SDoH risk factors and corresponding referrals | SDoH increase the predictive power of regression models, and addressing them may help with nonclinical factors associated with readmissions. This cuts across both inpatient and outpatient interventions. Addressing SDoH has been shown to reduce readmissions. |
Primary care, specialty care, and other outpatient care appointments | Scheduling of PCP, specialty, and public health referral nurses | Primary care may be an essential tool to reduce readmissions through the appropriate transition of care and innovations in care delivery outside the hospital. Case-control studies and other statistical analyses have provided evidence that advances in primary care delivery help in reducing hospital readmissions. The importance of access to primary care may also result in (desirable) increases in use among patients who need it, such as chronically ill patients with previously untreated conditions. |
Medication reconciliation and related interventions | Medications-to-bed and other related optimization initiatives | Medication reconciliation and teach-back methods addressing medication components of discharge planning may result in reduced readmissions but are more likely to be effective when used in combination with other interventions. |
Other inpatient or discharge planning | DNNs, discharge planning on admission | This is integral to addressing complex problems during care in ways unlikely to result in prolonged length of stay. |
Postdischarge calls and/or follow-up | Postdischarge communication with PCP, public health nurse referrals, follow-up PCP appointments (if required), primary care coordinator or insurance manager’s postdischarge calls, handoffs to outpatient care manager or social workers, and telehealth appointment | Postdischarge follow-up may help with additional patient education about discharge conditions and appropriate use of health care resources. Primary care follow-up is required to ensure that patients do not miss their initial or subsequent appointments. |
Intervention Category . | Examples . | Evidence and Rationale . |
---|---|---|
SDoH surveys and interventions | Screening for SDoH risk factors and corresponding referrals | SDoH increase the predictive power of regression models, and addressing them may help with nonclinical factors associated with readmissions. This cuts across both inpatient and outpatient interventions. Addressing SDoH has been shown to reduce readmissions. |
Primary care, specialty care, and other outpatient care appointments | Scheduling of PCP, specialty, and public health referral nurses | Primary care may be an essential tool to reduce readmissions through the appropriate transition of care and innovations in care delivery outside the hospital. Case-control studies and other statistical analyses have provided evidence that advances in primary care delivery help in reducing hospital readmissions. The importance of access to primary care may also result in (desirable) increases in use among patients who need it, such as chronically ill patients with previously untreated conditions. |
Medication reconciliation and related interventions | Medications-to-bed and other related optimization initiatives | Medication reconciliation and teach-back methods addressing medication components of discharge planning may result in reduced readmissions but are more likely to be effective when used in combination with other interventions. |
Other inpatient or discharge planning | DNNs, discharge planning on admission | This is integral to addressing complex problems during care in ways unlikely to result in prolonged length of stay. |
Postdischarge calls and/or follow-up | Postdischarge communication with PCP, public health nurse referrals, follow-up PCP appointments (if required), primary care coordinator or insurance manager’s postdischarge calls, handoffs to outpatient care manager or social workers, and telehealth appointment | Postdischarge follow-up may help with additional patient education about discharge conditions and appropriate use of health care resources. Primary care follow-up is required to ensure that patients do not miss their initial or subsequent appointments. |
PCP, primary care provider.
SDoH
The need to address SDoH is well known.2,3,27,29,31 All patients with a length of stay of ≥48 hours were assigned a social worker, who performed a social worker assessment. This standardized social worker assessment identified any barriers or needs related to SDoH. This assessment included conversations with the patient as well as the parent. The hospital sought and received grants to address social determinants related to access to food and transportation by using vouchers. Starting in July 2019, in addition to the standardized social work assessment, social workers used the Protocol for Responding to and Assessing Patients’ Assets, Risks, and Experiences (PRAPARE) screening tool for moderate- and high-risk patients.29,31 We estimate that ∼25% of families received some form of SDoH support after screening.
Medication Reconciliation and Related Interventions
According to the literature, medication reconciliation and teach-back methods addressing medication components of discharge planning may result in reduced readmissions but are more likely to be effective when used in combination with other interventions.24,32,33,35,36,38 Medication reconciliation involves validating that patients are taking their medications as prescribed and ordered in the medical record. This also includes documentation of the status of administration of existing medication and the administration of new ones. In addition to medication reconciliation and the teach-back method, a program to deliver discharge medications was implemented as the medications-to-bed (or meds-to-bed) program by the hospital’s pharmacy. The discharge medications are brought to the bedside in preparation for discharge. This way the patient and parents have the medication in hand before they leave the hospital. This mitigates any possible challenges with the family not being able to acquire important medication post discharge. So we included a program that ensures that discharge medications are provided to patients while they are at the bedside in addition to proper education on the use of the medications. This medications-to-bed program is managed by the hospital’s pharmacy.
Other Inpatient or Discharge Planning
Discharge planning was initiated within an hour of admission on the basis of the predicted risk of each patient. The risk icon visible in the EHR facilitated more robust discussions on rounds with the family and with each member of the care team. Additionally, a list of patients and corresponding predicted probabilities of readmission was distributed each morning to the appropriate case managers. This allowed the case manager to consider the probability of readmission and related risk factors to tailor discharge conversations and to identify modifiable risk factors, such as the inefficient use of health care resources.
The inclusion of information related to readmission risk strata in conversations about inpatient or discharge planning during multidisciplinary rounds raised awareness of readmission risk and barriers to discharge. This information provided insight to help teams focus decision-making on specific impactful interventions that could potentially prevent readmission. In several studies, researchers have assessed the impact of discharge planning on readmissions and quality of care.35,40,55,56
One of the more novel interventions implemented for higher-risk patients was consultation with a discharge nurse navigator (DNN).57 The DNN participated in daily rounds on any patient deemed moderate or high risk. During the consultation, the DNN focused on patient education and assessed patient health literacy using teach-back methodology to evaluate the family’s understanding of medications, therapies, and discharge plans. The DNN also advocated for the family if any barriers or concerns were identified. The DNN worked to ensure that both the family and the care team were comfortable with discharge and were as prepared as possible for the transition from hospital care. The initial phase of the program reached ∼20% of the general pediatric patients.
Outpatient Care Appointments and Postdischarge Calls and/or Follow-up
Previous studies have indicated that appropriate transition of care55,57,58 from hospitalization to outpatient or primary care services is an important component of efforts to reduce readmissions. As a result, we focused on ensuring that patients make the follow-up care appointments. There are, however, studies indicating that these outpatient appointments on their own may be ineffective.56,59 Our choice of focusing on these transition of care interventions is based on an expectation that it takes a multifactorial approach to reduce readmission rather than focusing on just one form of intervention. Most postdischarge calls were made by the care coordination team to improve transition of care.
A novel intervention implemented for high-risk patients was the postdischarge telehealth visit. This intervention was created to serve as an additional touch point with families soon after discharge to ensure that the transition home proceeded smoothly. An appointment with the assigned social worker and a registered nurse case manager was scheduled for a few days after discharge. This appointment provided an opportunity to assess each family’s ability to review discharge instructions, to schedule appointments, and to obtain and administer medications to the patient. Family members also had the opportunity to ask questions and to obtain necessary resources. The postdischarge telehealth visits also provided an opportunity for the case manager to perform a quick assessment of clinical status.
Impact on Hospital Resources
To secure the resources required to operationalize the interventions, the hospital was initially able to use existing personnel. During the initial phase of the program, the responsibilities of the DNN were assumed by a pediatric nurse whose role was redefined and rebranded to align with the new program. After members of the readmission team were educated about the readmission risk tool, they were able to effectively champion its use. The care coordination team expanded the role of the use review nurse to allow care managers to partner with patients and families more effectively for discharge planning, including follow-up phone calls and scheduling of appointments. We estimate that between one-third and one-half of all patients receive services from the care manager assistant. To improve understanding of the issues and barriers to discharge, the social worker and care manager joined the medical-surgical multidisciplinary care teams during patient rounds. Attending physicians, bedside nurses, and respective leadership were instrumental in strengthening collaboration between disciplines, which was critical for success.
This initial phase of the intervention was piloted with 1 of 6 medical-surgical teams at the hospital and reached ∼20% of general pediatric patients. Because of favorable feedback from providers and patients, the initiative was gradually expanded over 24 months to include other teams. This expansion of the program required hiring additional personnel, including 2 additional DNNs and several care managers, social workers, and care manager assistants. The staffing goal for each team to have a dedicated care manager and social worker has largely been accomplished, with the occasional exception for cross coverage. This has resulted in expansion of the initiative to all medical-surgical patients.
Notably, although additional resources were required to expand the initiative, the pilot phase of the project was completed through the creative use of existing resources. The intervention protocol did require a substantial investment of resources in the form of staff time, but this may prove to be a less formidable barrier than the need to hire additional staff.
Measuring and Comparing Readmission Rate Before and After Model Implementation With Clinical Interventions
Readmission was defined as rehospitalization for any reason after a previous discharge from the hospital within 7 days (for the 7-day metric) and 30 days (for the 30-day metric). Each readmission counted as index admission for subsequent readmission when applicable.
A decision to compare both 7- and 30-day readmissions 3 years (January 2015 to December 2017) before implementation of the readmission model (“Before Predictive Model” in the statistical process control chart) and 2 years (January 2018 to December 2019) after implementation of the predictive model and clinical interventions (“After Predictive Model” in the statistical control process chart) was made a priori. The model and clinical interventions were in place as of January 2018 (taken to be the start date of interventions, although it took several months of previous preparation). The Western Electric rules for process control charts were used to establish whether a significant change in readmission rates had occurred.60,61 We generated the process control charts using monthly readmission rates. Please refer to Fig 2 for the 30-day readmissions chart and Supplemental Fig 5 for the 7-day chart. Furthermore, we estimated the 7- and 30-day readmission rates and corresponding 95% confidence intervals (CIs) for periods before and after the deployment of the model.
Statistical process control p-chart of 30-day readmissions. CL, center line; LCL, lower control limit; UCL, upper control limit.
Statistical process control p-chart of 30-day readmissions. CL, center line; LCL, lower control limit; UCL, upper control limit.
Summary statistics and balancing measures were provided to compare patients’ demographics before and after the predictive modeling period. These measures include characteristics of patients, such as age, sex, and race, as well as hospital characteristics, such as average daily admissions and average hospital length of stay, as a balancing measure. Summary statistics and other measures were retrieved from validated sources of data for the hospital to ensure completeness and accuracy. The proportion of patients classified into each readmission risk strata was also provided (Table 3).
Summary Statistics and Balancing Measures
Variable . | Before Predictive Model . | After Predictive Model . |
---|---|---|
Age, y, mean (SD) | 7 (6) | 7 (6) |
Patient sex, % | ||
Female | 45.2 | 45.2 |
Male | 54.8 | 54.8 |
Patient race, % | ||
American Indian or Alaskan native | 0.1 | 0.1 |
Asian American | 9.3 | 9.8 |
Black or African American | 2.7 | 2.6 |
Native Hawaiian or other Pacific Islander | 0.5 | 0.5 |
White | 61.3 | 61.9 |
Other or unknown | 26.1 | 25.1 |
Hospital statistics | ||
No. hospitalizations | 42 515 | 30 175 |
Readmissions within 7 d | 1636 | 1042 |
Readmissions within 30 d | 5313 | 3363 |
7-d readmission rate (95% CI) | 0.038 (0.037–0.040) | 0.035 (0.033–0.037) |
30-d readmission rate (95% CI) | 0.125 (0.122–0.128) | 0.111 (0.108–0.115) |
Average hospitalizations per day | 39 | 42 |
Length of stay, d, mean (SD) | 5.4 (11.5) | 5.5 (11.8) |
Readmission risk strata, % | ||
Low | N/A | 74.5 |
Moderate | N/A | 13.2 |
High | N/A | 12.7 |
Variable . | Before Predictive Model . | After Predictive Model . |
---|---|---|
Age, y, mean (SD) | 7 (6) | 7 (6) |
Patient sex, % | ||
Female | 45.2 | 45.2 |
Male | 54.8 | 54.8 |
Patient race, % | ||
American Indian or Alaskan native | 0.1 | 0.1 |
Asian American | 9.3 | 9.8 |
Black or African American | 2.7 | 2.6 |
Native Hawaiian or other Pacific Islander | 0.5 | 0.5 |
White | 61.3 | 61.9 |
Other or unknown | 26.1 | 25.1 |
Hospital statistics | ||
No. hospitalizations | 42 515 | 30 175 |
Readmissions within 7 d | 1636 | 1042 |
Readmissions within 30 d | 5313 | 3363 |
7-d readmission rate (95% CI) | 0.038 (0.037–0.040) | 0.035 (0.033–0.037) |
30-d readmission rate (95% CI) | 0.125 (0.122–0.128) | 0.111 (0.108–0.115) |
Average hospitalizations per day | 39 | 42 |
Length of stay, d, mean (SD) | 5.4 (11.5) | 5.5 (11.8) |
Readmission risk strata, % | ||
Low | N/A | 74.5 |
Moderate | N/A | 13.2 |
High | N/A | 12.7 |
N/A, not applicable.
Estimating Cost Savings
In a study by Markham et al,62 the adjusted mean length of stay and cost of a pediatric readmission within 30 days were 3.6 (95% CI: 3.5–3.6) and $6328 (95% CI: $6184–$6475), respectively. These estimates were obtained from the 2013 Agency for Healthcare Research and Quality Nationwide Readmissions Database; appropriate adjustments for age, sex, payer, number of complex chronic conditions, index hospital, and severity of illness were applied.62 The costs per hospitalization of each individual encounter in the database were estimated from hospital-level charges by using hospital-specific cost/charge ratios available through the Agency for Healthcare Research and Quality.62
We made the following assumptions in estimating potential financial savings in readmissions due to changes in readmission rates. First, we estimated the difference between the point estimates of readmission rates between periods during and before the period and used the difference to calculate the expected number of readmissions potentially prevented. This is under the assumption that the point estimate of the readmission rate (during the period before predictive model adoption) would remain unchanged in the absence of the predictive model. We used the national estimates of the cost of a pediatric readmission and the number of readmissions potentially prevented to estimate the health care cost savings.
This study was approved by the institutional review board of the corresponding author with institutional review board number 180743.
Results
The pediatric hospital of study had 42 515 hospitalizations during the period before predictive modeling and 30 175 hospitalization during the predictive modeling period. Patients visiting the hospital were predominantly Hispanic (51%), and summary statistics (including some balancing measures) are shown in Table 3 by predictive modeling period. Table 3 indicates that there has been no shift in patients’ demographics, hospitalizations, and hospital length of stay, although there was an average of 3 more admissions per day during the predictive modeling period.
The statistical process control chart for 30-day readmission is provided in Fig 2, and that for 7-day readmission can be found in Supplemental Fig 5. The charts for 30-day and 7-day readmissions indicate a reduction in readmissions during the predictive modeling period, with such reduction in rates more pronounced in the 30-day metric. In all cases, the significant change in process (reduction in readmissions) occurred during the predictive modeling period per the Western Electric rules for process control charts.
The estimate of savings during the 2-year predictive modeling period was $2 673 264 (95% CI: $2 612 431–$2 735 364) on the basis of the adjusted national estimates of a readmission, a 30-day readmission rate of 11.1% during the predictive modeling period, a 30-day readmission rate of 12.5% during the period before predictive modeling, and a total of 30 175 encounters during the predictive modeling period. In a similar way, the estimate of savings within 7 days was $572 842.20 (95% CI: $559 806.60–586 149.60).
Discussion
In this interventional study, the results from statistical tests of proportions and statistical process control charts revealed that a combination of predictive modeling and targeted clinical interventions were associated with successfully reductions in pediatric hospital readmissions. This association was seen throughout the period after adoption of the predictive model. Predictive models help in identifying children who are most at risk for readmission, but clinical thought and expertise are required to determine which interventions are appropriate and to determine associated risk factors that may be modifiable. There were no significant changes in readmission rates in the years before targeted interventions. This suggests that clinical interventions need to be applied in a targeted manner to achieve maximum benefit.
In this study, the hospital had no tools for identifying at-risk patients before model implementation. At that time, the readmission teams’ approach was rather reactionary than proactive and generalized rather than individualized. In other words, the effort of the readmission team was mainly a universal approach to prevent a second readmission for a patient as opposed to a focused approach to preventing all subsequent readmissions. While providing targeted interventions to both the moderate- and high-risk strata, the clinical interventions team observed that most patients in the high-risk stratum were complex patients for whom readmissions may be difficult to prevent.
The difference in the impact of interventions on 7-day and 30-day readmission is interesting and highlights the complexity of the problem of readmission and the inherent difficulty in understanding and addressing it effectively. It may be expected that the interventions would result in greater reductions in 7-day readmissions, but we saw a more pronounced effect on 30-day readmissions. This may be a statistical effect due to the expected value of preventable readmissions if we can assume that patients readmitted within 7 days and those readmitted within 30 days share more clinical similarities than dissimilarities. Although the reductions in 7-day readmissions were modest, the corresponding clinical and financial impact may be significant given that the reductions in 7-day readmissions may translate to hundreds of thousands of dollars in annual financial savings. The financial savings that may be attributable to reductions in 30-day readmissions was in the millions of dollars. The significant interventional efforts to bridge the outpatient, medical, community, and care coordination resources as well as the 30-day duration of monitoring moderate- and high-risk patients may explain the impact on 30-day readmission rates. The change (drop) in 30-day readmission rates was steep near the start of the deployment of the predictive model. This may be related to the increased awareness on readmissions engendered by the development and adoption of the readmissions model, by the presence of long-standing interventions now buttressed with focused interventions, and by additional interventions put in place.
Over the years, the clinical intervention team observed that families of patients with complex medical issues reported being, at times, overwhelmed with the child’s care and reported experiencing other barriers to good health maintenance, such as food insecurity or transportation issues. These observations indicate that the patients are likely to be readmitted if the needs of the family member caring for the child are not met as well. Communication with the patient and family and the care team is vital for a successful transition and quality of care outcomes. Alignment in efforts within the system of care by all team members contributed to reductions in readmissions to the hospital. It became apparent over time that all providers have important roles to play, and the collaborative work effort led to patients’ successful transition of care across the continuum.
In addition to the direct benefits of decreased readmissions, there have been other benefits of implementing a readmission risk tool. Chief among them is the less tangible but palpable positive effects the initiative has had on the culture of the hospital. Including the predicted readmission risk discussions into daily rounds has had an impact on the workflow of all the team members. Attending physicians now regularly discuss barriers to discharge and risk factors for readmission with the team, aided by case managers who actively participate in rounds. The nursing staff have learned to model the approach of their DNN colleagues to identify barriers and ways to mitigate them. These attributes have amplified the effect of the initiative beyond the capacity of the patients who the DNNs can interface with directly. Lastly, resident physicians are benefitting from an education that incorporates data analytics with bedside medicine in a way that the previous generation of physicians did not have. All of these have created a new social norm that strengthens and reinforces the readmission reduction effort at the hospital.
Moreover, this initiative to decrease readmissions aligns with the new strategic priorities of the hospital. The hospital has adopted a population health strategy that recognizes the importance of what occurs beyond the walls of the hospital. One part of this is connecting patients with community resources (primary and specialty care follow-up, social work, other community resources, etc) so they are less likely to need hospital admission in the future. This creates true value from the patient’s perspective and is beneficial to the entire health system.63
Modifiable risk factors of readmission may include inefficient use of health care resources, such as missing primary care appointments, and risk factors due to poor SDoH. All interventions addressed modifiable risk factors, but interventions around primary care use and on social determinants (such as access to food or malnutrition) have more targeted impact on well-known drivers of readmissions. It is not possible for us to highlight a single intervention as responsible for changes but rather the bundle or group of interventions.20,64 Other institutions should consider combinations that make clinical sense for their populations rather than pursue a single intervention method or strategy.
In this study, we found that several categories of interventions were associated with a reduced 30-day readmission rate. A validated predictive model for readmission is required to replicate these outcomes. This could be a similar readmission risk tool to the model by Ehwerhemuepha et al10 or other methods in addition to EHR chart review. Several such alternate tools exist in the literature.11,65–67 The key is to identify actionable opportunities for targeted interventions. Future studies will define a point-based system for estimating readmission risk on paper for use by organizations that cannot implement this, in the EHR, and via statistical or machine learning models. Medical centers that are interested in similar care management interventions but are limited by resource constraints may be able to pilot these initiatives by reimagining how best to leverage existing skill sets and use data to focus their limited resources. It is expected that a cost will be incurred in the process of setting up the teams and processes required to intervene on patients effectively.
A limitation of this study is that we did not address the identification of which interventions were most impactful or what strategies were associated with higher reductions in hospital readmissions. This could be the focus of future studies because it would reduce any waste of resources associated with application of less effective interventions. But organizations may want to shift attention from focusing on a single, effective intervention because studies on identifying the impact of specific interventions have mixed outcomes.20–23 This may indicate that the most effective approach may be to adopt the most meaningful set of interventions that each organization can afford.20,64 We provided only a qualitative description of some interventions, not the exact number of times certain interventions were conducted. In estimating readmission rates, we reported all-cause (unplanned and planned) readmissions because of the large sample sizes from the 5 years of readmission data that were analyzed that made manual chart reviews infeasible. This implies that we may have underestimated the impact of model-targeted interventions. The tool used in this study was obtained from the published work by Ehwerhemuepha et al10 in 2018. Although details of the model are available for free replication, there will be a cost to translating any such model to a real-time or near real-time EHR tool. The simplest model for readmission that can be (or is) paper-based is the LACE readmission model,16 which has had suboptimal performance in the population from this study10 but is potentially helpful in other populations.68 Condition-specific models may be required when targeting only a subpopulation, such as patients in oncology, neurology, or trauma units.4,12,69,70 Lastly, it must be noted that secular trend or random effects may be captured during the predictive modeling period that may confound any quality improvement outcome, as shown by Finkelstein et al71 in 2020 in their randomized trial on interventions for reducing 180-day readmissions. Our choice of a 30-day metric may guard against such confounding.
Conclusions
We addressed the application of predictive models to inform clinical interventions toward preventing pediatric hospital readmission. This included the strategies for deployment in the EHR, development of clinical interventions for patients based on predicted risk for readmission (or readmission risk strata), and subsequent improvement of care resulting in statistically significant reductions in hospital readmissions. The processes outlined here, lessons learned, and success in clinical outcomes indicate the promise of using predictive analytics in tandem with clinical interventions.
Acknowledgments
The following Children’s Health of Orange County (CHOC) associates and/or providers played a key role in the implementation of interventions outlined in this study: Brittney Anderson, Grace Magedman, Katie Tu, Erika Jewell, Claudia Begino, Megan Beckerle, and Renee Moore. This includes members of the discharge navigation, pharmacy, social worker, care management, and CHOC Readmission Committee teams at CHOC. Neil Garde and his team provided important support during the implementation of the readmission model at CHOC.
FUNDING: No external funding.
Mrs Pugh and Drs Granger, Lusk, and Weiss led all the clinical aspect of this research, including clinical interventions, and drafted, reviewed, and revised the clinical contents of the study; Ms Wright drafted, reviewed, and revised the clinical contents of the study; Dr Feaster participated in the development of the predictive model outlined in this work and reviewed and revised the manuscript; Dr Ehwerhemuepha led all the statistical analyses, machine learning, and model implementation and deployment and drafted, reviewed, and revised corresponding components of the study while overseeing its write-up; and all authors approved the final manuscript as submitted.
Deidentified individual participant data will not be made available because of institutional review board restrictions.
References
Competing Interests
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
Comments