BACKGROUND:

One in five pediatric patients suffers from adverse events related to hospital discharge. Current literature lacks evidence on effective interventions to improve caregiver comprehension (CC) of discharge instructions. We examined if a standardized framework for written and verbal discharge counseling was associated with increased CC of key discharge instructions after discharge from a general pediatric inpatient unit.

METHODS:

An interprofessional team created the SAFER Care framework to encourage standard, comprehensive discharge counseling. Plan-do-study-act cycles included electronic health record smartphrases, educational initiatives, data feedback, visual aids, and family outreach. Caregivers were surveyed by phone within 4 days of discharge. Our primary outcome was the proportion of caregivers correctly responding to all questions related to discharge care, comparing pre- and postintervention periods. Data were plotted on a statistical process control chart to assess the effectiveness of interventions.

RESULTS:

A total of 171 surveys were analyzed in the preintervention period, and 262 surveys were analyzed in the postintervention period. A total of 37% of caregivers correctly responded to all questions in the preintervention period, compared with 62% of caregivers in the postintervention period, meeting rules for special cause variation.

CONCLUSIONS:

Development of the SAFER Care framework and its use in written and verbal discharge counseling was associated with significantly improved CC of discharge instructions in a general pediatric inpatient unit. Further studies should be focused on expanding this to other populations, particularly limited–English-proficiency families.

Patients transitioning from inpatient to outpatient care are susceptible to preventable adverse events1  and medical errors.2  One in five pediatric patients experience problems requiring physician intervention after discharge.3  Pediatric patients may be especially vulnerable because of reliance on caregivers for postdischarge care, requiring caregiver comprehension (CC) to minimize risk.

However, provider-caregiver miscommunication and limited CC of their child’s diagnosis, hospital course, and discharge plan occur commonly.49  More than 70% of caregivers may not recognize symptoms requiring medical attention after discharge.10  Decreased comprehension is associated with decreased adherence to treatment plans, increasing the risk of poor clinical outcomes.11  Additionally, patients and caregivers often do not realize they are misinformed12  and may overestimate their own comprehension.13 

In previous studies, researchers have aimed to improve CC. In the emergency department (ED), interventions associated with improved CC included providing written and verbal discharge instructions.14,15  Inpatient studies were focused on singular aspects of the discharge process, such as medications16,17  or follow-up appointments,18,19  and were often conducted in specific populations.2022  Successful inpatient interventions associated with decreased adverse events or increased CC included diagnosis-specific written discharge plans, counseling in complex-care patients, and nursing-led discharge educational interventions.2224  However, little is known about how to effectively implement standardized discharge counseling in a general pediatric inpatient setting to ensure CC for every child’s postdischarge care.

Our interprofessional team examined the impact of a standardized framework on CC in an urban population at high risk for low health literacy. Our goal was to increase postdischarge CC from 37% to 62% (a 25% absolute increase) by using a standardized, diagnosis-specific written and verbal communication framework. We also examined sustainability and postdischarge health care use.

This quality improvement (QI) research project was conducted at an academic, tertiary care children’s hospital in an urban setting. The pediatric hospital medicine (PHM) service included housestaff teams composed of one physician assistant and four resident teams (including third- and fourth-year medical students) rotating every 2 to 4 weeks, caring for general pediatric and surgical subspecialty patients. Before our interventions, general discharge counseling was not standardized. Housestaff were responsible for verbally counseling caregivers at discharge and completing an after-visit summary (AVS). The AVS included automatically populated medications, a dosing calendar, follow-up appointment information, and free-texted discharge instructions. Nurses completed the dosing calendar and then printed and reviewed the AVS with caregivers and patients. As a part of routine care, nursing leadership completed postdischarge follow-up calls and asked caregivers questions, including whether follow-up appointments were scheduled and whether they filled all medications prescribed. Before and during our interventions, an ongoing independent QI project, entitled “MEDRITES,” was focused on standardizing discharge medication counseling, with the active intervention period occurring between April 2017 and March 2018.9,25  In the MEDRITES project, researchers increased CC on timing, dosage, administration, and side effects of medications.

The preintervention period (from December 2017 to March 2018) was followed by the postintervention period (from March 2018 to March 2019). During the sustainability period (from April to September 2019), the QI team stopped active implementation and measured whether improvements continued.

We included English-speaking caregivers of patients from 0 to 17 years old discharged from the hospital from the PHM service, excluding those comanaged by surgical or complex-care services. Caregivers previously surveyed within the same study period were excluded.

An interprofessional stakeholder team was assembled, including nurses, nurse educators, QI specialists, housestaff, family liaisons, and PHM attending physicians. A family liaison is a caregiver of a frequently admitted child employed by our institution to provide caregiver support. Six key aspects of discharge instructions were identified through expert opinion and literature review, examining discharge instructions reported as essential by caregivers and providers or reported as often omitted by providers.4,10,2632  The QI team developed the SAFER Care (SC) framework (Fig 1) for written and verbal discharge counseling and as an acronym for easy recall: when to return to school, activity restrictions, follow-up appointments, expected symptoms, when to return for care, and who to call with any questions.4,10,2632  The QI team used the Model for Improvement33  to develop and test interventions centered around the SC framework: electronic health record (EHR) changes, educational initiatives, data feedback, visual aids, and family outreach.

FIGURE 1

SC framework. This standardized framework was developed by the QI team, allowed easy recall of 6 key aspects of discharge instructions, and was used by medical staff during written and verbal discharge counseling.

FIGURE 1

SC framework. This standardized framework was developed by the QI team, allowed easy recall of 6 key aspects of discharge instructions, and was used by medical staff during written and verbal discharge counseling.

Close modal

We embedded “smartphrases” into our EHR, Epic Systems. A smartphrase is automatically generated text appearing after typing a specific keyword. Starting in March 2018, SC smartphrases in English and Spanish were used in the AVS as written discharge instructions to guide verbal discharge counseling. Smartphrases were diagnosis based, written at grade-5 to grade-6 reading level, and based on current literature and included a prompt to include smartphrases created through MEDRITES (Supplemental Fig 4).3436  We created and iteratively improved smartphrases for 13 common PHM diagnoses identified by expert opinion (Supplemental Fig 5) and a general smartphrase for other diagnoses not captured.

Housestaff workshops were held at the start of the postintervention period to review the use of the SC framework and effective communication techniques, such as teach-back.37,38  Housestaff workshops were repeated in July 2018 for new interns. Deliberate practice sessions were held within the first week of intern inpatient rotations to practice discharge counseling. Fourth-year medical students had an educational session reviewing the SC framework. Concepts were also shared regularly with PHM attending physicians, and attending physicians on service received biweekly e-mails with the current progress to prompt discussion of SC during daily rounds. The SC framework was shared at nursing meetings, and SC nurse champions distributed a short presentation to colleagues.

Monthly e-mails were sent collectively to all PHM attending physicians and housestaff, highlighting current run charts, project progress, and targeted areas for improvement. In December 2018, we initiated personalized positive feedback, in which individual staff were informed of a caregivers’ positive comments during data collection.

Visual aids were used to raise awareness and remind medical staff to use the SC framework. Posters with the SC framework and smartphrases were placed in housestaff work rooms. We distributed identification badge cards with the SC framework to facilitate quick recall and SC lanyards to promote project awareness.

Worksheets for caregivers in English and Spanish described the SC framework and prompted questions they should ask medical providers before discharge. Caregivers had space under each SC question to write answers if they desired. Worksheets were placed in admission folders given to all caregivers in the inpatient unit. Our family liaison met with caregivers at least once during admission to review SC and distribute additional worksheets.

The primary author (A.U.) identified and called eligible patients from an EHR autogenerated list of patients discharged within the last 4 days. Caregivers of eligible patients discharged within the last 4 days were called in sequential order until a predetermined number of participants were consented (see Statistical Analysis section below), starting at a random patient determined by using a Web-based number randomizer and the patient’s medical record number.

Patient demographics, such as age, sex, race, ethnicity, length of stay (LOS), type of insurance, initial admission unit, and discharge diagnoses, were extracted from the institution’s clinical research database (Looking Glass Clinical Analytics; Streamline Health, Atlanta, GA). Discharge diagnoses were coded and grouped by systems by using the International Classification of Diseases, 10th Revision (ICD-10).39 

Caregivers were administered a seven-question survey based on the six key aspects of discharge instructions (Table 1).4,10,2632  These questions were pilot-tested with five caregivers and two family liaisons and edited on the basis of feedback. For clarity, questions about follow-up appointments were split into multiple parts (Table 1) to capture patients scheduled for follow-up with specialty providers. A correct answer was defined as a caregiver response consistent with expert opinion, current evidence,34,4042  or provider notes from chart review (including AVS), on the basis of methodology from literature investigating CC.43,44  For questions 1 to 5, if a caregiver reported at least one answer consistent with current evidence or chart review, they were marked as correct. For questions 6 to 7, regarding follow-up appointments, caregivers had to know if an appointment was made, the date of the appointment, and the type of doctor. If a question did not apply to the patient (for example, if he or she was asked when the child could return to school, but the child did not ever attend school), then that individual question was excluded from analysis (Table 1).

TABLE 1

Survey Questions and Responses

QuestionCorrect Response DefinitionExamples of Correct Responses for a Caregiver of a Patient With AsthmaPreintervention Caregivers With Correct Responses, n (%)Postintervention Caregivers With Correct Responses, n (%)P
1. What symptoms did your doctor or nurse tell you to look out for that would make you call a health care provider immediately?a Aware of concerning symptoms necessitating acute medical care (eg, difficulty breathing or not eating or drinking). On the basis of current literature, expert opinion, AVS, and medical chart. “Trouble breathing.” 146 (85) 243 (92) .01 
“Using stomach muscles to breathe.” 
2. Who would you call if you had any questions about your child after discharge from the hospital?a Aware to call child’s physician or hospital phone number, on the basis of AVS or medical chart. “Her pediatrician.” 152 (88) 245 (93) .04 
3. What is a possible symptom that your child might continue to have after discharge from the hospital? This is something your provider might have told you to expect after going home.a Aware of symptoms likely to continue after discharge from the hospital, such as coughing. On the basis of current literature, Centers for Disease Control and Prevention guidelines, expert opinion, AVS, and medical chart. “Some coughing.” 122 (71) 202 (77) .27 
4. When can your child return to school or day care?b Aware of any (or none) restrictions to going back to school or day care. On the basis of current literature, expert opinion, AVS, and medical chart. “They told me she could go back the following day.” 68 (85) 114 (93) .04 
5. What activity restrictions does your child have after discharge?c Aware of any (or none) activity restrictions as per AVS or medical chart. On the basis of current literature, expert opinion, AVS, and medical chart. “They said she could go back to her normal activity.” 98 (75) 183 (90) <.01 
6. Did you have a follow-up appointment scheduled for you before discharge from the hospital? When is it?d On the basis of AVS or medical chart documentation. “Yes, her pediatrician on Monday.” 158 (92) 254 (96) .03 
103 (88) 159 (95) .01 
7. What other upcoming appointments do you have scheduled?e On the basis of AVS or medical chart documentation. “Seeing the lung doctor in 3 weeks.” 35 (74) 82 (82) .21 
QuestionCorrect Response DefinitionExamples of Correct Responses for a Caregiver of a Patient With AsthmaPreintervention Caregivers With Correct Responses, n (%)Postintervention Caregivers With Correct Responses, n (%)P
1. What symptoms did your doctor or nurse tell you to look out for that would make you call a health care provider immediately?a Aware of concerning symptoms necessitating acute medical care (eg, difficulty breathing or not eating or drinking). On the basis of current literature, expert opinion, AVS, and medical chart. “Trouble breathing.” 146 (85) 243 (92) .01 
“Using stomach muscles to breathe.” 
2. Who would you call if you had any questions about your child after discharge from the hospital?a Aware to call child’s physician or hospital phone number, on the basis of AVS or medical chart. “Her pediatrician.” 152 (88) 245 (93) .04 
3. What is a possible symptom that your child might continue to have after discharge from the hospital? This is something your provider might have told you to expect after going home.a Aware of symptoms likely to continue after discharge from the hospital, such as coughing. On the basis of current literature, Centers for Disease Control and Prevention guidelines, expert opinion, AVS, and medical chart. “Some coughing.” 122 (71) 202 (77) .27 
4. When can your child return to school or day care?b Aware of any (or none) restrictions to going back to school or day care. On the basis of current literature, expert opinion, AVS, and medical chart. “They told me she could go back the following day.” 68 (85) 114 (93) .04 
5. What activity restrictions does your child have after discharge?c Aware of any (or none) activity restrictions as per AVS or medical chart. On the basis of current literature, expert opinion, AVS, and medical chart. “They said she could go back to her normal activity.” 98 (75) 183 (90) <.01 
6. Did you have a follow-up appointment scheduled for you before discharge from the hospital? When is it?d On the basis of AVS or medical chart documentation. “Yes, her pediatrician on Monday.” 158 (92) 254 (96) .03 
103 (88) 159 (95) .01 
7. What other upcoming appointments do you have scheduled?e On the basis of AVS or medical chart documentation. “Seeing the lung doctor in 3 weeks.” 35 (74) 82 (82) .21 
a

Denominator includes all eligible patients included in the study: preintervention (n = 171); postintervention (n = 262).

b

Denominator only includes patients whose caregivers reported that they attended school or day care: preintervention (n = 80); postintervention (n = 122).

c

Denominator only includes patients >6 mo of age: preintervention (n = 130); postintervention (n = 203).

d

Denominator only includes patients who had appointment scheduled by medical team as per medical chart: preintervention (n = 117); postintervention (n = 166).

e

Denominator only includes patients with documented subspecialty follow-up appointments. Caregivers had to know the type of doctor and date of appointment. Preintervention (n = 47); postintervention (n = 99).

The primary outcome was the proportion of caregivers correctly responding to all survey questions. We analyzed this in a binary “all-or-none” variable because all elements were deemed essential knowledge.4,10,2632  To determine the frequency of failure and identify targets for interventions, individual question failure rates were tracked.

Secondary outcomes included health care use, defined as any ED visit or inpatient admission between 7 and 30 days of hospital discharge at our institution or affiliated institutions. Repeat ED visits included only those who presented in the ED and were not admitted; those admitted were counted only as readmissions. We also examined CC in the top three most frequent discharge diagnoses by ICD-10 coding groups.

The process measure was the proportion of surveyed patients with an appropriately completed SC framework (using the smartphrase) on their AVS.

The balancing measures included LOS and the proportion of surveyed patients discharged with appropriate discharge medication counseling. The LOS of surveyed patients was measured in hours from the initial presentation to discharge and assessed because of concern that increased focus on discharge counseling could increase the length of the discharge process. Discharge medication counseling was measured by MEDRITES smartphrase presence on the AVS and assessed to ensure discharge medication counseling did not decline when the focus shifted to overall discharge instructions.

The primary outcome was measured weekly. A post hoc power analysis for two independent proportions, with a significance level of 0.05 and goal of increasing postdischarge CC from 37% to 62%, identified the study was powered at 99%. During the preintervention period, ten to fifteen caregivers were surveyed weekly, with 10 caregivers surveyed in the beginning of the preintervention period and, then, 15 caregivers, starting on January 28, 2018, after revising the power analysis on the basis of initial baseline data. During the postintervention period, an average of five caregivers was surveyed weekly. During the sustainability period, 10 caregivers were surveyed in a single week each month.

Our primary outcome was analyzed by using statistical process control charts. Preintervention data were recorded weekly, and postintervention data were recorded weekly and presented as every 4 weeks, except for an extra week at the end of the postintervention period (presented as 5 weeks) to maintain similar subgroup sizes for analysis.45  Sustainability data were recorded as a 1-week sample monthly. Data were recorded on a P chart by using QI Charts 2.0.23 (Process Improvement Products, Austin, TX) for Microsoft Excel 2010 (Microsoft, Redmond, WA). We identified system shifts using rules for special cause variation in control charts.33,45 

For process measures and one of the balancing measures (the proportion of surveyed patients discharged with MEDRITES presence on AVS), we used run charts to identify shifts, with six consecutive points all over or below the median, indicating a significant system change requiring a shift of the center median line.45  The baseline rate of the process measure could not be determined because this was a new process included in the intervention bundle. The baseline rate of the balancing measure was determined by tracking the median weekly proportion of surveyed patients discharged with MEDRITES presence on AVS. Both balancing measures were also analyzed by using the Wilcoxon rank test, comparing preintervention and postintervention periods.

Demographics, individual survey questions, discharge diagnoses, and health care use were analyzed by using χ2 analysis or Fisher’s exact test, comparing preintervention and postintervention periods. The Wilcoxon rank test was used for continuous variables with nonnormal distribution. Data were analyzed by using Stata 15.1 (Stata Corp, College Station, TX).

Verbal consent was obtained before survey administration. The Institutional Review Board at the Albert Einstein College of Medicine approved this study.

A total of 1098 caregivers were called. In the preintervention period, 435 caregivers were called, and 172 caregivers consented to be surveyed. In the postintervention period, 663 caregivers were called, and 266 caregivers consented to be surveyed (Fig 2). Three surveys were excluded because they were erroneously completed as part of the postintervention period before the intervention start date or because the weekly sampling size was exceeded. One survey was dropped from both the pre- and postintervention groups because the same caregiver was called twice (for separate admissions).

FIGURE 2

Flowchart. This reveals the percentage of caregivers called, surveyed, and, ultimately, included in the study in preintervention, postintervention, and sustainability periods.

FIGURE 2

Flowchart. This reveals the percentage of caregivers called, surveyed, and, ultimately, included in the study in preintervention, postintervention, and sustainability periods.

Close modal

A total of 433 caregiver surveys were analyzed: 171 in the preintervention period and 262 in the postintervention period. There were no differences in age, sex, race and/or ethnicity, type of insurance (public versus private), initial inpatient admission unit, or discharge diagnosis (Table 2). The time to survey administration from discharge differed significantly between the pre- and postintervention periods (2.0 vs 2.4 days; Table 2).

TABLE 2

Demographics and Basic Characteristics of Patients

Preintervention (n = 171)Postintervention (n = 262)P
Age, y, median (IQR) 1.41 (0.5–4) 1.9 (0.6–4) .14 
Male sex 108 (63) 166 (63) .91 
Race and/or ethnicity, n (%)   .58 
 Hispanic 76 (44) 110 (42)  
 Non-Hispanic Black 55 (32) 72 (27)  
 Non-Hispanic white 5 (3) 10 (4)  
 Non-Hispanic other 15 (9) 33 (13)  
 Unknown 20 (12) 37 (14)  
Time from discharge at survey administration, d, mean (SD) 2.0 (1.1) 2.4 (1.1) <.001 
LOS, d, median (IQR) 1.8 (1.1–3.0) 1.8 (1.1–2.9) .85 
Public insurance,an (%) 111 (70) 192 (75) .24 
Initial admission unit,bn (%) 145 (90) 226 (87) .33 
Diagnoses,c most common, n (%)   .16 
 Respiratory diseases 110 (64) 157 (60)  
 Gastrointestinal diseases 32 (19) 39 (15)  
 Abnormal symptoms or laboratory valuesd 8 (5) 25 (9.5)  
Preintervention (n = 171)Postintervention (n = 262)P
Age, y, median (IQR) 1.41 (0.5–4) 1.9 (0.6–4) .14 
Male sex 108 (63) 166 (63) .91 
Race and/or ethnicity, n (%)   .58 
 Hispanic 76 (44) 110 (42)  
 Non-Hispanic Black 55 (32) 72 (27)  
 Non-Hispanic white 5 (3) 10 (4)  
 Non-Hispanic other 15 (9) 33 (13)  
 Unknown 20 (12) 37 (14)  
Time from discharge at survey administration, d, mean (SD) 2.0 (1.1) 2.4 (1.1) <.001 
LOS, d, median (IQR) 1.8 (1.1–3.0) 1.8 (1.1–2.9) .85 
Public insurance,an (%) 111 (70) 192 (75) .24 
Initial admission unit,bn (%) 145 (90) 226 (87) .33 
Diagnoses,c most common, n (%)   .16 
 Respiratory diseases 110 (64) 157 (60)  
 Gastrointestinal diseases 32 (19) 39 (15)  
 Abnormal symptoms or laboratory valuesd 8 (5) 25 (9.5)  

IQR, interquartile range.

a

Compared to private insurance.

b

The percent is of those on the general pediatric floor in comparison with PICU admissions.

c

Grouped by ICD-10 codes.

d

Per ICD-10 coding, includes febrile infant, febrile seizures, and brief resolved unexplained event.

In the preintervention period, 37% of caregivers correctly responded to all questions. In the postintervention period, special cause variation occurred (8 points above the previous center line), raising the mean to 62% and meeting our goal (Fig 3). Mann–Whitney U test analysis comparing similar time periods (from December 2017 to March 2018 and December 2018 to March 2019) revealed a significant change (P = .002). These improvements were sustained, with a mean of 70% for 6 months postinterventions (Fig 3).

FIGURE 3

P chart: percentage of caregivers answering all questions correctly. This control chart reveals the mean percentage of caregivers answering all questions correctly in preintervention, postintervention, and sustainability periods. LCL, lower control limit; UCL, upper control limit.

FIGURE 3

P chart: percentage of caregivers answering all questions correctly. This control chart reveals the mean percentage of caregivers answering all questions correctly in preintervention, postintervention, and sustainability periods. LCL, lower control limit; UCL, upper control limit.

Close modal

The questions most commonly answered incorrectly in both the pre- and postintervention periods were related to subspecialty appointments and expected symptoms after discharge (Table 1). The proportions of caregivers answering these questions correctly did not show significant change (P = .21 and P = .27, respectively), although we did find improvements comparing the pre- and postintervention periods for all other survey questions (Table 1).

We did not find differences in 7- and 30-day readmissions, ED visits, or combined use between the pre- and postintervention periods (Table 3).

TABLE 3

Primary and Secondary Outcomes

Preintervention (n = 171)Postintervention (n = 262)P
Proportion of caregivers with all correct responses, comparing similar time periods preinterventions and postinterventions,a %, mean (SD) 38 (0.12) 66 (0.21)b .002 
Caregivers with correct responses, by diagnosis, n (%)    
 Respiratory diseasesc 45 (41) 104 (66) <.001 
 Gastrointestinal diseasesc 9 (28) 22 (58) .01 
 Abnormal symptoms or laboratory valuesd 2 (25) 15 (60) .09 
7 d health used,e 4 (2) 5 (2) .74 
 7-d repeat ED presentationd,f 3 (2) 5 (2) 1.00 
 7-d readmissionsd 1 (0.5) 0 (0) .40 
30 d health usea,c 21 (12) 28 (10) .64 
 30-d repeat ED presentationc,f 14 (8) 24 (9) .86 
 30-d readmissionsc 8 (4) 7 (3) .29 
Preintervention (n = 171)Postintervention (n = 262)P
Proportion of caregivers with all correct responses, comparing similar time periods preinterventions and postinterventions,a %, mean (SD) 38 (0.12) 66 (0.21)b .002 
Caregivers with correct responses, by diagnosis, n (%)    
 Respiratory diseasesc 45 (41) 104 (66) <.001 
 Gastrointestinal diseasesc 9 (28) 22 (58) .01 
 Abnormal symptoms or laboratory valuesd 2 (25) 15 (60) .09 
7 d health used,e 4 (2) 5 (2) .74 
 7-d repeat ED presentationd,f 3 (2) 5 (2) 1.00 
 7-d readmissionsd 1 (0.5) 0 (0) .40 
30 d health usea,c 21 (12) 28 (10) .64 
 30-d repeat ED presentationc,f 14 (8) 24 (9) .86 
 30-d readmissionsc 8 (4) 7 (3) .29 
a

Time periods are December 2017 to March 2018 and December 2018 to March 2019 to compare similar time periods. The Mann–Whitney U test was used for analysis.

b

n = 70.

c

χ2 analysis.

d

Fisher’s exact test.

e

Defined as any ED visit or readmission within 7 or 30 d after hospital discharge.

f

Discharged from ED.

We found improvement in CC in the 2 most common discharge ICD-10 coding groups, respiratory and gastrointestinal diseases (P < .001 and P = .01, respectively). Although we did find an increase in CC in our third most common discharge diagnosis, abnormal symptoms or laboratory values (including febrile infants and febrile seizures), this was not significant (P = .09; Table 3).

Analysis of our process measures comparing pre- and postintervention periods revealed that an SC smartphrase was used in 89% of all postintervention AVSs reviewed. When plotted on a run chart, the postintervention median was 100% because of multiple weeks in which all AVSs reviewed revealed appropriate usage of SC (Supplemental Fig 6A).

Analysis of our balancing measures via Wilcoxon rank tests revealed no significant reduction in MEDRITES smartphrase presence (71% vs 79%; P = .05; Supplemental Fig 6B) or increase in LOS (1.8 vs 1.8 days; P =.85l Table 2) when comparing pre- and postintervention periods. When plotted on a run chart, analysis of MEDRITES smartphrase presence (Supplemental Fig 6B) identified a significant shift, with 6 consecutive points all over or below the preintervention median.45 

Our interprofessional QI team developed interventions centered around SC, a standardized communication framework. These interventions were associated with improved CC of discharge instructions, despite a longer average time to survey administration in the postintervention group, suggesting a longer retention of discharge information. Improvements were sustained for 6 months. There was high fidelity to the interventions and no change in balancing measures. To our knowledge, this is the first study to examine the impact of the relatively low-cost interventions on CC of postdischarge care from a general pediatric inpatient setting.

Our findings are consistent with studies in which researchers examine the impact of standardized discharge counseling on CC. In a systematic review, researchers examining CC at discharge from acute hospital settings (an ED and an inpatient burn unit) found that caregivers receiving standardized written and verbal information had higher knowledge scores about their child’s diagnosis and care compared with caregivers receiving only written or verbal information.14  We expanded on these studies by implementing a standardized framework for written and verbal discharge counseling for multiple diagnoses in a general pediatric inpatient setting. We believe standardization of discharge information across multiple formats contributed to the success of this project.

Ease of access to the SC framework also contributed to its success. In previous studies, it is suggested that EHR-based smartphrases can improve provider practices, documentation, and patient adherence.46,47  EHR-based smartphrases likely enabled high fidelity to interventions because of easy incorporation into housestaff workflow.

CC about expected symptoms after discharge did not significantly improve comparing pre- and postintervention periods. Despite targeted efforts by our QI team, including emphasizing expected symptoms as an area for improvement during routine monthly data feedback and educational sessions, it remained the category with the highest percentage of incorrect responses. Similarly, Navanandan et al,31  examining caregiver knowledge in patients presenting to the ED within 72 hours of ED discharge, found that 47% of caregivers reported that ED staff did not explain how long their child’s symptoms would last. The reason for this gap in CC is likely multifactorial, both because of suboptimal counseling by providers and because caregivers may not expect their child to still have symptoms after discharge.48  Interventions devoted to decreasing adverse outcomes and improving CC may be well directed to focus on expected symptoms, including the duration of illness, after hospital discharge.

Although we did not power for this outcome, in our study, we did not find significant decreases in health care use after SC implementation. This is consistent with previous studies studying implementation of discharge interventions in inpatient populations.23,30,49,50  In addition, SC was not associated with increased readmission rates, in contrast to Wu et al’s50  study, in which discharge-related failures decreased and family discharge readiness improved (including through improved provider-caregiver communication), by using a children’s hospital collaborative. This may have been for several reasons; we included only those admitted to the PHM service, whereas Wu et al50  included sickle cell disease patients, who likely require more frequent health care use. Our study was also conducted at a single institution, allowing for enhanced consistency.

There are several limitations to this study. Generalizability may be limited because this study was conducted at a single institution, and the MEDRITES project may have improved our outcomes because of previous institutional focus on discharge medication counseling. However, preintervention data revealing poor CC was collected during active MEDRITES implementation. Other institutions hoping to replicate this study may require a modified SC framework to fully include discharge medication counseling. Our preintervention data collection period was limited to 12 weeks in contrast to our year-long postintervention period, which may have impacted our findings. CC was measured by parental self-report, which may not accurately measure CC, although our strategy to measure CC mirrored previous studies.43,44  Having a single chart reviewer may introduce the possibility of bias but, also, allowed for consistency. We did not include limited–English-proficiency caregivers, who may be at a higher risk for poor comprehension. A nonvalidated framework and survey were used, although validated frameworks and surveys for CC do not currently exist. Survey participation may have selected for patients with better comprehension, although rates of survey participation were similar in all periods. The process and balancing measures tracked for this project only captured written counseling, although verbal counseling may have increased adherence to our interventions. Finally, our study was not powered to find significant differences in our secondary outcomes or our balancing and process measures, and we may be subject to a type 2 error. In particular, the analysis of our balancing measure of MEDRITES smartphrase presence may be underpowered because the absolute percentage of MEDRITES smartphrase presence, paradoxically, increased during the postintervention phase and a run chart plotting this balancing measure identified a significant shift.

Increasing CC at hospital discharge remains challenging. A potential approach includes developing interventions around a standardized written and verbal communication framework. Future studies should be focused on increasing CC of expected symptoms and examining whether this framework can be expanded to limited–English-proficiency families and those comanaged by surgical or complex-care services.

We acknowledge Anjali Modi, Meghan Kelly and Judilka Lalane for their help implementing this project. We also thank the pediatric housestaff, nurses, staff, and attending physicians at the Children’s Hospital at Montefiore for their dedication and support.

Dr Uong conceptualized and designed the study, designed the data collection instruments, collected data, conducted analyses, and drafted the initial manuscript; Dr Rinke conceptualized and designed the study, helped design the data collection instruments, and supervised data collection and data analysis; Drs Hametz and Philips helped design the data collection instruments and helped design and conduct interventions; Drs Dunbar, Jain, O’Connor, Offenbacher, and Eliezer and Ms Pilnick and Ms Kiely helped design and conduct interventions; and all authors and reviewed and revised the manuscript, approved the final manuscript as submitted, and agree to be accountable for all aspects of the work.

FUNDING: Supported by the National Institutes of Health National Center for Advancing Translational Sciences through Einstein-Montefiore Clinical and Translational Science Awards (grants TL1TR001072 and UL1TR001073). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the National Institutes of Health. Funded by the National Institutes of Health (NIH).

     
  • AVS

    after-visit summary

  •  
  • CC

    caregiver comprehension

  •  
  • ED

    emergency department

  •  
  • EHR

    electronic health record

  •  
  • ICD-10

    International Classification of Diseases, 10th Revision

  •  
  • LOS

    length of stay

  •  
  • PHM

    pediatric hospital medicine

  •  
  • QI

    quality improvement

  •  
  • SC

    SAFER Care

1
Forster
AJ
,
Clark
HD
,
Menard
A
, et al
.
Adverse events among medical patients after discharge from hospital. [published correction appears in CMAJ. 2004;170(5):771]
.
CMAJ
.
2004
;
170
(
3
):
345
349
2
Moore
C
,
Wisnivesky
J
,
Williams
S
,
McGinn
T
.
Medical errors related to discontinuity of care from an inpatient to an outpatient setting
.
J Gen Intern Med
.
2003
;
18
(
8
):
646
651
3
Heath
J
,
Dancel
R
,
Stephens
JR
.
Postdischarge phone calls after pediatric hospitalization: an observational study
.
Hosp Pediatr
.
2015
;
5
(
5
):
241
248
4
Bhansali
P
,
Washofsky
A
,
Romrell
E
,
Birch
S
,
Winer
JC
,
Hoffner
W
.
Parental understanding of hospital course and discharge plan
.
Hosp Pediatr
.
2016
;
6
(
8
):
449
455
5
Khan
A
,
Furtak
SL
,
Melvin
P
,
Rogers
JE
,
Schuster
MA
,
Landrigan
CP
.
Parent-provider miscommunications in hospitalized children
.
Hosp Pediatr
.
2017
;
7
(
9
):
505
515
6
Khan
A
,
Rogers
JE
,
Melvin
P
, et al
.
Physician and nurse nighttime communication and parents’ hospital experience
.
Pediatrics
.
2015
;
136
(
5
).
7
Khan
A
,
Rogers
JE
,
Forster
CS
,
Furtak
SL
,
Schuster
MA
,
Landrigan
CP
.
Communication and shared understanding between parents and resident-physicians at night
.
Hosp Pediatr
.
2016
;
6
(
6
):
319
329
8
Khan
A
,
Spector
ND
,
Baird
JD
, et al
.
Patient safety after implementation of a coproduced family centered communication programme: multicenter before and after intervention study
.
BMJ
.
2018
;
363
:
k4764
9
Philips
K
,
Zhou
R
,
Lee
DS
, et al
.
Caregiver medication management and understanding after pediatric hospital discharge
.
Hosp Pediatr
.
2019
;
9
(
11
):
844
850
10
Glick
AF
,
Farkas
JS
,
Nicholson
J
, et al
.
Parental management of discharge instructions: a systematic review
.
Pediatrics
.
2017
;
140
(
2
):
e20164165
11
Clarke
C
,
Friedman
SM
,
Shi
K
,
Arenovich
T
,
Monzon
J
,
Culligan
C
.
Emergency department discharge instructions comprehension and compliance study
.
CJEM
.
2005
;
7
(
1
):
5
11
12
Engel
KG
,
Heisler
M
,
Smith
DM
,
Robinson
CH
,
Forman
JH
,
Ubel
PA
.
Patient comprehension of emergency department care and instructions: are patients aware of when they do not understand?
Ann Emerg Med
.
2009
;
53
(
4
):
454.e15
-
461.e15
13
Glick
AF
,
Farkas
JS
,
Rosenberg
RE
, et al
.
Accuracy of parent perception of comprehension of discharge instructions: role of plan complexity and health literacy
.
Acad Pediatr
.
2020
;
20
(
4
):
516
523
14
Johnson
A
,
Sandford
J
.
Written and verbal information versus verbal information only for patients being discharged from acute hospital settings to home: systematic review
.
Health Educ Res
.
2005
;
20
(
4
):
423
429
15
Al-Harthy
N
,
Sudersanadas
KM
,
Al-Mutairi
M
, et al
.
Efficacy of patient discharge instructions: a pointer toward caregiver friendly communication methods from pediatric emergency personnel
.
J Family Community Med
.
2016
;
23
(
3
):
155
160
16
Twagirumukiza
M
,
Kayumba
PC
,
Kips
JG
, et al
.
Evaluation of medication adherence methods in the treatment of malaria in Rwandan infants
.
Malar J
.
2010
;
9
:
206
17
Voirol
P
,
Kayser
SR
,
Chang
CY
,
Chang
QL
,
Youmans
SL
.
Impact of pharmacists’ interventions on the pediatric discharge medication process
.
Ann Pharmacother
.
2004
;
38
(
10
):
1597
1602
18
Wiens
MO
,
Kumbakumba
E
,
Larson
CP
, et al
.
Scheduled follow-up referrals and simple prevention kits including counseling to improve post-discharge outcomes among children in Uganda: a proof-of-concept study
.
Glob Health Sci Pract
.
2016
;
4
(
3
):
422
434
19
Bergert
L
,
Patel
SJ
,
Kimata
C
,
Zhang
G
,
Matthews
WJ
 Jr
.
Linking patient-centered medical home and asthma measures reduces hospital readmission rates
.
Pediatrics
.
2014
;
134
(
1
).
20
Jenkins
HM
,
Blank
V
,
Miller
K
,
Turner
J
,
Stanwick
RS
.
A randomized single-blind evaluation of a discharge teaching book for pediatric patients with burns
.
J Burn Care Rehabil
.
1996
;
17
(
1
):
49
61
21
Staveski
SL
,
Parveen
VP
,
Madathil
SB
,
Kools
S
,
Franck
LS
.
Parent education discharge instruction program for care of children at home after cardiac surgery in Southern India
.
Cardiol Young
.
2016
;
26
(
6
):
1213
1220
22
Coller
RJ
,
Klitzner
TS
,
Lerner
CF
, et al
.
Complex care hospital use and postdischarge coaching: a randomized controlled trial
.
Pediatrics
.
2018
;
142
(
2
):
e20174278
23
Mallory
LA
,
Osorio
SN
,
Prato
BS
, et al.;
IMPACT Pilot Study Group
.
Project IMPACT pilot report: feasibility of implementing a hospital-to-home transition bundle
.
Pediatrics
.
2017
;
139
(
3
):
e20154626
24
Staveski
SL
,
Zhelva
B
,
Paul
R
, et al
.
Pediatric cardiac surgery Parent Education Discharge Instruction (PEDI) program: a pilot study
.
World J Pediatr Congenit Heart Surg
.
2015
;
6
(
1
):
18
25
25
Philips
K
,
Zhou
R
,
Lee
DS
, et al
.
Implementation of a standardized approach to improve the pediatric discharge medication process
.
Pediatrics
.
2021
;
147
(
2
):
e20192711
26
Curran
JA
,
Murphy
A
,
Burns
E
, et al
.
Essential content for discharge instructions in pediatric emergency care: a Delphi study
.
Pediatr Emerg Care
.
2018
;
34
(
5
):
339
343
27
Amin
D
,
Ford
R
,
Ghazarian
SR
,
Love
B
,
Cheng
TL
.
Parent and physician perceptions regarding preventability of pediatric readmissions
.
Hosp Pediatr
.
2016
;
6
(
2
):
80
87
28
Berry
JG
,
Ziniel
SI
,
Freeman
L
, et al
.
Hospital readmission and parent perceptions of their child’s hospital discharge
.
Int J Qual Health Care
.
2013
;
25
(
5
):
573
581
29
Hamline
MY
,
Speier
RL
,
Vu
PD
, et al
.
Hospital-to-home interventions, use, and satisfaction: a meta-analysis
.
Pediatrics
.
2018
;
142
(
5
):
e20180442
30
Auger
KA
,
Kenyon
CC
,
Feudtner
C
,
Davis
MM
.
Pediatric hospital discharge interventions to reduce subsequent utilization: a systematic review
.
J Hosp Med
.
2014
;
9
(
4
):
251
260
31
Navanandan
N
,
Schmidt
SK
,
Cabrera
N
,
DiStefano
MC
,
Mistry
RD
.
The caregiver perspective on unscheduled 72-hour return visits to pediatric acute care sites: a focus on discharge processes
.
Acad Pediatr
.
2017
;
17
(
7
):
755
761
32
Vepraskas
SH
,
O’Day
P
,
Zhang
L
,
Simpson
P
,
Gage
S
.
Parents support teach-back, demonstration, and a postdischarge phone call to augment discharge education
.
Hosp Pediatr
.
2018
;
8
(
12
):
778
784
33
Langley
GJ
,
Moen
RD
,
Nolan
K
,
Nolan
T
,
Norman
CL
,
Provost
LP
.
The Improvement Guide: A Practical Approach to Enhancing Organizational Performance
.
San Francisco, CA
:
Jossey-Bass
;
1996
34
Castor
ML
.
Safe Swimming: Talk to Parents About Preventing Recreational Water Illnesses
.
Itasca, IL
:
American Academy of Pediatrics
;
2004
35
Unaka
NI
,
Statile
A
,
Haney
J
,
Beck
AF
,
Brady
PW
,
Jerardi
KE
.
Assessment of readability, understandability, and completeness of pediatric hospital medicine discharge instructions
.
J Hosp Med
.
2017
;
12
(
2
):
98
101
36
Yin
HS
,
Gupta
RS
,
Mendelsohn
AL
, et al
.
Use of a low-literacy written action plan to improve parent understanding of pediatric asthma management: a randomized controlled study
.
J Asthma
.
2017
;
54
(
9
):
919
929
37
Dinh
HTT
,
Bonner
A
,
Ramsbotham
J
,
Clark
R
.
Cluster randomized controlled trial testing the effectiveness of a self-management intervention using the teach-back method for people with heart failure
.
Nurs Health Sci
.
2019
;
21
(
4
):
436
444
38
Kornburger
C
,
Gibson
C
,
Sadowski
S
,
Maletta
K
,
Klingbeil
C
.
Using “teach-back” to promote a safe transition from hospital to home: an evidence-based approach to improving the discharge process
.
J Pediatr Nurs
.
2013
;
28
(
3
):
282
291
39
Centers for Medicare and Medicaid Services
.
ICD-10-CM Official Guidelines for Coding and Reporting. FY 2019
.
Baltimore, MD
:
Centers for Medicare and Medicaid Services
;
2019
40
Schleich
K
,
Slayman
T
,
West
D
,
Smoot
K
.
Return to play after exertional rhabdomyolysis
.
J Athl Train
.
2016
;
51
(
5
):
406
409
41
Reddel
HK
,
Taylor
DR
,
Bateman
ED
, et al.;
American Thoracic Society/European Respiratory Society Task Force on Asthma Control and Exacerbations
.
An official American Thoracic Society/European Respiratory Society statement: asthma control and exacerbations: standardizing endpoints for clinical asthma trials and clinical practice
.
Am J Respir Crit Care Med
.
2009
;
180
(
1
):
59
99
42
McWilliams
DB
,
Jacobson
RM
,
Van Houten
HK
,
Naessens
JM
,
Ytterberg
KL
.
A program of anticipatory guidance for the prevention of emergency department visits for ear pain
.
Arch Pediatr Adolesc Med
.
2008
;
162
(
2
):
151
156
43
Lion
KC
,
Mangione-Smith
R
,
Martyn
M
,
Hencz
P
,
Fernandez
J
,
Tamura
G
.
Comprehension on family-centered rounds for limited English proficient families
.
Acad Pediatr
.
2013
;
13
(
3
):
236
242
44
Needle
JS
,
O’Riordan
M
,
Smith
PG
.
Parental anxiety and medical comprehension within 24 hrs of a child’s admission to the pediatric intensive care unit
.
Pediatr Crit Care Med
.
2009
;
10
(
6
):
668
674; quiz 674
45
Provost
LP
,
Murray
SK
.
The Health Care Data Guide: Learning from Data for Improvement
.
San Francisco, CA
:
Jossey-Bass
;
2011
46
Nusrat
M
,
Parkes
A
,
Kieser
R
, et al
.
Standardizing opioid prescribing practices for cancer-related pain via a novel interactive documentation template at a public hospital
.
J Oncol Pract
.
2019
;
15
(
11
):
e989
e996
47
Pfeifer
K
,
Slawski
B
,
Manley
A-M
,
Nelson
V
,
Haines
M
.
Improving preoperative medication compliance with standardized instructions
.
Minerva Anestesiol
.
2016
;
82
(
1
):
44
49
48
Augustine
EM
,
Bekker
TD
,
Goel
A
,
Breslin
KA
,
Chamberlain
JM
.
A survey of return visits to the pediatric emergency department: the caretakers’ perspective
.
Pediatr Emerg Care
.
2018
;
34
(
1
):
27
32
49
Verhaegh
KJ
,
Buurman
BM
,
Veenboer
GC
,
de Rooij
SE
,
Geerlings
SE
.
The implementation of a comprehensive discharge bundle to improve the discharge process: a quasi-experimental study
.
Neth J Med
.
2014
;
72
(
6
):
318
325
50
Wu
S
,
Tyler
A
,
Logsdon
T
, et al
.
A quality improvement collaborative to improve the discharge process for hospitalized children
.
Pediatrics
.
2016
;
138
(
2
):
e20143604

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

Supplementary data