Sim-Based Home Tracheostomy Care: A Mixed Methods Study on Outcomes and Parental Preparedness

The study employed a mixed methods approach, comprising qualitative interviews with caregivers after participation in SDP and a retrospective case-control comparison to assess clinical outcomes. This study and its design were approved by the Institutional Review Board (IRB Protocol# 21-07023718).

A qualitative study was performed to elicit and analyze feedback from caregivers of children with tracheostomies. The research team included experts in pediatric critical care and emergency medicine, an education associate professor, and biostatisticians. One team member, experienced in working with families with children with tracheostomies but not directly involved in SDP, conducted the caregiver interviews. Participants included caregivers whose child or dependent was <21 years of age and was hospitalized from August 2018 (ie, initiation of SDP) to July 2021, and who personally participated in SDP before discharge.

Researchers attempted to contact caregivers using recorded telephone numbers in the institution's electronic medical record (EMR) to inform them of the study being conducted and to enroll interested participants. In instances where multiple caregivers from a family had participated in SDP, we sought to include all available caregivers. Of those who were reachable, after completing an informed consent process, participating SDP caregivers engaged in a 15-minute interview via telephone using a scripted set of interview questions (Supplemental Fig 1). The interview script was developed collaboratively by our research team, drawing on their collective expertise in qualitative methodologies and clinical aspects of tracheostomy care. It covered the entirety of the SDP experience, including teaching, simulation, and peer-to-peer support.

All data were anonymized or deidentified before analysis to protect participant confidentiality. Transcripts from interviews were collated and used to conduct a thematic, descriptive analysis using the widely accepted framework laid out by Braun and Clarke: transcripts were read and coded systematically across the entire data set, and codes were subsequently collated into themes.⁸  The initial coding of transcripts was conducted by 1 individual. To enhance the robustness of our findings, this was subsequently reviewed and refined by 3 additional researchers, each skilled in qualitative methodologies.

A retrospective case-control study was designed to compare outcomes between children with new tracheostomies whose parents have completed SDP versus those whose caregivers have not participated.

Children included in the SDP group were aged <21 years with tracheostomy placement during hospitalization at the authors’ institution between August 2018 (ie, initiation of SDP) and March 2020 whose caregivers completed SDP. Children included in the control group were aged <21 years with tracheostomy placement during hospitalization at the authors’ institution between January 2010 and March 2020 whose caregivers were not enrolled in SDP. March 2020 was used as a cutoff point as the implementation of SDP was temporarily altered during the coronavirus disease 2019 (COVID-19) pandemic. Participation in SDP was offered to all families of children with a tracheostomy. However, for the quantitative analysis, we excluded children with existing tracheostomies, focusing on the impact of SDP on new caregivers to account for the fact that previous experience with tracheostomy care could significantly influence outcomes. Children were also excluded if they had a previous history of tracheostomy reversed before admission or if they were transferred to another hospital before discharge.

Baseline demographic data were collected for both groups via chart review of the author’s institutional EMR. We incorporated self-reported “ethnicity” to account for potential cultural or socio-environmental factors that may influence the outcomes of our study. Regarding “Patient Disposition from the PICU/Hospital,” our EMR system provided only 2 options for categorizing discharge destinations: “home with home health services” and “acute rehabilitation facility.” Because of the relatively small sample size of the SDP group and to reduce bias, propensity scoring was employed to match children in the SDP group with children in the non-SDP group at a 1:1 ratio based on discharge disposition and age, as these variables were felt to have the greatest impact on subsequent clinical outcomes. Nearest neighbor matching without replacement was used to match children using propensity scores generated by a logistic regression.

To account for children with tracheostomies who may present to an outside institutions’ ED for acute care, outcomes were obtained via the INSIGHT clinical research network, which contains data from EMRs across 5 major New York City health systems (ie, Montefiore, Columbia, Cornell, Mount Sinai, and NYU).

The primary outcome measured was ED visits for tracheostomy-related issues within 1 year of discharge from the hospital identified via International Classification of Diseases, 9^th and 10^th Revisions (ICD9/10) coding. This primary outcome was selected based on the premise that well-prepared caregivers are better equipped to manage care at home, possibly reducing the need for emergency interventions. Secondary outcomes included readmissions for tracheostomy-related complications, chest x-rays performed (identified via procedure codes), and antibiotic treatment administered (Supplemental Table 4). These secondary outcomes were included to reflect the broader spectrum of care challenges that might necessitate these interventions.

Although a formal sample size or power calculation was not initially performed because of the exploratory nature of this investigation, we estimated that with our sample sizes, our study had approximately 80% power to detect a difference in proportions of 30% between groups, using a 2-sided Fisher’s Exact Test at a 0.05 significance level. This estimation is based on an assumed 60% ED return rate in the control group, drawing from preliminary findings in similar patient populations at our institution. Descriptive statistics, including N (%), median, and IQR were used to summarize the SDP and non-SDP groups. Pearson’s Χ² test or Fisher’s exact test, where appropriate, was used to compare baseline characteristics between the 2 groups. We tested the normality of each of our outcomes of interest using the Shapiro-Wilk test. The Wilcoxon rank sum test was performed for outcomes with non-normal distributions and the Student’s t test was used when the data were continuous and followed a normal distribution. Statistical significance was evaluated at the .05 α level. All analyses were conducted in R Version 4.2.3 (R Foundation for Statistical Computing, Vienna, Austria).

Participants eligible for this qualitative analysis were principally primary caregivers, including mothers (n = 25), fathers (n = 17), grandmothers (n = 3), a stepfather (n = 1), a sister (n = 1), a patient (n = 1), and a home aid (n = 1). We conducted interviews with caregivers of 16 out of the 27 children and 1 actual patient. This selection was guided by the principle of thematic saturation, where data collection is discontinued once new interviews cease to provide additional insights. The follow-up period for each caregiver’s interview post-SDP participation varied, with a range of 1 year extending up to 4 years and 9 months. After coding transcripts of interviews, a thematic table was constructed and honed through an iterative and collaborative process (Table 1). The 5 themes revealed from our analysis are: knowledge acquisition, active learning, comfort and preparedness, home application of skills, and overall assessment.

The SDP group consisted of 27 children (median age 26 months [IQR 5.5 months–11 years]) whose caregivers underwent SDP training. After propensity score matching, the non-SDP group contained 27 children (median age 16 months [IQR 3.5 months–10 years]) whose caregivers did not participate in SDP. Demographic information for these 2 groups were compared in Table 2.

There was no significant difference between the SDP group and the non-SDP group in our primary outcome, ED visits for tracheostomy-related complications within the first year of discharge from the hospital (2 [IQR 0–2] vs 1 [IQR 0–2], P = .2). Further, there was no significant difference between the SDP group and the non-SDP group for number of readmissions (3 [IQR 1–6] vs 2 [IQR 0–4], P = .2), number of total chest x-rays (4 [IQR 1–10] vs 5 [IQR 0–11], P > .9), or any antibiotic use (54% vs 61%, P = .6). (Table 3)

Caregivers who participated in SDP subjectively reported overwhelmingly positive experiences that persisted well after discharge. One of SDP’s strengths consistently reported was that it helped to allay anxieties about taking care of a child with a tracheostomy at home, thus providing a smoother transition out of the hospital. This invaluable benefit alone speaks to the importance of the patient experience and the need for instituting and funding programs like this across all hospitals that provide care for children with tracheostomies and other critical, medical technologies. Despite this, caregiver participation in SDP did not yield a statistically significant difference for children with tracheostomies in incidence of ED visits within 1 year of discharge for tracheostomy-related complications compared with matched, control counterparts. It is difficult to encapsulate all the different benefits of simulation education, but a mixed-methods approach focused on both qualitative and quantitative data attempts to provide a more complete picture.

In previous needs assessments, caregivers reported a desire for tracheostomy-specific training to help address feelings of trepidation and uncertainty about managing their child at home; this feeling of fear and helplessness can continue even after significant time is spent in the home environment with some caregivers reporting development of physical and psychological overload in caring for their technology-dependent child.^9,10  According to the feedback we elicited, it appears that SDP helped to alleviate some of the psychological burden for many families in both the short-term and the long-term, even up to 4 years after participation in the program.

In 2020, the International Pediatric Otolaryngology Group surveyed the expert opinion of its members to provide guidance for home care of children with tracheostomies to avoid tracheostomy-related complications. Ultimately, a teaching protocol for caregivers that is agreed upon by all providers within an institution is deemed ideal; however, the authors note these protocols and assessment of caregiver competency are highly variable.¹¹ 

One such study that addresses this concern conducted by Baker et al concentrated on reducing hospitalization duration and cost through an organized quality improvement intervention.¹²  Although they focused on optimizing the discharge process, our study extends this concept by examining long-term caregiver confidence and preparedness in tracheostomy care, underscoring the significance of comprehensive caregiver education in complex pediatric care. Specifically, SDP tracks caregiver performance in tracheostomy care skills and evaluates their comfort. Our approach aligns with the need for consistent training protocols, but future iterations of SDP and similar programs could consider incorporating formal competency assessments. Based on our research, the strengths of SDP we would recommend incorporating in similar programs at other institutions are (1) basic information about tracheostomy care that caregivers can keep for reference, (2) an opportunity to physically practice new skills on a simulation manikin, and (3) peer-to-peer support to humanize and validate challenges experienced by families while instilling hope for the future.

Although much of simulation research in the past has been focused on learner acquisition of knowledge and skills, there is a recent push in the field to investigate “next level” impacts of simulation on patient care practices and clinical outcomes using a translational simulation framework.¹³  This study contributes to the field of translational simulation by linking the educational experience provided by SDP to tangible patient outcomes. Through our mixed-methods approach, we explored both the qualitative experiences of families participating in SDP and the quantitative impact on healthcare resource utilization.

We believe our mixed methods approach is a strength of our study, but we must acknowledge several limitations. The team's clinical background and experiences with tracheostomy care may have influenced our approach to focus on educational interventions. This is a single-center, small, convenience sample of participants. The SDP group is a convenience sample since assignment was not randomized; caregivers had to choose to participate in SDP, which may make them more likely to discuss the benefits of the program. In addition, we acknowledge the potential for recall bias, particularly as some families were asked to reflect on their experiences with SDP up to 4 years after participation. There may be a component of selection bias as only families reachable via EMR contact information participated in interviews. Although our qualitative study provided anecdotal accounts of tracheostomy-related events at home, we did not quantify these occurrences. We focused on interviewing families who participated in SDP, and we did not gather comparative feedback from families who did not participate in the program.

In the quantitative study, although we examined indications for tracheostomy placement, the use of the INSIGHT database, which relies on ICD-9/10 coding, restricted our ability to capture detailed information on every possible comorbidity. The INSIGHT database also does not capture the full spectrum of outpatient visits for our specific patient population. Not capturing the full breadth of data to include outpatient visits may underestimate hardships that families and patients face and overestimate the impact of our intervention. Further, we acknowledge the limitations of ICD9/10 coding in fully capturing the range of tracheostomy-related complications and treatments. Although we hoped to capture most children who presented to the ED in New York City through use of the INSIGHT database, we recognize discharged children may have sought emergency care outside of New York City. For the propensity score matching, we were limited to using hospital discharge disposition and age as the primary matching criteria because of the small sample size of our cohort. Additionally, our EMR data did not include detailed information on ventilator use at home or the presence of home nursing care, which are factors that could influence healthcare utilization. The time course of 1 year for assessing outcomes may have been too lengthy to detect differences between the groups that could exist in a shorter time frame. In our baseline or demographic evaluation, we were limited to using self-reported “ethnicity” data from our institutional EMR. We recognize the limitation of not having access to more comprehensive demographic information, such as race, socioeconomic status, and parental education levels, which could have provided a more holistic understanding of the variables impacting our outcomes. The antibiotics included in our analysis were selected based on their common usage in our patient population, guided by regional antibiogram and sensitivity patterns. We acknowledge that this list may not be exhaustive, potentially omitting other antibiotics used for tracheitis or pneumonia. Our study was also limited by the available data on initial discharge destinations, which did not include subsequent patient transitions from acute rehabilitation facilities to home or long-term care settings. Lastly, outcomes reported from single institutions may be influenced by center-specific tracheostomy care practices, which affects the generalizability of this study.

Simulation programs for caregivers of technology-dependent children have been consistently shown to improve caregiver-reported comfort and observed skills immediately after participation.^14,15  Our findings importantly highlight the substantial benefit that families perceived from participating in SDP, particularly in terms of increased confidence and preparedness in managing home care for children with tracheostomies. Feedback from caregivers indicated that the structure of SDP as 1 long session could be overwhelming. In light of this, we are considering modifications to the program, including offering the option of dividing the training into shorter sessions. Our study underscores the need for further prospective research with larger participant pools and more robust data collection to refine training programs like SDP and definitively evaluate their effects on clinical outcomes and healthcare system burdens. Importantly, these outcomes should be contextualized within the framework of family- and patient-centered benefits. For instance, our qualitative findings led to a reevaluation of the primary quantitative outcomes, revealing that caregivers often made informed and appropriate decisions to visit the ED based on knowledge gained from SDP. This highlights that although reducing ED visits is a traditional metric, the true impact of such programs extends to long-term enhancements in caregiver comfort and overall quality of life for both caregivers and patients. Therefore, future research must focus not only on quantitative outcomes but also on exploring these enduring qualitative benefits to fully appreciate the comprehensive value of caregiver training programs like SDP.

We thank the dedicated staff who continue to run SDP for our families caring for children with tracheostomies; and our partners on the Family Advisory Council who are strong advocates for SDP in its effort to support families.