To assess the performance of pediatric residents in recognizing a decompensating patient with impending respiratory failure and appropriately escalating care using a virtual reality (VR) simulated case of an infant with bronchiolitis after an extended period of decreased clinical volumes during the coronavirus disease 2019 (COVID-19) pandemic.
Sixty-two pediatric residents at a single academic pediatric referral center engaged in a 30-minute VR simulation on respiratory failure in a 3-month-old admitted to the pediatric hospital medicine service with bronchiolitis. This occurred in a socially distant manner across the Zoom platform during the COVID-19 pandemic (January–April 2021). Residents were assessed on their ability to (1) recognize altered mental status (AMS), (2) designate clinical status as “(impending) respiratory failure,” and (3) escalate care. Statistical differences between and across postgraduate year (PGY) levels were examined using χ2 or Fisher’s exact test, followed by pairwise comparison and posthoc multiple testing using the Hochberg test.
Among all residents, 53% successfully recognized AMS, 16% identified respiratory failure, and 23% escalated care. No significant differences were seen across PGY levels for recognizing AMS or identifying respiratory failure. PGY3+ residents were more likely to escalate care than PGY2 residents (P = .05).
In the setting of an extended period with decreased clinical volumes during the COVID-19 pandemic, pediatric residents across all PGY levels demonstrated challenges with identifying (impending) respiratory failure and appropriately escalating care during VR simulations. Though limited, VR simulation may serve as a safe adjunct for clinical training and assessment during times of decreased clinical exposure.
Pediatric residents learn to care for patients with respiratory distress and failure through repetition and bedside exposure.1 Typical conditions include bronchiolitis, pneumonia, asthma, and more recently severe acute respiratory syndrome coronavirus 2 (coronavirus disease 2019 [COVID-19]).2 Unfortunately, few tools exist to assess residents’ skill acquisition related to identifying respiratory failure and escalating care. Virtual reality (VR) represents a potential modality to assess residents’ skills through its realistic portrayal of clinical cases in a standardized and safe environment for both learners and patients.3,4 Our team has previously implemented VR training on recognition of impending respiratory failure with medical students and nurses and found that exposure to VR training improved medical student end of clerkship performance.5–7
During the initial phase of the COVID-19 pandemic, pediatric emergency departments experienced a 70% reduction in visits for respiratory disorders from March to August 2020.8 The decrease in clinical volumes led to concerns among supervising pediatric hospitalists regarding resident clinical exposure and its potential negative impact on achieving necessary clinical competency.9,10
The aims of this pilot study were to use a VR simulation to formatively assess pediatric residents’ skills at and provide training on identifying signs of impending respiratory failure and recognizing the need for care escalation after an extended period with decreased clinical volumes.
Setting and Study Population
This cross-sectional observational study used pediatric residents at a large free-standing academic children’s hospital. A convenience sample of residents spanning postgraduate years (PGY) 1 through 5 were recruited from the hospital-based continuity clinic. PGY4 and 5 residents were completing combined training in Pediatrics-Psychiatry-Child Psychiatry. Recruitment by the clinical research coordinator (A.M.) was via e-mail as well as in person during preclinic teaching sessions with a $50 incentive to participate. Consent was obtained at enrollment, and the study was approved by the local institutional review board.
Virtual Reality Simulation
VR sessions occurred from January to April 2021. The 30-minute simulation included (1) orientation to the VR patient and environment, (2) participation in a case of impending respiratory failure from bronchiolitis, (3) and debriefing the scenario with a facilitator. The curriculum used the framework of reflective practice to support adult learning through debriefings that emphasized reflection on action.11
The VR environment was constructed using Unity (Unity Technologies, San Francisco, CA). The 3-month-old VR patient with impending respiratory failure exhibited altered mental status (eyes closed, no vocalizations, no extremity movement); displayed signs of increased work of breathing (head bobbing; nasal flaring; suprasternal, intercostal, and subcostal retractions); and had abnormal vital signs (tachycardia, tachypnea) displayed on an in-room monitor. The VR content underwent extensive piloting by 4 critical care attending physicians, 2 hospital medicine attending physicians, 2 general pediatricians, 4 critical care fellows, and 4 pediatric chief residents with universal consensus that the virtual patient exhibited signs of impending respiratory failure and required transfer to the PICU.5,6
The facilitators (J.O., F.R.) operated the VR simulation using an Oculus Rift (Meta Technologies, Menlo Park, CA) headset and transmitted the virtual environment to participants via Zoom (Zoom Video Communications, Inc., San Jose, CA). Residents and medical students have previously indicated acceptability and adequate immersion with such screen-based VR interventions.12,13
At the start of the VR experience, residents were provided the case prompt (a “3-month-old male admitted to the hospital medicine service with bronchiolitis”) with basic details on the history of present illness and review of systems. Residents were instructed to share their thought process out loud as they evaluated the patient and were informed the simulation would conclude once they provided a formal assessment and plan.
The primary outcomes were descriptions of residents’ behaviors during the VR simulation using a binary scale of present versus not present for the following tasks: (1) recognized altered mental status (AMS) by stating that the patient was unresponsive or had AMS, (2) specifically verbalized concerns for “(impending) respiratory failure,” and (3) escalated care via calling for the institutional medical response team (MRT) or indicating the need for transfer to PICU. The facilitators scored 3 sample cases jointly to ensure consistent use of the assessment instrument.
As a secondary outcome, we categorized the management plans proposed by the resident to 1 of the following tiers: (1) provide supportive care (oxygen, suctioning), (2) perform additional work-up (chest radiograph, blood gas), (3) increase respiratory support without transferring to the PICU (low- or high-flow nasal cannula), or (4) MRT activation or transfer to the PICU. Residents received credit for the highest level of management proposed according to this hierarchy (Fig 1).
The number of inpatient hospitalizations for admission diagnosis International Classification of Diseases, 10th Revision codes corresponding to bronchiolitis between July 2018 and June 2021 were queried from the electronic health record to provide context for residents’ exposure to bronchiolitis during the study period (Fig 2).
Data were described as frequency counts and percentages. Statistical differences between and across PGY levels for each outcome were examined using χ2 or Fisher’s exact test as applicable, followed by pairwise comparison and posthoc multiple testing using the Hochberg test. All testing was done as 2-sided tests and P ≤ .05 was considered as statistically significant. All analyses were performed using SAS version 9.4 (SAS Institute Inc, Cary, NC).
Of 83 potential participants, 62 (75%) pediatric residents participated in this study (PGY1 34%, PGY2 31%, PGY3 32%, PGY4 2%, PGY5 2%) with balanced representation across PGY levels when PGY3–5 (PGY3+) were cohorted. Sixty residents (97%) had previous exposure to a VR curriculum that included verbal interaction with screen-based avatars.14,15
Among all residents, 53% (n = 33) successfully recognized AMS, 16% (n = 10) appropriately identified “(impending) respiratory failure,” and 23% (n = 14) arranged for an escalation of care by activating an MRT or indicating the patient needed to transfer to the PICU. There were no statistical differences between PGY levels in recognition of AMS (PGY1 38%, PGY2 47%, PGY3+ 73%, P = .06) or verbal identification of “(impending) respiratory failure” (PGY1 14%, PGY2 16%, PGY3+ 18%). PGY3+ residents were statistically more likely to escalate care than PGY2 residents (P = .05), but no other differences were observed between PGY levels (PGY1 24%, PGY2 5%, PGY3+ 36%). There were no statistically significant differences between PGY levels with regards to management plans when rated on a hierarchical scale (P = .11) (Table 1).
In the setting of decreased clinical volumes during the COVID-19 pandemic, pediatric residents across all PGY levels at our institution demonstrated limited ability to verbally identify “(impending) respiratory failure” and appropriately escalate care during VR simulations.
COVID-19 led to a reduction in pediatric respiratory admissions during the pandemic.16 Our institution witnessed a 16-month reduction in admissions for bronchiolitis from historical norms. We hypothesize our findings are potentially because of the pandemic-related decrease in clinical exposure to respiratory pathology that may have led to a delay or depreciation of skills. This is supported by the observation that PGY3+ residents, who had previous exposure to typical volumes of bronchiolitis in the prepandemic period, were statistically more likely to escalate care than PGY2 residents who had more limited exposure. Notably, there were no changes to the educational curriculum, clinical policies around escalation from the acute care setting to the PICU, or orientation or training exposures during the study period.
It is important to consider the limitations of this study. First, this pilot study was conducted with a convenience sample of residents at 1 institution, limiting generalizability. However, their clinical exposure is likely comparable to other large pediatric residency programs. Second, although we hypothesize the findings were because of a lack of clinical exposure, we do not have baseline data before the onset of the COVID-19 pandemic for comparison. Third, we deployed a strict assessment strategy in our primary outcomes that required residents to use precise language to receive credit. Although our findings may represent a lack of comfort with stating “(impending) respiratory failure,” the low rates of appropriate proposed management (ie, escalate care to the PICU) suggests that there is a true gap in recognition of these high-risk patients. Finally, it is possible that observed behaviors during VR simulations do not reflect real-world clinical practice. We used VR in response to social distancing guidelines, low patient volumes, and the necessity for scenario standardization. Moreover, as the majority of participants had prior exposure to VR curricula, we do not anticipate that unfamiliarity with the platform markedly impacted performance. The extensive piloting with experienced providers, improved performance for some metrics with increasing PGY year, and prior reports of efficacy also support the validity and appropriateness of the platform for formative assessment.5,6 Key next steps include (1) using this VR curriculum to assess the performance of pediatric residents exposed to normalized volumes of respiratory pathology, and (2) characterizing the performance among other acute care providers with varying experience levels using the same VR simulation to further validate this platform as an assessment tool.
An assessment of pediatric residents’ skills at identifying and managing respiratory failure after a period of reduced clinical volumes indicated room for improvement. VR allowed for skills to be assessed in a remote, socially distant manner, and allowed for just-in-time education on respiratory failure. However, further data are needed to understand the role of VR as an adjunct for clinical training and assessment.
Dr Odum conceptualized and designed the study, led data collection and interpretation, drafted the initial manuscript, and reviewed and revised the manuscript; Drs Real and Zackoff supervised the conceptualization and design of the study, supervised data collection, analysis, and interpretation, and critically reviewed and revised the manuscript; Dr Rice participated in data collection and interpretation; Ms Meisman recruited participants and managed the database; Drs Sahay and Zhang led data analysis and participated in interpretation; and all authors approved the final manuscript as submitted.
FUNDING: This work was supported through the Cincinnati Children’s Hospital Medical Center’s Procter Scholar Program.
CONFLICT OF INTEREST DISCLOSURES: The authors have indicated they have no potential conflicts of interest to disclose.