Resident physicians are expected to recognize patients requiring escalation of care on day 1 of residency, as outlined by the Association of American Medical Colleges. Opportunities for medical students to assess patients at the bedside or through traditional simulation-based medical education have decreased because of coronavirus disease 2019 restrictions. Virtual reality (VR) delivered remotely via video teleconferencing may address this educational gap.
A prospective pilot study targeting third-year pediatric clerkship students at a large academic children’s hospital was conducted from April to December 2020. Groups of 6 to 15 students participated in a 1.5-hour video teleconferencing session with a physician facilitator donning a VR headset and screen sharing interactive VR cases of a hospitalized infant with respiratory distress. Students completed surveys assessing the immersion and tolerability of the virtual experience and reported its perceived effectiveness to traditional educational modalities. Comparisons were analyzed with binomial testing.
Participants included third-year medical students on their pediatric clerkship. A total of 140 students participated in the sessions, with 63% completing the survey. A majority of students reported VR captured their attention (78%) with minimal side effects. Students reported remote VR training as more effective (P < .001) than reading and online learning and equally or more effective (P < .001) than didactic teaching. Most students (80%) rated remote VR as less effective than bedside teaching.
This pilot reveals the feasibility of remote group clinical training with VR via a video conferencing platform, addressing a key experience gap while navigating coronavirus disease 2019 limitations on training.
Pediatric residents are expected to identify patients requiring urgent escalation of care on day 1 of residency. This expectation was codified by the Association of American Medical Colleges in their core entrustable professional activities.1 Prompt identification of impending respiratory failure in patients with bronchiolitis is one such clinical scenario requiring urgent escalation of care. Bronchiolitis is the most common cause of pediatric hospital admission during the first year of life, accounting for ∼100 000 admissions annually in the United States.2,3 Of these admissions, 1% to 12% eventually require critical care for respiratory failure.4 However, opportunities for medical students to assess patients with impending respiratory failure is limited given restricted learner autonomy,5,6 declines in bedside teaching because of increasing patient turnover and time constraints,7,8 and shortcomings of traditional simulation-based medical education strategies such as mannequins and standardized patients, which cannot easily replicate key examination findings such as altered mental status or work of breathing.9–11
The coronavirus disease 2019 (COVID-19) pandemic further limited education opportunities for students when the Association of American Medical Colleges designated students as nonessential health care workers in March 2020, recommending that training institutions prioritize student safety over clinical exposure.12 Although many students have returned to the wards, social distancing guidelines and personal protective equipment conservation efforts limit student clinical exposure. Furthermore, hospitalizations due to bronchiolitis have significantly decreased during the COVID-19 pandemic.13 Educators now face the difficult task of teaching clinical skills to the next generation of physicians away from the bedside, demonstrating a need for novel education techniques.
Virtual reality (VR), a form of simulation-based medical education, may play a key role in addressing this educational gap. Authors of a meta-analysis reviewing the use of virtual patients, including case-based modules and simulated clinical encounters, found a large pooled effect toward improvement in clinical reasoning and communication skills compared with traditional educational methods.14 We have previously described that single-user immersive VR training via a headset improved students’ recognition of respiratory distress and need for escalation of care.15,16 The emergence of COVID-19 and mandatory remote learning for students prompted exploration of the delivery of this VR curriculum via group video teleconferencing. We aimed to assess the acceptability and tolerability of remote VR education to pediatric clerkship students compared with traditional educational strategies.
Methods
Study Setting and Population
A prospective pilot study was conducted at a large academic, pediatric medical center, affiliated with a college of medicine, from April to December 2020. Third-year pediatric clerkship medical students were eligible to participate. This study was approved by the institutional review board.
Curriculum Design
Each 6-week clerkship included student participation in a scheduled education session via the remote teleconferencing platform Zoom (Zoom Video Communications, Inc, San Jose, CA). A 1.5-hour group VR session on respiratory distress was designed by using situated learning, a conceptual framework characterized by guidance from an instructor and active participation from learners to complete a task. The goal of situated learning is enhancing knowledge and skills to promote independent future practice.17 During each session, 6 to 15 students met virtually with a pediatric faculty instructor with extensive group facilitation experience. After a 30-minute didactic, the instructor donned an OculusRift (Facebook Technologies, LLC, Menlo Park, CA) headset and projected the headset view via the video teleconferencing screen share function using a VR-capable computer. The virtual environment replicated an inpatient hospital room with graphical character representatives (ie, realistic animations) of a patient and bedside nurse. The facilitator then delivered the same VR training experience as the previous single-user study15 : 3 cases created on and delivered through the Unity (Unity Technologies, San Francisco, CA) game engine demonstrating no distress, respiratory distress, and impending failure. The virtual patient portrayed vital signs, mental status, accessory muscle usage, and breath sounds audible to the learner that were reflective of the 3 clinical states. The virtual bedside nurse responded in real-time to students’ questions regarding the patient’s history. The students instructed the facilitator where to navigate within the virtual world (eg, lean toward the patient to assess accessory muscle use or turn head to view vital sign monitor) and shared their observed findings and assessment of the patient’s clinical status aloud, similar to a traditional bedside teaching experience.
Outcome Measures
After completion of the virtual training session, students voluntarily completed a survey. Demographic information was obtained. Attitudes regarding the level of immersion were assessed through a subset of questions from the Measurements, Effects, Conditions Spatial Presence Questionnaire that used a 5-point Likert scale from strongly disagree to strongly agree.18 Tolerability was assessed by measuring side effects related to participation.19 Lastly, students rated the perceived efficacy of remote VR patient cases compared with other learning experiences as less, equal, or more effective. Data were collected in a deidentified manner and stored in a Research Electronic Data Capture database, a secure Web-based application.20
Statistical Analysis
Descriptive and summary statistics were used for demographics, degree of immersion, and tolerability. Student attitudes comparing remote VR to other educational strategies were analyzed with binomial testing.
Results
Participant Demographics
A total of 140 students participated in the remote VR education sessions, with 63% completing the survey. (Table 1)
Student Demographics, N = 88
. | n (%) . |
---|---|
Age, y | |
18–24 | 36 (41) |
25–29 | 48 (55) |
30–34 | 4 (5) |
Sex | |
Male | 43 (49) |
Female | 45 (51) |
Ethnicity | |
Hispanic | 3 (3) |
Asian American | 9 (10) |
Black | 5 (6) |
White | 63 (72) |
Mixed | 7 (8) |
No answer | 1 (1) |
. | n (%) . |
---|---|
Age, y | |
18–24 | 36 (41) |
25–29 | 48 (55) |
30–34 | 4 (5) |
Sex | |
Male | 43 (49) |
Female | 45 (51) |
Ethnicity | |
Hispanic | 3 (3) |
Asian American | 9 (10) |
Black | 5 (6) |
White | 63 (72) |
Mixed | 7 (8) |
No answer | 1 (1) |
Assessment of the Virtual Environment
A majority of students agreed or strongly agreed that the remote VR experience captured their senses (78%), and they were completely dedicated to the virtual experience (70%), were able to make a good estimate of the size of the presented space (77%), and retained a concrete mental image of the environment (83%). A plurality of students agreed or strongly agreed that they felt physically present in the environment (40%) and took part in the actions of (43%) and could do things with the objects in (43%) the simulation. Reported side effects from the VR experience were minimal (Table 2).
Student-Reported VR Side Effects, N = 87
. | Yes, n (%) . | No, n (%) . |
---|---|---|
Blurred vision | 1 (1) | 86 (99) |
Burping | 1 (1) | 86 (99) |
Difficulty concentrating | 5 (6) | 82 (94) |
Disorientation | 3 (3) | 84 (97) |
Dizziness | 5 (6) | 82 (94) |
Drowsiness | 0 (0) | 87 (100) |
Eye Strain | 5 (6) | 82 (94) |
Fatigue | 1 (1) | 86 (99) |
General discomfort | 1 (1) | 86 (99) |
Headache | 1 (1) | 86 (99) |
Nausea | 0 (0) | 87 (100) |
Salivation | 0 (0) | 87 (100) |
Stomach awareness | 0 (0) | 87 (100) |
Sweating | 0 (0) | 87 (100) |
. | Yes, n (%) . | No, n (%) . |
---|---|---|
Blurred vision | 1 (1) | 86 (99) |
Burping | 1 (1) | 86 (99) |
Difficulty concentrating | 5 (6) | 82 (94) |
Disorientation | 3 (3) | 84 (97) |
Dizziness | 5 (6) | 82 (94) |
Drowsiness | 0 (0) | 87 (100) |
Eye Strain | 5 (6) | 82 (94) |
Fatigue | 1 (1) | 86 (99) |
General discomfort | 1 (1) | 86 (99) |
Headache | 1 (1) | 86 (99) |
Nausea | 0 (0) | 87 (100) |
Salivation | 0 (0) | 87 (100) |
Stomach awareness | 0 (0) | 87 (100) |
Sweating | 0 (0) | 87 (100) |
1 participant did not submit answers.
Attitudes Toward Virtual Training
Students reported remote VR training as more effective than reading (response rate: 80% [95% confidence interval: 71%–88%], P < .001) and online learning (78% [68%–86%], P < .001). Remote VR training was rated as equally or more effective than traditional didactic teaching (91% [83%–96%], P < .001). Most students (80%) rated remote VR as less effective than bedside teaching (Fig 1).
Binomial testing demonstrates that a statistical majority of students found remote VR training more effective than reading and online learning and equally or more effective than didactic teaching and low fidelity mannequins. aVR training rated as more effective than comparison modality (P< .001). bVR training rated as equally or more effective than comparison modality (P < .001)
Binomial testing demonstrates that a statistical majority of students found remote VR training more effective than reading and online learning and equally or more effective than didactic teaching and low fidelity mannequins. aVR training rated as more effective than comparison modality (P< .001). bVR training rated as equally or more effective than comparison modality (P < .001)
Discussion
This study is novel in the use of VR to facilitate clinical training in a large group setting through video teleconferencing. A majority of students reported remote VR captured their attention with minimal distractions and was tolerable. Students rated remote VR as more effective than reading and online learning and equally or more effective than traditional didactics, the primary options for distance-learning available to students during the COVID-19 pandemic.
Our study was designed to address core entrustable professional activities no. 10, identifying a patient who requires urgent escalation of care, such as a patient with impending respiratory failure.1 Clerkship students who were removed from the bedside because of pandemic restrictions will soon serve as frontline providers. Our VR curriculum allowed students to practice assessing a patient with impending respiratory failure with real-time feedback, a training experience that cannot easily be replicated on the wards, especially during the COVID-19 pandemic era. Our remote VR curriculum offers a unique method to engage with students at the “virtual bedside,” allowing exposure to patients less commonly encountered in the hospitalized setting during the COVID-19 pandemic (eg, bronchiolitis).13
However, group video teleconferencing does not fully reap the benefits previously reported with immersive VR. A previous cohort of third-year medical students who experienced a single-user immersive VR curriculum rated it as equally or more effective than standardized patients and high-fidelity mannequins, which was not seen with remote VR.21 Similarly, students reported a greater degree of immersion (ie, feeling physically present) in the single-user VR experience (85%) than with remote VR (40%).21 These findings suggest remote and immersive VR may be unequal in terms of perceived effectiveness. Educators should consider the best virtual modality to pursue on the basis of the learning objectives, required degree of immersion, and feasibility of implementation.
This study has several limitations. It was conducted at a single site, with only third-year medical students, and was focused on students’ perceptions toward the effectiveness of remote VR training. Additionally, we are unable to verify the students’ previous experience with the comparator education modalities. However, students were instructed to select “not applicable” for a comparator if they had never experienced it, attempting to minimize the impact of variable previous experiences. With this pilot study, we were unable to capture the impact of remote VR on clinical performance. Formally assessing knowledge application represents a key next step on the path to establishing a clinical performance impact of remote VR. Lastly, the generalizability of our approach is currently limited given the absence of universal access to VR technology or curricula. However, access to VR is becoming more feasible with declining equipment costs22 and increasing availability of content sharing.
Conclusion
The COVID-19 pandemic has revealed the need for innovative educational methods to reach distanced learners and teach key clinical skills, such as assessment of respiratory distress. VR patient cases, delivered via remote video teleconferencing, are a potential strategy. Importantly, the majority of students in our study and in previous work rate bedside teaching as most effective.21 This highlights the difficulty in creating realistic patient encounters virtually and that nothing can fully replace the power of precepting a student in a patient’s room. However, remote VR may serve as a supplement for bedside teaching. As social distancing guidelines are lifted and medical students ward exposure normalizes, educators should redouble their efforts to seek opportunities to engage students at the bedside.7,8
Acknowledgments
We thank the medical students from the University of Cincinnati College of Medicine for their participation in this curriculum.
FUNDING: No external funding.
Dr Young conceptualized and designed the study, participated in data collection sessions, assisted with data analysis, and drafted the initial manuscript; Dr Real conceptualized and designed the study, coordinated and facilitated all data collection sessions, and reviewed and revised the manuscript; Dr Sahay performed the data analysis, and reviewed and revised the manuscript; Dr Zackoff conceptualized and designed the virtual reality cases used in this study, conceptualized and designed the study, assisted with data analysis, and reviewed and revised the manuscript; and all authors approved the final manuscript as submitted.
References
Competing Interests
POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.
Comments