Teaching the Skill of Shared Decision Making Utilizing a Novel Online Module: A Pilot Randomized Controlled Study Free

The study employed a randomized crossover trial of an online, interactive educational module to promote behaviors essential to SDM.¹⁹  The online curriculum was created with resources available at the Monroe Carell Jr Children’s Hospital utilizing a small institutional grant. Experts in clinical communication blinded to participants’ study arms evaluated for presence of SDM behaviors in video-taped clinical simulations with standardized parent dyads. The study protocol was approved by the Vanderbilt University Institutional Review Board.

Pediatric subspecialty fellows were defined as the target study population as they were likely to have previously received training as residents in the primary communication skills, which are necessary to implement SDM, while still being receptive to acquiring new communication skills as individuals seeking subspecialty training. Pediatric critical care medicine, neonatology, hematology and oncology, and pulmonology fellows at the Monroe Carell Jr Children’s Hospital at Vanderbilt were eligible to participate, if approved by their respective program directors. Thirty-three such individuals were identified. All were informed of the intent of the study to assess novel methods for improving behaviors associated with ideal shared decision making. No compensation was offered for participation other than access to the curriculum itself. Participants were randomized by specialty to control or intervention groups to ensure equal representation from each specialty in each group.

This module was developed with guidance provided by experts in online curriculum development at the Vanderbilt Peabody School of Education and the Vanderbilt Institute for Digital Learning. The goal to create a fully online, interactive, easily accessible module that could be completed from any computer with an internet connection in approximately 30 minutes over 1 or several settings. The module’s content included background information on SDM and conversational framing, along with recommended approaches, phrasings, and mnemonic devices to remember essential components of patient and family SDM interactions. The module design attempted to use expert consensus “best practice” regarding general format and user interface, a combination of media and text for presentation of the educational material, and interspersed multiple-choice questions to assess comprehension. The module concludes with a simulated conversation presenting a fictional “family meeting” focused on eliciting goals of care, assessing understanding, providing information and recommendations, and assessing response to recommendations given. A branching-logic model was used with multiple potential user responses offered at each step. Particular responses selected by the user generated different responses from the simulated family and module feedback on why certain responses were “best” and others considered less helpful. Users were then given the opportunity to reselect their responses and continue moving through the simulation.

To establish baseline skill level, all participants underwent an initial video-recorded simulated encounter with a standardized parent dyad made up of trained actors from the Vanderbilt Center for Experiential Learning and Assessment.

The clinical scenario centered on a 6 month infant with severe chronic lung disease and pulmonary hypertension unable to wean from mechanical ventilation whose parents faced a decision to either seek an invasive life-extending procedure for their child (tracheostomy) or transition their child’s care to a course aimed at comfort. Participants were given 45 minutes to complete the simulation and were given no feedback following its conclusion.

After completing the baseline simulation, participants were randomized in a 1:1 ratio to either an intervention group or a control group, with randomization stratified by specialty. The intervention group gained access to the online module focused on teaching the skill of SDM and conversational framing, whereas the control group received no additional education in SDM as part of the study. Participants from both control and intervention groups then underwent a second simulation approximately 4 months later where they were presented the same scenario as the baseline simulation. The control group was then crossed over to receive access to the online curriculum and subsequently underwent a third simulated family encounter again utilizing the same scenario approximately 1 month following their second simulation. At no point during the simulations did any of the participants receive feedback about their performance.

Following the completion of all 3 rounds of simulated conversations, video-taped interactions were evaluated by specialists in physician communication who possess PhDs in education, have published multiple peer-reviewed publications related to physician-patient-colleague communications, and who have previous experience rating or assessing health professionals’ communications and related skills (J.P. and L.W.). The evaluators were otherwise unrelated to the study. Participants completed their simulations in an average of 24 minutes across all sessions with none exceeding the time provided. Conversations were coded utilizing a validated tool to measure SDM behaviors by White et al.¹⁹  The tool was modified from 10 to 11 coded behaviors (with the additional behavior being the making of a recommendation), with all coded behaviors scored as yes or no variable for total score of 0 to 11 (Table 2). Videos were randomized and evenly divided between raters who were blinded to session date, participant medical specialty, and intervention status (control versus intervention versus crossover, pre or postintervention). A 20% sample of videos were coscored and evaluated using intraclass correlation to assess interrater reliability.

The primary assessment looked for difference in mean total scores between groups at the preintervention and the postintervention simulated encounters. Between group differences in individual behaviors were assessed utilizing the Pearson χ² test; between group difference in mean total score for each simulation was assessed via Wilcoxon rank sum test; within group difference in mean total score between simulations was assessed utilizing Wilcoxon signed rank test. Assessment for change in between-group mean total score from the first to second simulation was then performed via a proportional odds logistic regression model.

For the 20% of sessions with 2 sets of scores available from interrater reliability analysis, 1 value was picked at random for use in the logistic regression model. Raw scores are presented graphically as a spaghetti plot (Fig 1) and extended box plot (Fig 2). All analyses were performed using statistical software R version 3.3.0.

Thirty-three specialty fellows met eligibility criteria, with 17 participating and 16 completing the study (9 in the intervention group, 8 in the control or delayed intervention group, with 1 participant not completing the crossover phase). Those who chose not to participate received access to the intervention following completion of the study to be used for their own learning if desired. No participant had previously received formal education in regard to SDM per preparticipation screening. Information regarding control and intervention group characteristics including specialty, year of training, and gender appear in Table 1. Online course completion logs indicated that 9 members of the intervention group (100%) and (7) members of the delayed intervention group (87.5%) completed the intervention module. Those who completed the module without interruption did so in an average of 31 minutes. Although all participants were given access to the intervention weeks before their scheduled follow up simulation, all participants who completed the module did so within 48 hours of their scheduled simulation, with the exception of 1 participant who completed the module 6 days in advance.

Moderate interrater reliability was demonstrated with intraclass correlation of total scores calculated at 0.6. Primary comparison of average scores between groups showed significantly better performance by the intervention group compared with the control group for average total score in both the pre and post intervention simulations with the intervention group’s average score starting 1.83 points better than the control group (7.5, SD ± 0.93 vs 9.33, SD ±1.66, P = .025) in the preintervention simulation, increasing to a 1.9 point better average score (8.2, SD ± 1.0 vs 10.1, SD ± 1.1, P = .002) in the second simulation. The delayed-intervention group difference in mean total score increased 0.5 points (P = .12) between their 2 preintervention simulations, increasing another 2 points (P = .05) in their postintervention simulation. Within-group comparison showed control, intervention, and delayed-intervention groups improved their mean total scores between the first and second simulations: control group from 7.5, SD ± 0.93 to 8.2, SD ± 1.0 (P = .12), intervention group from 9.33, SD ± 1.66 to 10.1, SD ± 1.1 (P = .22), and delayed intervention group from 8.26 to 10.28 (P = .05), with only the delayed-intervention group’s change being statistically significant.

Between group comparison of individual behaviors showed significantly better performance in 2 behaviors for the intervention group during the preintervention simulation (behavior 6 - eliciting values and preferences, and behavior 7 - discussing the family’s role in decision making), which increased to better performance in 3 behaviors following the intervention (behavior 6 - eliciting values and preferences, behavior 8 – assessing or offering opportunity for input from others, and behavior 11 - eliciting the family’s opinion about the treatment recommendation made). Within-group comparison of individual behaviors for all 3 groups did not demonstrate a statistically significant change in any individual behavior.

Linear proportional odds regression analysis incorporating scores from all 3 groups determined that participants receiving the intervention were 39.78 times more likely to improve their subsequent total score when compared with simply repeating the simulation alone (95% confidence interval, 1.72–919.29; P = .022) as displayed in Figure 3. This analysis also indicated that having a higher baseline score was associated with a smaller increase in postintervention score, also displayed in Figure 3.

In this randomized trial, an innovative online interactive educational module was effective at increasing observed SDM behaviors in pediatric specialty fellows with odds of improvement following the intervention nearly 40 times greater than the odds of improvement by simply repeating the simulated scenario alone (P = .022). In addition to this demonstrated effectiveness lacking in other published SDM resources, this curriculum was designed to help overcome the other frequently identified barriers to the teaching of SDM, namely lack of interactive instruction or materials, costs, travel requirements, and time commitment of in person instruction. Further, the online nature of this intervention lends itself to repeated viewing on demand and allows for just- in-time training, which proved to be the manner in which the module was used by participants in this trial.

All trainees successfully demonstrated behaviors 1 through 5 in all simulated conversations, a difference from the results noted by White et al¹⁹  in which only 25% of physicians demonstrated behavior 5 (assess family understanding). It is notable that these behaviors make up the essential components of informed consent, and it may be that recent efforts and campaigns to improve physician-patient communication such as the “teach-back” method (advocated for by organizations such as the Agency for Healthcare Research and Quality since at least 2015)²⁰  have resulted in medical education curricula emphasizing these behaviors and behavior 5 in particular. All participants demonstrating behaviors 1 through 5 suggests future versions of this curriculum could be slimmed down to focus more on the behaviors not consistently demonstrated at baseline.

Higher overall baseline score noted in the intervention group could be attributed to the more senior average training level in that group, which was not accounted for in the randomization process of this pilot study. It is notable that the intervention group also skewed more female than the control group.

This pilot study was limited mainly by the number of participants, along with randomization that only accounted for training specialty, leaving uncontrolled for and unknown variables that were likely reflected in the difference in baseline scores between groups as noted above. Increasing the number of participants and controlling for additional variables such as gender, level of training, and subspecialty may help minimize the difference in baseline scores between groups. Additionally, although utilizing the same simulated scenario was necessary to control for additional variables and allow fair comparisons across groups, utilizing the same simulated scenario to make comparisons across groups may limit the external validity of the study results, including specifics of the scenario itself. Effect duration was not assessed in this study design given limited availability of participants to complete additional follow up simulations. Goals of future iterations of this study are to assess effect duration as well as to further compare effectiveness of the intervention between specialty groups and participants at different levels of training (postgraduate year).

This pilot study demonstrates that an online, interactive module can be an effective way of teaching the skill of SDM to pediatric subspecialty trainees. Tools like this one could help overcome the limitations inherent in currently published SDM resources and meet the need for interventions with demonstrated effectiveness, helping to increase the utilization of SDM skills by providing primary or supplemental education at institutions across the resource spectrum. Because of the positive nature with which this intervention was received by the participants, coupled with statistical analysis demonstrating its effectiveness, we plan to expand our trial across several institutions in the near future. This should allow an even more precise understanding of the effect of the module on trainees from different specialties and at different training levels.