Social Robots for Hospitalized Children

This is a feasibility and acceptability trial with preliminary efficacy data, set on pediatric medical and surgical floors, in which a convenience sample of patients was used. Our multidisciplinary design and evaluation team included clinicians (critical care physician, CLS), psychologists (hospital-based clinical and experimental), roboticists, and engineers. We obtained input from patients, families, and medical unit staff (eg, nurses) who were introduced to the robot in hospital room visits and clinical team meetings. The hospital’s institutional review board approved the study, and parents provided written informed consent for children’s participation.

We enrolled 54 medically or surgically hospitalized children ages 3 to 10 years (18 in robot, 17 in avatar, 19 in plush conditions). Inclusion criteria included being English-speaking and admitted to one of several participating general and hematology-oncology floors for >48 hours of hospitalization. We excluded patients with pacemakers (to avoid potential wireless interference) or significant developmental delays and those admitted to nonparticipating floors. Clinical staff preapproved patient participation on the basis of current clinical status (patient awake and alert, not in medical or emotional crisis, etc).

We piloted a between-groups, randomized open trial with 3 conditions: (1) a tele-operated Huggable robot in full physical form (“robot”); (2) a tablet version of a tele-operated Huggable, providing an interactive comparison condition without physical presence (“avatar”); and (3) a static plush teddy bear presented by a CLS to provide a physically comforting but noninteractive comparison (“plush”). To counterbalance conditions demographically, block random assignment was applied. We assigned children ages 3 to 5 years to the younger-age block and children ages 6 to 10 years to the older-age block. Age groups were determined a priori to align with preschool-aged and school-aged developmental stages. Within each age-by-sex block, children were randomly assigned to intervention condition. In Fig 1, we depict study flow.

For this stage of development, Huggable (both forms) was tele-operated through “Wizard of Oz” methodology,¹³ in which a human (CLS staff in this case) tele-operated the robot or avatar. Wizard of Oz is a standard approach in the human-robot interaction field to inform the creation of autonomous SRs. Remotely controlling the robot or avatar in a real-world setting allows researchers to collect data and observe behaviors to identify important usage opportunities and necessary capabilities. The Huggable tele-operator could trigger facial expressions and body actions, talk through Huggable in a childlike, pitch-shifted voice, and see and hear participants and their surroundings via camera feed.

A CLS was in the patient’s room during interventions to facilitate interactions. The avatar and robot Huggable engaged participants by conversing about their likes and dislikes, singing nursery rhymes, and playing an I spy game. The plush was presented and “puppeteered” by CLSs, interacting as they typically would clinically, thus providing a human element to this comparison condition. Patients, parents, and any medical staff present were instructed to act as they typically would, to test the intervention in a natural setting, allowing care routines to unfold as needed. Interventions ended after 30 minutes or sooner if participants appeared ready to end the encounter (eg, because of fatigue). Interventions were videotaped and subsequently transcribed and coded by trained raters to evaluate affect, speech, and engagement.

The robot had a plush bear exterior and used an Android smartphone as its primary computational unit, with the screen depicting digitally animated eyes. The virtual Huggable avatar ran on an Android tablet with identical degrees of freedom and animations. All equipment, including Huggable’s fur, was cleaned between each intervention in accordance with hospital infection control standards.

Additional study equipment included the Q Sensor (Affectiva, Inc) wireless wrist-worn electrodermal activity (EDA) sensor to passively and continuously measure peripheral sympathetic nervous system arousal via skin conductance, skin surface temperature, and motor movements via 3-axis accelerometry.

Video capture was initially achieved through 2 wired Logitech C920 cameras positioned for a wide-angle view. A third camera was mounted underneath the bedside table to capture the patient’s face during interactions. Over time, we transitioned to battery-operated GoPro Hero 3+ cameras that enabled easier setup, wireless monitoring, faster troubleshooting, higher-quality image capture, and easier footage recovery.

Indicators of feasibility included (1) tracking of enrollment and completion rates and (2) analysis of procedural difficulties that emerged during intervention implementation. Acceptability indicators included (1) whether participants found playing with Huggable fun and whether they would want to play with Huggable again (yes or no) and (2) open-ended questions administered to CLS project staff, whose responses were qualitatively analyzed via content analysis to elicit common themes related to perceived benefits and risks of the intervention and suggestions for process improvement. Specific cutoffs for these indicators were not stated a priori; rather, we sought to evaluate overall feasibility and acceptability across multiple indicators.

Patient and parent report questionnaires included in the study were used to assess perceived emotional state and current pain intensity. Measures were collected before and immediately after intervention exposure.

The Facial Affective Scale consists of 9 faces varying by level of overt distress.¹⁴ Children are instructed to identify the face that “looks like how you feel inside.”¹⁴ We used a modified scoring transformation, scoring faces from 1 to 9, with lower numbers indicating more positive affect. Parent proxy ratings were also obtained.

Positive and Negative Affect Scales for Children (PANAS-C), Brief version¹⁵ is a 10-item self-report measure of child affect. Children rate the degree to which they currently feel emotions (eg, “nervous”) on a 1 to 5 scale. Higher numbers indicate more emotion for each subscale, Positive and Negative. Reliability and validity data for the measure support its use in school-aged children.¹⁵ We administered the Brief PANAS-C to children ages ≥6 years.

State-Trait Anxiety Inventory (STAI) and State-Trait Anxiety Inventory for Children (STAIC)¹⁶ measure general proneness to anxiety (trait anxiety) and anxiety as a transient emotional state (state anxiety). We used the STAIC scale to assess changes in anxiety with the intervention. The 20 items are rated on a 1 to 4 scale; higher scores indicate more anxiety. Parents completed the STAI trait measure (pre-intervention only) to be included as a covariate. Validity and reliability of both versions are well established and reported in the instrument manuals. The STAIC was originally validated on children ages ≥8 years but has been used in past studies with younger children.^17,18 

To rate pain intensity, younger children used the Faces Pain Rating Scale Revised,¹⁹ and older children used a numeric rating scale (NRS). In both scales, a metric of 0 to 10 is employed for rating pain intensity, wherein higher scores indicate higher pain. Both measures are valid and reliable in children.^19,20 

Feasibility data were summarized by using descriptive analyses of recruitment and study completion processes along with qualitative thematic analysis of CLS interviews. We used repeated measures analysis of variance techniques (including covariates when appropriate) to analyze self-report data. To analyze participants’ verbal utterances from video coding, we transcribed and coded footage using a Health Insurance Portability and Accountability Act–compliant vendor. Sentiments for total (all individuals in the room) and patient-only utterances were analyzed with IBM Watson’s Tone Analyzer,²¹ a computerized cognitive-linguistic analysis tool assessing emotional tones (anger, fear, joy, sadness, and disgust) in written text. Analyses were focused on scoring average joy, agreeableness, and sadness. Joy and sadness scores were calculated for total utterances. Agreeableness scores were calculated for patient-only utterances. When analyzing agreeableness, 18 participants (all hematology-oncology patients) were removed for failing to produce enough utterances (minimum 100) for reliable analysis. We also analyzed intervention duration and extent of patients’ physical movement.²² Generalized linear modeling approaches were used for analyses, with predictor variables contrast coded as ordered values [−1 0 1], for robot, avatar, and plush.

EDA recorded via Q Sensor was used to measure physiologic and emotional arousal. Engagement levels during the last third of the interaction period were annotated for comparison with children’s EDA responses, wherein the rate and detrended SD of nonspecific skin conductance responses were analyzed to indicate arousal levels.

In Tables 1 and 2, we present information on study feasibility, including participation rates and reasons for missing data. In Fig 2, we show length of intervention by condition (robot: mean = 26.4 minutes, SD = 17.0; avatar: mean = 21.7, SD = 11.4; plush: mean = 7.8, SD = 5.1).

Initial feasibility challenges we identified included minimizing clinical workflow impact and overcoming clinical staff reservations. Equipment had to be installed and removed for every experimental session without disturbing families. We installed most devices and wires on a mobile cart to minimize in-room setup time. However, this had the negative side effect of sometimes compromising video capture because of constraints of the physical environment and/or patient movement. Various wireless devices and lead-lined walls of some units created interference in the heavily used hospital wireless signal, causing periodic but significant delays or malfunctions in Huggable operation. To allay staff concerns, we avoided scheduling interactions at times when clinicians needed patient access (eg, rounds procedures).

Regarding acceptability, 93% of child participants endorsed that meeting the SR was “fun,” and 92% expressed a desire to play with the SR again. Numbers were too small for statistical analysis by group, but acceptability for the SR was similar to the tablet (94% for fun, 92% for playing again) but greater than the plush (75% for fun, 63% for playing again). Content analysis of qualitative CLS interviews highlighted the rewards to hospitalized children that emerged from the introduction of an SR into the hospital setting and elucidated areas for future development. Several themes arose from these interviews to inform our subsequent efforts. Prevalent themes are summarized in Table 3.

In Table 4, we list descriptive results from self-report questionnaires, and in Table 5 we summarize participant diagnoses and reasons for hospitalization, included as covariates where relevant. Data conformed to assumptions of normality and were found appropriate for parametric statistical tests. Overall, children reported statistically significantly greater positive affect (preintervention Brief PANAS-C–positive mean = 3.48, postintervention mean = 3.89; P < .01), and parents reported lower perceptions of children’s pain (preintervention NRS mean = 1.71, postintervention mean = 0.81; P < .01) and more positive child affect (Facial Affective Scale preintervention mean = 3.12, postintervention mean = 2.58; P < .05) after the intervention, regardless of condition. There was a statistically significant group effect on positive affect, with children exposed to the robot reporting greater positive affect relative to those in the plush condition (robot group postintervention Brief PANAS-C–positive mean = 4.57, plush group mean = 3.39; P < .05). No other statistically significant group differences were found on self-report.

We examined group differences in behavioral responses. Total joyfulness utterances showed a statistically significant increase across the 3 conditions (robot > avatar > plush; P = .003). Agreeableness utterances showed a statistically significant increase (robot > avatar > plush; P = .001). Sadness utterances showed a statistically significant decrease across conditions (robot < avatar < plush; P = .026). (See Figs 3–5.)

There was a reduction in participants’ physiologic arousal over time across all groups (average video-coded engagement in final third of the interaction period correlated negatively with skin conductance [EDA] response rate; r = −0.446, P = .037). Because of missing data, cell sizes were too small for valid comparisons by group.

In the context of shared desire among chronically ill children, families, and providers to leverage technology for more personalized pediatric care, we sought in this study to examine feasibility and acceptability of SR technology in the inpatient setting. Traditional methods for addressing emotional aspects of pediatric hospitalization typically have shown positive but short-lived and small effects.^23,–26 Our innovative partnership between pediatric clinicians and engineering and technology experts sought to explore whether advanced SR technology can expand the emotional comfort we can offer hospitalized children.

The solid participation rate (79%) in the study demonstrates feasible enrollment and patient and family enthusiasm for the technology. Rates of completed data, although excellent overall (93%), highlight several challenges, such as discomfort with the Q Sensor, inability to complete postintervention questionnaires, or insufficient verbal output for speech analysis. Because this is the first known study in which SR technology has been explored in an inpatient setting with ill children, there are no comparable studies against which to evaluate our success. Previous work with SR technology and children outside the laboratory have included small numbers,²⁷ were designed as observational studies in public settings,²⁸ or included few or no study-specific requirements, such as questionnaires or additional monitoring procedures.¹¹ Our rate of incompletion due to technical failures appears similar to or better than previous published studies,²⁹ which is particularly promising given the added logistic challenges of the inpatient hospital setting. Patients found the SR engaging and expressed a desire to play with it again, demonstrating acceptability. In-depth interviews with CLSs highlight that CLSs valued the opportunities this technology provides to connect with patients in new ways. They also underscore the role of individual differences that likely make each child’s and family’s interaction with the SR unique, an important area to attend to in future research.

Through our team’s combined expertise, we implemented some creative problem-solving to address the feasibility of integrating SR technology into the clinical environment. Reflecting on this process may be useful for others considering introducing SRs in similar settings. Examples included tethering devices directly to a router and using a high-quality wireless access point that boosted and directed signals to overcome the demands on wireless signal and the disruptions to transmission due to environmental features, such as lead-lined walls. Although staff were at times wary of this new technology, we found ways to increase buy in. Unfortunately, avoiding times when clinicians required patient access limited our ability to assess the effects of the SR on patient response to stressful procedures in vivo. We intend to explore this in future work. Fortunately, as clinicians observed patients and families having positive interactions, they appeared to grow more supportive of SR presence.

Not surprisingly, the medical issues facing our sample required adaptations to approaches developed in the Massachusetts Institute of Technology team’s previous experiences with children.^30,–32 Fatigue may have contributed to brevity of some interventions, influenced engagement levels and speech output, or limited completion of postintervention self-reports. Because we focused on capturing immediate affective responses in our pilot, we could not defer data collection. We continue to explore optimal outcome measures and postintervention assessment timing. The team also had to be sensitive to issues with tolerating monitoring devices such as the wrist-worn EDA biosensor. Despite our team’s experience using these technologies successfully in healthy children, we recognized that children in the inpatient setting must tolerate many monitors and invasive procedures. Because hospitalized children have limited control over what is happening to them,^33,34 at times we chose to sacrifice research data to avoid children feeling coerced or experiencing added discomfort.

Overall, preliminary findings suggest that hospitalized children benefit from SR technology, as evidenced by increases in reported positive affect after exposure to the robot relative to the other interventions, along with greater expressed joyfulness and agreeableness among children in the robot condition. Decreased EDA reactivity across groups toward the end of interventions tentatively indicates that “quality” of engagement or attention often waned over time. Because of missing EDA data points, sample size was not sufficient to reliably investigate this effect on a group level, requiring further study to determine if an SR could foster more sustained engagement. Recent advances in biometric data collection approaches, such as the use of laboratory-grade ambulatory monitors,³⁵ may help in this regard. Results augment our previously published findings that children who interact with a robot show greater increase in physical movement over the course of an intervention and have longer verbal exchanges relative to other conditions.²² These emotional, physical, and verbal outcomes are all positive factors that could contribute toward better and faster recovery in hospitalized children.^36,37 

All technologies have different affordances and strengths. For instance, a wearable device is suitable for seamless measurement of physiologic data without affecting users’ attention or behavior, whereas screen-based devices offer the interface to display various types of visual information. According to our results, as well as over 30 published studies,³⁸ physically present and embodied robots engage humans socioemotionally better than virtual avatars. This makes SRs an interesting and relevant technology in the pediatric care context in comparison with other devices. Recent neuroscientific findings reveal that interpersonal interaction with a trusted ally can mitigate perceptions of physical pain and increase comfort.³⁹ At Boston Children’s Hospital and many institutions, a multidisciplinary team provides emotional support and pain management, with CLSs serving a prominent role. However, CLSs cannot be with every patient continuously. Tools such as SRs can increase reach and ultimately provide continuous monitoring and assessment of children’s variable emotional states throughout the hospital experience to enhance comfort and care.

Our feasibility and preliminary outcome data and insights will inform development of a fully autonomous SR, over a projected time frame of 2 to 3 years, to scale the application of SR technology that augments human capacity and enhances the positive emotional experience of patients. Although developing emotional intelligence that can fully understand how people feel and behave is a long-term goal, interactive technologies and devices are starting to be widely used by the general population, with some already targeting health care application (eg, KidsMD on Amazon Echo and Mabu [http://www.cataliahealth.com]). Thus, we believe it is feasible to build a fully autonomous SR for pediatric patients within this time frame. In future studies, our technical focus shall be to improve our affective computing algorithms to better characterize and monitor children’s affective state, coupled with computational methods for the SR to learn how to effectively support and engage each child through dialogue and playful activities, in concert with clinical staff.

It is important to note the limitations of this pilot study. Within the real-world clinical setting, a between-subjects design was an important first step, but it failed to control for individual patient variability or to expose all patients to each intervention. In future within-subjects studies, we can explore whether SRs are consistently more effective than nontechnical interventions for all hospitalized children, under specific circumstances (eg, times of greater fatigue), or for patient subgroups (eg, age, health condition, etc). Of note, our sample was weighted toward oncology patients; in future studies, researchers should assess the use of SRs in large groups of patients with diverse medical conditions. CLS facilitators were not blinded in this design, potentially biasing interactions. Our small sample size limited power and generalizability, and factors such as current physical state may have influenced patient responses. Finally, opportunities exist to further expand our understanding of the impact of SR technology through in-depth interviews with families, as we have shown in previous work.⁴⁰ 

Despite these limitations, our experiences underscore significant opportunities that exist to further develop an engaging and therapeutically valuable SR platform for children with chronic physical illnesses. Our hope is for rigorous development and validation of SR technology in pediatrics to result in scalable, cost-effective tools to reduce length of stay and recidivism. More importantly, this technology can offer entirely new paradigms to optimize emotional experiences, outcomes, and quality of life for children and families as they navigate the health care journey.

We thank the research team, including Alex Ahmed, Laurel Anderson, Melissa Burke, Jimmy Day (video production), Pauline Dimaano, Katie Fitzpatrick, Suzanne Graca, Honey Goodenough, Katherine Jamieson, Brianna Jehl, Nicole Stenquist, Taylor Turrisi, and Tessa Wihak for contributions to this work.

CLS

child life specialist

EDA

electrodermal activity

NRS

numeric rating scale

PANAS-C

Positive and Negative Affect Scales for Children

SR

social robot

STAI

State-Trait Anxiety Inventory

STAIC

State-Trait Anxiety Inventory for Children