Physiologic Monitor Alarm Burden and Nurses’ Subjective Workload in a Children’s Hospital

This work was undertaken by the Agency for Healthcare Research and Quality–funded Patient Safety Learning Laboratory (PSLL) at a tertiary children’s hospital. A primary aim of the PSLL is to reengineer the system of monitoring hospitalized children within our hospital, with the goal to reduce noninformative alarms and accelerate nurse response to critical events. Two general (non–intensive care) pediatric inpatient units are participating in the PSLL’s longitudinal quality improvement work. Evaluation of the relationship between alarm count and nurse workload was part of the initial problem analysis phase of this improvement initiative.

The hospital committees for the protection of human subjects (the institutional review board) determined that the PSLL’s problem analysis portfolio (which included this project) was consistent with quality improvement activity and did not meet criteria for human subjects’ research.

Nurses on the 2 participating general pediatric inpatient units participated. All patient rooms include General Electric Dash monitors for physiologic monitoring. Default alarm thresholds for physiologic parameters are defined by age but can be adjusted by nurses to account for patient physiology. Monitor alarms are relayed to bedside nurses’ institutional mobile telephones (Ascom d62; Ascom Holding AG, Baar, Switzerland). On the units we studied, ∼40% of patients are continuously monitored.

We prospectively measured nurse subjective workload using the NASA-TLX (Fig 1) in a cohort of nurses whose workload we could assess repeatedly across a wide range of alarm exposure rates. This allowed us to use analytic methods designed to assess both within-nurse and between-nurse differences in workload at different alarm count exposure rates.¹¹  The NASA-TLX provides a scale to quantifying workload at the task or job level while accounting for individual differences.¹¹ 

We approached nurses from the 2 general pediatric units partnering with the PSLL for participation in workload evaluations. An in-person data collector obtained participating nurses’ demographic information and guided them through completing their first NASA-TLX assessments. Subsequently, nurses were asked to complete the NASA-TLX at least once per shift between July 8, 2019, and August 31, 2019, using personalized paper forms available in envelopes with their names on them in a bin on their unit. To measure workload at all hours and days of the week, we reviewed nursing assignments ∼1 week in advance and, on the basis of these advanced assignments, programmed a system to automatically deliver personalized text messages to each assigned nurse during their shift. The personalized text messages requested that the nurse complete a workload evaluation, reflecting on the past 2 hours of the shift. We aimed to obtain at least 10 evaluations per nurse to allow for adjustment based on within-nurse differences.

We collected data on all 6 dimensions of workload that compose the original NASA-TLX designed for the aerospace industry. However, in pediatric hospital settings like ours, the 4-dimension version of the NASA-TLX that is validated in pediatric nurses is more appropriate than the original version.¹⁰  This modified version of the NASA-TLX evaluates the following dimensions in the preceding 2 hours: mental demand, physical demand, temporal demand (ie, time pressure), and effort required.^6,10  In our analysis we used the 4-dimension instrument; we calculated the sum of scores in these 4 dimensions and scaled it to 100 to facilitate interpretation of the results and to inform subsequent analyses.⁶  We performed a sensitivity analysis using the original 6-dimension scale (scaled to 100).

In addition to nurse demographics, we collected information about the work environment and patient factors that potentially contribute to subjective workload, including unit, time of shift (day versus night), time within a shift the NASA-TLX was completed, nurse to patient ratio, patient acuity, the number of assigned patients receiving physiologic monitoring, and alarm exposure in the preceding 2 hours. Nurses recorded the time of NASA-TLX completion and listed the medical record numbers of all patients they were caring for at the time of NASA-TLX completion. Patient acuity was evaluated by using the Pediatric Rothman Index (PRI) score,¹⁸  a measure of projected probability of clinical deterioration computed on the basis of vital signs, laboratory test results, and nursing assessments for each patient. At the time of data collection, PRI scores were embedded into the electronic health record. PRI scores <65 denoted patients at risk for clinical deterioration or transfer to the ICU¹⁹ ; scores were dichotomized such that patients with PRI scores <65 were categorized as having high acuity and patients with PRI scores ≥65 were categorized as normal or lower acuity.

We extracted monitor alarm counts (the absolute number of abnormal vital signs that triggered a monitor to alarm) from the hospital’s clinical data warehouse. We paired NASA-TLX assessments with the total count of alarms for the nurses’ assigned patients during the preceding 2 hours (the period evaluated in the NASA-TLX). We also included alarms from patients whom nurses were only covering for part of the 2-hour window (eg, when covering another nurse’s patients because the other nurse is on a break).

Employing an approach used in our previous work evaluating alarm count and response time, we evaluated alarm counts categorically because we did not anticipate a linear relationship between alarms and workload.¹⁶  When alarms are infrequent, nurses are likely able to respond to them, whereas at high levels, we hypothesized alarms would impose additional workload burden. We visually examined a locally weighted scatterplot smoother to inform alarm count categorization. We divided alarm counts over the preceding 2 hours into 3 categories: 0 to 5 alarms to represent a low or typical alarm rate (50% of time periods), 6 to 39 alarms to represent an elevated alarm rate (50%–90%), and ≥40 alarms to represent a very high alarm rate (the top 10% in terms of alarm frequency).

To model this dynamic system in which numbers of repeat observations varied for each nurse, we used a hierarchical linear mixed-effects regression model that accounted for clustering of observations within nurses and clustering of nurses within units by including unit- and nurse-specific random effects. To select model variables, we examined the association of the NASA-TLX workload score (the primary outcome) with each of the nurse, patient, and work environment variables as fixed effects in bivariate analyses and performed a Wald test to evaluate the linear hypothesis. Variables with P ≤ .20 in these bivariate, unadjusted analyses were included in the subsequent multivariable model of scaled, composite workload scores. Because alarm counts and the number of monitored patients being cared for measured overlapping constructs, we included only alarm counts in our multivariable analysis. Informed by the literature and awareness of the complex relationships between staffing ratios and patient acuity from our previous work²⁰  and the work of others,⁶  we included a 2-way interaction between patient acuity and nurse to patient ratio in our model. We reported scaled, adjusted estimates of workload using predictive margins for each alarm count category.

We used Research Electronic Data Capture hosted by our institution for data management^21,22  and Stata version 16.1 (Stata Corp, College Station, TX) for statistical analysis.

We measured the subjective workload of 26 nurses 394 times using the NASA-TLX. After guiding nurses through completion of initial NASA-TLX evaluations in person (n = 39), we transitioned to delivering automated text messages to prompt NASA-TLX completion. We delivered 586 automated messages and subsequently collected 355 workload evaluations. Our response rate (based on responses to automated text messages) was 61%. In attempting to understand nonresponse, it became clear that the preshift assignments that we based our automated messaging strategy on were subject to change because of nurses being called off or calling out of a shift, reassignment of nurses to different units, or assignment of nurses to serve as a charge nurse. Nonetheless, participating nurses completed a median of 11 NASA-TLX evaluations (interquartile range [IQR]: 6–17; range: 1–35) during the study period. Nurses had a median 3 years of nursing experience (IQR: 2–5 years), and 6 nurses (23%) were in their first year of practice. Most nurses completed at least 1 workload evaluation during a nightshift (22 nurses; 85%), and 25 nurses (96%) evaluated a time period in which they cared for ≥1 patient with high acuity (defined as having a PRI score <65).

In terms of shift characteristics (Table 1), 46% of NASA-TLX evaluations were completed during day shifts, and 54% of evaluations were completed during night shifts. In 63% of assessed shifts, nurses cared for patients with high acuity. Nurses cared for a median 4 patients (IQR: 3–4, range: 2–5). Nurses cared for at least 1 patient receiving continuous monitoring in 74% of shifts. Nurses received a median of 6 alarms (IQR: 0–21) in the 2 hours before NASA-TLX completion. The median reported NASA-TLX workload score was 68.6 (IQR: 57.3–79.0).

In unadjusted analyses, all variables met the a priori criterion of P ≤ .2 for inclusion in the multivariable model (Table 2). In multivariable mixed-effects modeling, after we controlled for other contributors to workload, experiencing a very high alarm rate increased the workload scores by 5.5 points (95% confidence interval [CI]: 5.2 to 5.7). Nurses with 2 to 5 years’ experience, midshift (hours 3–7 of a 12-hour shift), increasing from 1 to 3 patients with low acuity to 4 to 5 patients with low acuity, and caring for at least 1 patient with high acuity (among nurses caring for 1–3 patients) were associated with increased workload. In Table 3, we modeled different conditions to illustrate the effect of alarms on nurse workload using predictive margins.

The sensitivity analysis using the 6-dimension NASA-TLX revealed similar results. In multivariable models using the 6-domain scale, high alarm count increased workload scores by 5.8 points (95% CI: 5.7 to 5.8). Although the coefficient values for the other covariates changed slightly, none changed in statistical significance or direction.

We sought to evaluate the relationship between alarm count and subjective workload using the NASA-TLX. When we accounted for other contributors to work load, high alarm count (≥40 alarms) was associated with a >5-point increase in subjective workload. The association between high alarm counts and workload is clinically significant; in terms of magnitude, it is similar to the increase in workload experienced when caring for 4 to 5 patients (as compared to caring for 1–3 patients). This finding is consistent with a simulation-based study that found that reduction in alarms resulted in a 5-point decrease in NASA-TLX workload scores.²³ 

Our findings support the negative impact of high alarm count on nurses’ provision of care. In work by Tubbs-Cooley et al,⁶  each 5-point increase in nurse subjective workload was associated with a 22% increased risk in self-reported missed nursing care. Both increased alarm count¹⁶  and subjective workload²³  have been associated with slowed alarm response time. Numerous other adverse outcomes are associated with high alarm burden, including poor nurse-patient communication,⁵  missed delivery of nursing care,⁶  hospital-acquired infection,⁴  and increased mortality.^1–3  More broadly, the inverse relationship between increased workload and task performance has been established in the human factors literature.^24,25 

Multiple factors influence workload, and thus (as highlighted in Table 3) the increased burden imposed by high alarm count may have different implications in different contexts. For example, nurses with 2 to 5 years of experience report high levels of subjective workload at low alarm counts, compared with nurses with >5 years of experience. The human factors literature suggests that professional experience may confer “attentional spare margin,” such that individuals with experience have additional “spare capacity” that reduces the workload associated with tasks.²⁶  Moreover “limited capacity” model posits that as individuals approach their capacity “threshold” (the point at which additional task demands cause performance to decline), their perceived workload increases.^27–30  These theories suggest that for nurses without spare capacity who are already experiencing high levels of workload, the addition of alarm-associated workload may have serious implications for patient safety. Of note, nurses with 2 to 5 years of experience report higher levels of subjective workload than nurses with <2 years of experience. A variety of factors may contribute to the lower subjective workload of new nurses, including reporting bias, additional supervision and support given to new nurses (generally by nurses with 2 to 5 years of experience), and/or reduced or lower complexity patient assignments. The impact of additional workload on nurse performance when workload is already high (ie, the nurse with 2–5 years’ experience) may differ from the performance impact of additional workload for a more experienced nurse whose baseline workload is lower. Similarly, there may be particularly vulnerable moments within a clinical shift when an interruption from an uninformative alarm is more burdensome. For instance, observational work has revealed the association of phone interruptions with medication administration errors.²⁰  Among the multitude of complex interacting factors that influence workload, alarm counts are modifiable. Reducing nonactionable alarms may be one of the most modifiable pragmatic targets for improving nurse workload.

To our knowledge this is the first study in which the relationship between alarm counts and subjective nursing workload is evaluated in clinical practice. Our reproducible approach allows for quantitative evaluation of the consequences of alarm fatigue. As part of our broader quality improvement work, workload will serve as a proximal outcome by which to evaluate future interventions focused on improving monitoring systems.

These findings must be contextualized within the limitations of our approach. We surveyed a limited number of nurses within a single pediatric hospital. Intuitional policies and practices may influence both the frequency and experience of alarms. We used automated text messages to request that nurses complete workload evaluations. Delivery of automated text messages was scheduled on the basis of preshift assignments. Although this approach allowed us to evaluate workload throughout shifts, we are unable to systematically evaluate why nurses did or did not complete workload evaluations. Some nonresponses were due to changes in staffing. However, because nurses were prompted to complete the NASA-TLX during their clinical shifts, at times, shift characteristics may have influenced completion. Busier nurses may have delayed or failed to complete evaluations of workload. Alternatively, nurses may have been eager to report workload during particularly challenging times. By obtaining frequent, repeated evaluations of workload on a routine basis, we aimed to overcome these sampling challenges. We adopted a human factors approach that accounted for nurse-, unit-, and patient-level factors; however, because nurses did not cross units, there may be unit-level factors we were unable to account for in our analysis. Finally, the literature regarding workload and expected workload of nurses is emerging. The original NASA-TLX measures 6 dimensions of workload; for both validity and interpretability within the existing nursing literature, we report scores from a 4-dimension version that has been validated among nurses. A sensitivity analysis using the 6-component NASA-TLX revealed similar results.

Increased physiologic monitor alarm exposure was associated with clinically and statistically significant increases in subjective nurse workload, which may lead to missed nursing care tasks and patient harm. Reducing nonactionable physiologic monitor alarm burden should be a high priority to improve patient safety.