BACKGROUND AND OBJECTIVES

Physiologic monitor alarms occur at high rates in children’s hospitals; ≤1% are actionable. The burden of alarms has implications for patient safety and is challenging to measure directly. Nurse workload, measured by using a version of the National Aeronautics and Space Administration Task Load Index (NASA-TLX) validated among nurses, is a useful indicator of work burden that has been associated with patient outcomes. A recent study revealed that 5-point increases in the NASA-TLX score were associated with a 22% increased risk in missed nursing care. Our objective was to measure the relationship between alarm count and nurse workload by using the NASA-TLX.

METHODS

We conducted a repeated cross-sectional study of pediatric nurses in a tertiary care children’s hospital to measure the association between NASA-TLX workload evaluations (using the nurse-validated scale) and alarm count in the 2 hours preceding NASA-TLX administration. Using a multivariable mixed-effects regression accounting for nurse-level clustering, we modeled the adjusted association of alarm count with workload.

RESULTS

The NASA-TLX score was assessed in 26 nurses during 394 nursing shifts over a 2-month period. In adjusted regression models, experiencing >40 alarms in the preceding 2 hours was associated with a 5.5 point increase (95% confidence interval 5.2 to 5.7; P < .001) in subjective workload.

CONCLUSION

Alarm count in the preceding 2 hours is associated with a significant increase in subjective nurse workload that exceeds the threshold associated with increased risk of missed nursing care and potential patient harm.

High nurse workload has been associated with adverse patient outcomes, including increased mortality,13  hospital-acquired harm,4  nurse-patient miscommunication,5  and missed delivery of nursing care.6  Workload is generally conceptualized as the ratio of physical and cognitive demands to an individual’s available resources,7,8  and nursing workload has been understood as the “performance required to carry out nursing activities.”9,10  Originally developed for the aerospace industry, the National Aeronautics and Space Administration Task Load Index (NASA-TLX) is a validated measure of subjective workload that is sensitive to individual differences used to assess work demands.11,12  The NASA-TLX has been applied to nursing,6,7,10,13  and a shortened 4-dimension scale composed of mental demand, physical demand, temporal demand, and effort has been validated to specifically measure nurse-specific workload.12  Recent work revealed that each 5-point increase in nurse workload on the nurse-specific 4-dimension NASA-TLX was associated with a 22% increase in the likelihood of self-reported omission of patient care tasks, such as double-checking high-risk medications and adhering to infection prevention bundles during the same shift.6 

Physiologic monitor alarms may contribute to nurse subjective workload. Physiologic monitoring of vital sign parameters (including heart rate, respiratory rate, and oxygen saturation) is commonplace on pediatric units, with estimates of up to 48% of non-ICU pediatric patients receiving continuous monitoring of vital signs.14  Abnormal vital signs trigger an alarm in the patient room and, in many institutions, relay a message to the bedside nurse’s institutional mobile telephone. Pediatric nurses are responsible for responding to up to 155 alarms per monitored patient per day.14,15  Analysis of alarms indicates that only ∼0.5% to 1% of alarms are considered actionable or informative on pediatric wards.16,17  High alarm count is associated with slower nurse response time to alarms,16  but the broader association between alarms and nurse workload has not been characterized in clinical practice. Our objective for this project was to directly measure the relationship between alarm count and subjective nurse workload by using the NASA-TLX.

This work was undertaken by the Agency for Healthcare Research and Quality–funded Patient Safety Learning Laboratory (PSLL) at a tertiary children’s hospital. A primary aim of the PSLL is to reengineer the system of monitoring hospitalized children within our hospital, with the goal to reduce noninformative alarms and accelerate nurse response to critical events. Two general (non–intensive care) pediatric inpatient units are participating in the PSLL’s longitudinal quality improvement work. Evaluation of the relationship between alarm count and nurse workload was part of the initial problem analysis phase of this improvement initiative.

The hospital committees for the protection of human subjects (the institutional review board) determined that the PSLL’s problem analysis portfolio (which included this project) was consistent with quality improvement activity and did not meet criteria for human subjects’ research.

Nurses on the 2 participating general pediatric inpatient units participated. All patient rooms include General Electric Dash monitors for physiologic monitoring. Default alarm thresholds for physiologic parameters are defined by age but can be adjusted by nurses to account for patient physiology. Monitor alarms are relayed to bedside nurses’ institutional mobile telephones (Ascom d62; Ascom Holding AG, Baar, Switzerland). On the units we studied, ∼40% of patients are continuously monitored.

We prospectively measured nurse subjective workload using the NASA-TLX (Fig 1) in a cohort of nurses whose workload we could assess repeatedly across a wide range of alarm exposure rates. This allowed us to use analytic methods designed to assess both within-nurse and between-nurse differences in workload at different alarm count exposure rates.11  The NASA-TLX provides a scale to quantifying workload at the task or job level while accounting for individual differences.11 

FIGURE 1

Copy of NASA-TLX paper survey provided to nurses. RN, registered nurse.

FIGURE 1

Copy of NASA-TLX paper survey provided to nurses. RN, registered nurse.

We approached nurses from the 2 general pediatric units partnering with the PSLL for participation in workload evaluations. An in-person data collector obtained participating nurses’ demographic information and guided them through completing their first NASA-TLX assessments. Subsequently, nurses were asked to complete the NASA-TLX at least once per shift between July 8, 2019, and August 31, 2019, using personalized paper forms available in envelopes with their names on them in a bin on their unit. To measure workload at all hours and days of the week, we reviewed nursing assignments ∼1 week in advance and, on the basis of these advanced assignments, programmed a system to automatically deliver personalized text messages to each assigned nurse during their shift. The personalized text messages requested that the nurse complete a workload evaluation, reflecting on the past 2 hours of the shift. We aimed to obtain at least 10 evaluations per nurse to allow for adjustment based on within-nurse differences.

We collected data on all 6 dimensions of workload that compose the original NASA-TLX designed for the aerospace industry. However, in pediatric hospital settings like ours, the 4-dimension version of the NASA-TLX that is validated in pediatric nurses is more appropriate than the original version.10  This modified version of the NASA-TLX evaluates the following dimensions in the preceding 2 hours: mental demand, physical demand, temporal demand (ie, time pressure), and effort required.6,10  In our analysis we used the 4-dimension instrument; we calculated the sum of scores in these 4 dimensions and scaled it to 100 to facilitate interpretation of the results and to inform subsequent analyses.6  We performed a sensitivity analysis using the original 6-dimension scale (scaled to 100).

In addition to nurse demographics, we collected information about the work environment and patient factors that potentially contribute to subjective workload, including unit, time of shift (day versus night), time within a shift the NASA-TLX was completed, nurse to patient ratio, patient acuity, the number of assigned patients receiving physiologic monitoring, and alarm exposure in the preceding 2 hours. Nurses recorded the time of NASA-TLX completion and listed the medical record numbers of all patients they were caring for at the time of NASA-TLX completion. Patient acuity was evaluated by using the Pediatric Rothman Index (PRI) score,18  a measure of projected probability of clinical deterioration computed on the basis of vital signs, laboratory test results, and nursing assessments for each patient. At the time of data collection, PRI scores were embedded into the electronic health record. PRI scores <65 denoted patients at risk for clinical deterioration or transfer to the ICU19 ; scores were dichotomized such that patients with PRI scores <65 were categorized as having high acuity and patients with PRI scores ≥65 were categorized as normal or lower acuity.

We extracted monitor alarm counts (the absolute number of abnormal vital signs that triggered a monitor to alarm) from the hospital’s clinical data warehouse. We paired NASA-TLX assessments with the total count of alarms for the nurses’ assigned patients during the preceding 2 hours (the period evaluated in the NASA-TLX). We also included alarms from patients whom nurses were only covering for part of the 2-hour window (eg, when covering another nurse’s patients because the other nurse is on a break).

Employing an approach used in our previous work evaluating alarm count and response time, we evaluated alarm counts categorically because we did not anticipate a linear relationship between alarms and workload.16  When alarms are infrequent, nurses are likely able to respond to them, whereas at high levels, we hypothesized alarms would impose additional workload burden. We visually examined a locally weighted scatterplot smoother to inform alarm count categorization. We divided alarm counts over the preceding 2 hours into 3 categories: 0 to 5 alarms to represent a low or typical alarm rate (50% of time periods), 6 to 39 alarms to represent an elevated alarm rate (50%–90%), and ≥40 alarms to represent a very high alarm rate (the top 10% in terms of alarm frequency).

To model this dynamic system in which numbers of repeat observations varied for each nurse, we used a hierarchical linear mixed-effects regression model that accounted for clustering of observations within nurses and clustering of nurses within units by including unit- and nurse-specific random effects. To select model variables, we examined the association of the NASA-TLX workload score (the primary outcome) with each of the nurse, patient, and work environment variables as fixed effects in bivariate analyses and performed a Wald test to evaluate the linear hypothesis. Variables with P ≤ .20 in these bivariate, unadjusted analyses were included in the subsequent multivariable model of scaled, composite workload scores. Because alarm counts and the number of monitored patients being cared for measured overlapping constructs, we included only alarm counts in our multivariable analysis. Informed by the literature and awareness of the complex relationships between staffing ratios and patient acuity from our previous work20  and the work of others,6  we included a 2-way interaction between patient acuity and nurse to patient ratio in our model. We reported scaled, adjusted estimates of workload using predictive margins for each alarm count category.

We used Research Electronic Data Capture hosted by our institution for data management21,22  and Stata version 16.1 (Stata Corp, College Station, TX) for statistical analysis.

We measured the subjective workload of 26 nurses 394 times using the NASA-TLX. After guiding nurses through completion of initial NASA-TLX evaluations in person (n = 39), we transitioned to delivering automated text messages to prompt NASA-TLX completion. We delivered 586 automated messages and subsequently collected 355 workload evaluations. Our response rate (based on responses to automated text messages) was 61%. In attempting to understand nonresponse, it became clear that the preshift assignments that we based our automated messaging strategy on were subject to change because of nurses being called off or calling out of a shift, reassignment of nurses to different units, or assignment of nurses to serve as a charge nurse. Nonetheless, participating nurses completed a median of 11 NASA-TLX evaluations (interquartile range [IQR]: 6–17; range: 1–35) during the study period. Nurses had a median 3 years of nursing experience (IQR: 2–5 years), and 6 nurses (23%) were in their first year of practice. Most nurses completed at least 1 workload evaluation during a nightshift (22 nurses; 85%), and 25 nurses (96%) evaluated a time period in which they cared for ≥1 patient with high acuity (defined as having a PRI score <65).

In terms of shift characteristics (Table 1), 46% of NASA-TLX evaluations were completed during day shifts, and 54% of evaluations were completed during night shifts. In 63% of assessed shifts, nurses cared for patients with high acuity. Nurses cared for a median 4 patients (IQR: 3–4, range: 2–5). Nurses cared for at least 1 patient receiving continuous monitoring in 74% of shifts. Nurses received a median of 6 alarms (IQR: 0–21) in the 2 hours before NASA-TLX completion. The median reported NASA-TLX workload score was 68.6 (IQR: 57.3–79.0).

TABLE 1

Shift Characteristics

Workload Evaluations, No. (%)
Total 394 
Years of nursing experience, y  
 <2 122 (31) 
 2–5 215 (55) 
 >5 57 (15) 
Unit  
 Unit 1 124 (32) 
 Unit 2 270 (69) 
Shift type  
 Day 180 (46) 
 Night 214 (54) 
Time within a shift of completion, h  
 <3 130 (33) 
 3–7 105 (27) 
 8–12 159 (40) 
No. patients cared for  
 1–3 patients 171 (43) 
 4 or 5 patients 223 (57) 
No. monitored patients cared for  
 None 103 (26) 
 1 patient 116 (29) 
 2–4 patients 175 (44) 
Caring for ≥1 patient with high acuity 249 (63) 
NASA-TLX workload score  
 0–50 73 (19) 
 51–75 186 (47) 
 75–100 135 (34) 
No. alarms in the preceding 2 h  
 0–5 alarms 195 (50) 
 6–40 alarms 159 (40) 
 >40 alarms 40 (10) 
Workload Evaluations, No. (%)
Total 394 
Years of nursing experience, y  
 <2 122 (31) 
 2–5 215 (55) 
 >5 57 (15) 
Unit  
 Unit 1 124 (32) 
 Unit 2 270 (69) 
Shift type  
 Day 180 (46) 
 Night 214 (54) 
Time within a shift of completion, h  
 <3 130 (33) 
 3–7 105 (27) 
 8–12 159 (40) 
No. patients cared for  
 1–3 patients 171 (43) 
 4 or 5 patients 223 (57) 
No. monitored patients cared for  
 None 103 (26) 
 1 patient 116 (29) 
 2–4 patients 175 (44) 
Caring for ≥1 patient with high acuity 249 (63) 
NASA-TLX workload score  
 0–50 73 (19) 
 51–75 186 (47) 
 75–100 135 (34) 
No. alarms in the preceding 2 h  
 0–5 alarms 195 (50) 
 6–40 alarms 159 (40) 
 >40 alarms 40 (10) 

Descriptive characteristics of the shifts in which NASA-TLX (N = 394) was completed to assess workload. Nurses completed up to 2 NASA-TLX workload evaluations per shift.

In unadjusted analyses, all variables met the a priori criterion of P ≤ .2 for inclusion in the multivariable model (Table 2). In multivariable mixed-effects modeling, after we controlled for other contributors to workload, experiencing a very high alarm rate increased the workload scores by 5.5 points (95% confidence interval [CI]: 5.2 to 5.7). Nurses with 2 to 5 years’ experience, midshift (hours 3–7 of a 12-hour shift), increasing from 1 to 3 patients with low acuity to 4 to 5 patients with low acuity, and caring for at least 1 patient with high acuity (among nurses caring for 1–3 patients) were associated with increased workload. In Table 3, we modeled different conditions to illustrate the effect of alarms on nurse workload using predictive margins.

TABLE 2

Predicted Change in NASA-TLX Workload Score (Scaled to 100)

UnadjustedAdjusted
PP
Predicted Change (95% CI)CategoryCompositePredicted Change (95% CI)CategoryComposite
Alarms in the preceding 2 h   <.001   <.001 
 0–5 alarms Reference — — Reference — — 
 6–39 alarms +6.9 (3.1 to 10.7) <.001 — +1.9 (0.9 to 2.9) <.001 — 
 ≥40 alarms +11.8 (5.6 to 17.9) <.001 — +5.5 (5.2 to 5.7) <.001 — 
Years of nursing experience, y   <.001   <.001 
 <2 Reference — — Reference — — 
 2–5 +15.5 (11.9 to 19.1) <.001 — +6.4 (5.4 to 7.4) <.001 — 
 >5 −6.4 (−11.5 to −1.3) .01 — −12.1 (−17.2 to −7.0) <.001 — 
Unit — — <.001 Random effect — — 
 Unit 1 Reference — — — — — 
 Unit 2 +15.0 (11.3 to 18.6) <.001 — — — — 
Shift type   .4   .09 
 Day Reference — — Reference — — 
 Night +1.6 (−2.1 to 5.2) .4 — −4.8 (−10.2 to 0.7) 0.09 — 
Time (within a shift), h   .003   0.9 
 <3 Reference — — Reference — — 
 3–7 +8.2 (3.5 to 13.0) .001 — +3.5 (0.5 to 6.5) 0.02 — 
 8–12 +3.9 (−0.4 to 8.1) .07 — +0.5 (−6.7 to 7.6) 0.9 — 
No. monitored patients — — <.001 Not included — — 
 None Reference — — — — — 
 1 patient +6.1 (1.5 to 10.8) .01 — — — — 
 2–4 patients +14.3 (10.0 to 18.5) <.001 — — — — 
No. and acuity of patients   <.001   .03 
 1–3 patients, none with high acuity Reference — — Reference — — 
 1–3 patients, ≥1 with high acuity +13.1 (7.6 to 18.6) <.001 — +8.1 (1.6 to 14.7) .02 — 
 4 or 5 patients, none with high acuity +8.5 (2.6 to 14.3) .005 — +7.8 (0.6 to 15.1) .03 — 
 4 or 5 patients, ≥1 with high acuity +12.1 (6.9 to 17.3) <.001 — +5.7 (2.0 to 9.4) .002 — 
UnadjustedAdjusted
PP
Predicted Change (95% CI)CategoryCompositePredicted Change (95% CI)CategoryComposite
Alarms in the preceding 2 h   <.001   <.001 
 0–5 alarms Reference — — Reference — — 
 6–39 alarms +6.9 (3.1 to 10.7) <.001 — +1.9 (0.9 to 2.9) <.001 — 
 ≥40 alarms +11.8 (5.6 to 17.9) <.001 — +5.5 (5.2 to 5.7) <.001 — 
Years of nursing experience, y   <.001   <.001 
 <2 Reference — — Reference — — 
 2–5 +15.5 (11.9 to 19.1) <.001 — +6.4 (5.4 to 7.4) <.001 — 
 >5 −6.4 (−11.5 to −1.3) .01 — −12.1 (−17.2 to −7.0) <.001 — 
Unit — — <.001 Random effect — — 
 Unit 1 Reference — — — — — 
 Unit 2 +15.0 (11.3 to 18.6) <.001 — — — — 
Shift type   .4   .09 
 Day Reference — — Reference — — 
 Night +1.6 (−2.1 to 5.2) .4 — −4.8 (−10.2 to 0.7) 0.09 — 
Time (within a shift), h   .003   0.9 
 <3 Reference — — Reference — — 
 3–7 +8.2 (3.5 to 13.0) .001 — +3.5 (0.5 to 6.5) 0.02 — 
 8–12 +3.9 (−0.4 to 8.1) .07 — +0.5 (−6.7 to 7.6) 0.9 — 
No. monitored patients — — <.001 Not included — — 
 None Reference — — — — — 
 1 patient +6.1 (1.5 to 10.8) .01 — — — — 
 2–4 patients +14.3 (10.0 to 18.5) <.001 — — — — 
No. and acuity of patients   <.001   .03 
 1–3 patients, none with high acuity Reference — — Reference — — 
 1–3 patients, ≥1 with high acuity +13.1 (7.6 to 18.6) <.001 — +8.1 (1.6 to 14.7) .02 — 
 4 or 5 patients, none with high acuity +8.5 (2.6 to 14.3) .005 — +7.8 (0.6 to 15.1) .03 — 
 4 or 5 patients, ≥1 with high acuity +12.1 (6.9 to 17.3) <.001 — +5.7 (2.0 to 9.4) .002 — 

—, not applicable.

TABLE 3

Adjusted Estimates of Workload Scores (Scaled to 100), by Alarm Frequency

0–5 Alarms, Predicted Score (95% CI)6–39 Alarms, Predicted Score (95% CI)≥40 Alarms, Predicted Score (95% CI)
Years of nursing experience, y    
 <2 61.0 (51.7 to 70.3) 62.9 (52.5 to 73.2) 66.4 (57.4 to 75.5) 
 2–5 67.3 (58.3 to 76.4) 69.2 (59.2 to 79.3) 72.8 (64.1 to 81.6) 
 >5 49.5 (36.0 to 63.0) 51.4 (36.8 to 65.9) 54.9 (41.7 to 68.2) 
Shift type    
 Day 64.3 (57.8 to 70.9) 66.3 (58.7 to 73.8) 69.8 (63.6 to 76.1) 
 Night 61.4 (49.0 to 73.9) 63.3 (49.8 to 76.9) 66.9 (54.7 to 79.2) 
Time (within a shift) of completion, h    
 <3 61.4 (55.7 to 67.0) 63.3 (56.6 to 70.0) 66.9 (61.4 to 72.3) 
 3–7 66.2 (57.7 to 74.7) 68.1 (58.6 to 77.7) 71.7 (63.4 to 79.9) 
 8–12 61.6 (47.7 to 75.6) 63.6 (48.6 to 78.5) 67.1 (53.5 to 80.8) 
Patients and acuity    
 1–3 patients with lower acuity 57.5 (44.3 to 70.7) 59.4 (45.2 to 73.6) 63.0 (50.0 to 75.9) 
 1–3 patients, including ≥1 with high acuity 64.6 (57.4 to 71.8) 66.5 (58.3 to 74.8) 70.1 (63.2 to 77.0) 
 4 or 5 patients with lower acuity 64.4 (57.3 to 71.4) 66.3 (58.2 to 74.4) 69.8 (63.0 to 76.7) 
 4 or 5 patients, including ≥1 with high acuity 63.0 (51.6 to 74.5) 64.9 (52.4 to 77.4) 68.5 (57.3 to 79.7) 
0–5 Alarms, Predicted Score (95% CI)6–39 Alarms, Predicted Score (95% CI)≥40 Alarms, Predicted Score (95% CI)
Years of nursing experience, y    
 <2 61.0 (51.7 to 70.3) 62.9 (52.5 to 73.2) 66.4 (57.4 to 75.5) 
 2–5 67.3 (58.3 to 76.4) 69.2 (59.2 to 79.3) 72.8 (64.1 to 81.6) 
 >5 49.5 (36.0 to 63.0) 51.4 (36.8 to 65.9) 54.9 (41.7 to 68.2) 
Shift type    
 Day 64.3 (57.8 to 70.9) 66.3 (58.7 to 73.8) 69.8 (63.6 to 76.1) 
 Night 61.4 (49.0 to 73.9) 63.3 (49.8 to 76.9) 66.9 (54.7 to 79.2) 
Time (within a shift) of completion, h    
 <3 61.4 (55.7 to 67.0) 63.3 (56.6 to 70.0) 66.9 (61.4 to 72.3) 
 3–7 66.2 (57.7 to 74.7) 68.1 (58.6 to 77.7) 71.7 (63.4 to 79.9) 
 8–12 61.6 (47.7 to 75.6) 63.6 (48.6 to 78.5) 67.1 (53.5 to 80.8) 
Patients and acuity    
 1–3 patients with lower acuity 57.5 (44.3 to 70.7) 59.4 (45.2 to 73.6) 63.0 (50.0 to 75.9) 
 1–3 patients, including ≥1 with high acuity 64.6 (57.4 to 71.8) 66.5 (58.3 to 74.8) 70.1 (63.2 to 77.0) 
 4 or 5 patients with lower acuity 64.4 (57.3 to 71.4) 66.3 (58.2 to 74.4) 69.8 (63.0 to 76.7) 
 4 or 5 patients, including ≥1 with high acuity 63.0 (51.6 to 74.5) 64.9 (52.4 to 77.4) 68.5 (57.3 to 79.7) 

The sensitivity analysis using the 6-dimension NASA-TLX revealed similar results. In multivariable models using the 6-domain scale, high alarm count increased workload scores by 5.8 points (95% CI: 5.7 to 5.8). Although the coefficient values for the other covariates changed slightly, none changed in statistical significance or direction.

We sought to evaluate the relationship between alarm count and subjective workload using the NASA-TLX. When we accounted for other contributors to work load, high alarm count (≥40 alarms) was associated with a >5-point increase in subjective workload. The association between high alarm counts and workload is clinically significant; in terms of magnitude, it is similar to the increase in workload experienced when caring for 4 to 5 patients (as compared to caring for 1–3 patients). This finding is consistent with a simulation-based study that found that reduction in alarms resulted in a 5-point decrease in NASA-TLX workload scores.23 

Our findings support the negative impact of high alarm count on nurses’ provision of care. In work by Tubbs-Cooley et al,6  each 5-point increase in nurse subjective workload was associated with a 22% increased risk in self-reported missed nursing care. Both increased alarm count16  and subjective workload23  have been associated with slowed alarm response time. Numerous other adverse outcomes are associated with high alarm burden, including poor nurse-patient communication,5  missed delivery of nursing care,6  hospital-acquired infection,4  and increased mortality.13  More broadly, the inverse relationship between increased workload and task performance has been established in the human factors literature.24,25 

Multiple factors influence workload, and thus (as highlighted in Table 3) the increased burden imposed by high alarm count may have different implications in different contexts. For example, nurses with 2 to 5 years of experience report high levels of subjective workload at low alarm counts, compared with nurses with >5 years of experience. The human factors literature suggests that professional experience may confer “attentional spare margin,” such that individuals with experience have additional “spare capacity” that reduces the workload associated with tasks.26  Moreover “limited capacity” model posits that as individuals approach their capacity “threshold” (the point at which additional task demands cause performance to decline), their perceived workload increases.2730  These theories suggest that for nurses without spare capacity who are already experiencing high levels of workload, the addition of alarm-associated workload may have serious implications for patient safety. Of note, nurses with 2 to 5 years of experience report higher levels of subjective workload than nurses with <2 years of experience. A variety of factors may contribute to the lower subjective workload of new nurses, including reporting bias, additional supervision and support given to new nurses (generally by nurses with 2 to 5 years of experience), and/or reduced or lower complexity patient assignments. The impact of additional workload on nurse performance when workload is already high (ie, the nurse with 2–5 years’ experience) may differ from the performance impact of additional workload for a more experienced nurse whose baseline workload is lower. Similarly, there may be particularly vulnerable moments within a clinical shift when an interruption from an uninformative alarm is more burdensome. For instance, observational work has revealed the association of phone interruptions with medication administration errors.20  Among the multitude of complex interacting factors that influence workload, alarm counts are modifiable. Reducing nonactionable alarms may be one of the most modifiable pragmatic targets for improving nurse workload.

To our knowledge this is the first study in which the relationship between alarm counts and subjective nursing workload is evaluated in clinical practice. Our reproducible approach allows for quantitative evaluation of the consequences of alarm fatigue. As part of our broader quality improvement work, workload will serve as a proximal outcome by which to evaluate future interventions focused on improving monitoring systems.

These findings must be contextualized within the limitations of our approach. We surveyed a limited number of nurses within a single pediatric hospital. Intuitional policies and practices may influence both the frequency and experience of alarms. We used automated text messages to request that nurses complete workload evaluations. Delivery of automated text messages was scheduled on the basis of preshift assignments. Although this approach allowed us to evaluate workload throughout shifts, we are unable to systematically evaluate why nurses did or did not complete workload evaluations. Some nonresponses were due to changes in staffing. However, because nurses were prompted to complete the NASA-TLX during their clinical shifts, at times, shift characteristics may have influenced completion. Busier nurses may have delayed or failed to complete evaluations of workload. Alternatively, nurses may have been eager to report workload during particularly challenging times. By obtaining frequent, repeated evaluations of workload on a routine basis, we aimed to overcome these sampling challenges. We adopted a human factors approach that accounted for nurse-, unit-, and patient-level factors; however, because nurses did not cross units, there may be unit-level factors we were unable to account for in our analysis. Finally, the literature regarding workload and expected workload of nurses is emerging. The original NASA-TLX measures 6 dimensions of workload; for both validity and interpretability within the existing nursing literature, we report scores from a 4-dimension version that has been validated among nurses. A sensitivity analysis using the 6-component NASA-TLX revealed similar results.

Increased physiologic monitor alarm exposure was associated with clinically and statistically significant increases in subjective nurse workload, which may lead to missed nursing care tasks and patient harm. Reducing nonactionable physiologic monitor alarm burden should be a high priority to improve patient safety.

FUNDING: Supported by grant R18HS026620 from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.

Dr Rasooly conducted the analyses and drafted the initial manuscript; Dr Kern-Goldberger participated in the analyses and reviewed and revised the manuscript; Drs Xiao and Bonafide conceptualized and designed the study, participated in the analysis, and reviewed and revised the manuscript; Drs Ponnala and Ruppel critically reviewed and revised the manuscript; Drs Luo and Craig and Ms McLoon coordinated National Aeronautics and Space Administration Task Load Index data collection and critically reviewed the manuscript; Ms Khan collected alarm data and critically reviewed the manuscript; Drs Ferro, Muthu, and Won conceptualized and designed the study and critically reviewed the manuscript; and all authors approved the final manuscript as submitted.

1.
Lamy Filho
F
,
da Silva
AAM
,
Lopes
JMA
,
Lamy
ZC
,
Simões
VMF
,
Dos Santos
AM
.
Staff workload and adverse events during mechanical ventilation in neonatal intensive care units
.
J Pediatr (Rio J)
.
2011
;
87
(
6
):
487
492
2.
Aiken
LH
,
Clarke
SP
,
Sloane
DM
,
Sochalski
J
,
Silber
JH
.
Hospital nurse staffing and patient mortality, nurse burnout, and job dissatisfaction
.
JAMA
.
2002
;
288
(
16
):
1987
1993
3.
Kiekkas
P
,
Sakellaropoulos
GC
,
Brokalaki
H
, et al
.
Association between nursing workload and mortality of intensive care unit patients
.
J Nurs Scholarsh
.
2008
;
40
(
4
):
385
390
4.
Rogowski
JA
,
Staiger
D
,
Patrick
T
,
Horbar
J
,
Kenny
M
,
Lake
ET
.
Nurse staffing and NICU infection rates
.
JAMA Pediatr
.
2013
;
167
(
5
):
444
450
5.
Carayon
P
,
Gurses
AP
.
Nursing workload and patient safety—a human factors engineering perspective
. In:
Hughes
RG
, ed.
Patient Safety and Quality: An Evidence-Based Handbook for Nurses
.
Rockville, MD
:
Agency for Healthcare Research and Quality
;
2008
:
203
216
6.
Tubbs-Cooley
HL
,
Mara
CA
,
Carle
AC
,
Mark
BA
,
Pickler
RH
.
Association of nurse workload with missed nursing care in the neonatal intensive care unit
.
JAMA Pediatr
.
2019
;
173
(
1
):
44
51
7.
Hoonakker
P
,
Carayon
P
,
Gurses
A
, et al
.
Measuring workload of ICU nurses with a questionnaire survey: the NASA Task Load Index (TLX)
.
IIE Trans Healthc Syst Eng
.
2011
;
1
(
2
):
131
143
8.
Xie
B
,
Salvendy
G
.
Review and reappraisal of modelling and predicting mental workload in single- and multi-task environments
.
Work Stress
.
2000
;
14
(
1
):
74
99
9.
Morris
R
,
MacNeela
P
,
Scott
A
,
Treacy
P
,
Hyde
A
.
Reconsidering the conceptualization of nursing workload: literature review
.
J Adv Nurs
.
2007
;
57
(
5
):
463
471
10.
Tubbs-Cooley
HL
,
Mara
CA
,
Carle
AC
,
Gurses
AP
.
The NASA Task Load Index as a measure of overall workload among neonatal, paediatric and adult intensive care nurses
.
Intensive Crit Care Nurs
.
2018
;
46
:
64
69
11.
Hart
SG
,
Staveland
LE
.
Development of NASA-TLX (Task Load Index): results of empirical and theoretical research
. In:
Hancock
PA
,
Meshkati
N
, eds.
Advances in Psychology
.
Vol 52
.
Amsterdam, Netherlands
:
Elsevier
;
1988
:
139
183
12.
Rubio
S
,
Diaz
E
,
Martin
J
,
Puente
JM
.
Evaluation of subjective mental workload: a comparison of SWAT, NASA-TLX, and workload profile methods
.
Appl Psychol
.
2004
;
53
(
1
):
61
86
13.
Colligan
L
,
Potts
HWW
,
Finn
CT
,
Sinkin
RA
.
Cognitive workload changes for nurses transitioning from a legacy system with paper documentation to a commercial electronic health record
.
Int J Med Inform
.
2015
;
84
(
7
):
469
476
14.
Schondelmeyer
AC
,
Brady
PW
,
Goel
VV
, et al
.
Physiologic monitor alarm rates at 5 children’s hospitals
.
J Hosp Med
.
2018
;
13
(
6
):
396
398
15.
Schondelmeyer
AC
,
Bonafide
CP
,
Goel
VV
, et al
.
The frequency of physiologic monitor alarms in a children’s hospital
.
J Hosp Med
.
2016
;
11
(
11
):
796
798
16.
Bonafide
CP
,
Lin
R
,
Zander
M
, et al
.
Association between exposure to nonactionable physiologic monitor alarms and response time in a children’s hospital
.
J Hosp Med
.
2015
;
10
(
6
):
345
351
17.
Bonafide
CP
,
Localio
AR
,
Holmes
JH
, et al
.
Video analysis of factors associated with response time to physiologic monitor alarms in a children’s hospital
.
JAMA Pediatr
.
2017
;
171
(
6
):
524
531
18.
Rothman
MJ
,
Tepas
JJ
 III
,
Nowalk
AJ
, et al
.
Development and validation of a continuously age-adjusted measure of patient condition for hospitalized children using the electronic medical record
.
J Biomed Inform
.
2017
;
66
:
180
193
19.
da Silva
YS
,
Hamilton
MF
,
Horvat
C
, et al
.
Evaluation of electronic medical record vital sign data versus a commercially available acuity score in predicting need for critical intervention at a tertiary children’s hospital
.
Pediatr Crit Care Med
.
2015
;
16
(
7
):
644
651
20.
Bonafide
CP
,
Miller
JM
,
Localio
AR
, et al
.
Association between mobile telephone interruptions and medication administration errors in a pediatric intensive care unit
.
JAMA Pediatr
.
2020
;
174
(
2
):
162
169
21.
Harris
PA
,
Taylor
R
,
Thielke
R
,
Payne
J
,
Gonzalez
N
,
Conde
JG
.
Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support
.
J Biomed Inform
.
2009
;
42
(
2
):
377
381
22.
Harris
PA
,
Taylor
R
,
Minor
BL
, et al
;
REDCap Consortium
.
The REDCap consortium: building an international community of software platform partners
.
J Biomed Inform
.
2019
;
95
:
103208
23.
Shanmugham
M
,
Strawderman
L
,
Babski-Reeves
K
,
Bian
L
.
Alarm-related workload in default and modified alarm settings and the relationship between alarm workload, alarm response rate, and care provider experience: quantification and comparison study
.
JMIR Hum Factors
.
2018
;
5
(
4
):
e11704
24.
Cummings
ML
,
Nehme
CE
.
Modeling the impact of workload in network centric supervisory control settings
. In:
Kornguth
S
,
Steinberg
R
,
Matthews
MD
, eds.
Neurocognitive and Physiological Factors During High-Tempo Operations
. 1st ed.
London, United Kingdom
:
CRC Press
;
2010
:
23
40
25.
Hebb
DO
.
Drives and the C.N.S. (conceptual nervous system)
.
Psychol Rev
.
1955
;
62
(
4
):
243
254
26.
Leedal
JM
,
Smith
AF
.
Methodological approaches to anaesthetists’ workload in the operating theatre
.
Br J Anaesth
.
2005
;
94
(
6
):
702
709
27.
Kahneman
D
.
Attention and Effort
.
Englewood Cliffs, NJ
:
Prentice Hall
;
1973
28.
Hockey
GR
.
Compensatory control in the regulation of human performance under stress and high workload; a cognitive-energetical framework
.
Biol Psychol
.
1997
;
45
(
1–3
):
73
93
29.
Mallat
C
,
Cegarra
J
,
Calmettes
C
,
Capa
RL
.
A curvilinear effect of mental workload on mental effort and behavioral adaptability: an approach with the pre-ejection period
.
Hum Factors
.
2020
;
62
(
6
):
928
939
30.
Hancock
PA
,
Szalma
JL
, eds.
Performance Under Stress: Human Factors in Defence
.
Burlington, VT
:
Ashgate
;
2008

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.