Changes Made to Orders Placed by Overnight Admitting Residents on Teaching Rounds the Next Day

With increasing national attention to patient safety and quality of care over the past 2 decades, many hospitals and training programs have increased attending physicians’ supervision of trainees. The effect of increased attending oversight on patient safety remains unknown, and possible benefits may be tempered by potentially deleterious effects on trainees’ development of autonomy.^1,2  The degree of attending supervision overnight varies nationwide.³  At our freestanding children’s hospital, residents on Pediatric Hospital Medicine (PHM) overnight shifts place orders that are not specifically reviewed by an attending physician until morning rounds. Although an overnight, in-house attending receives handoffs from the emergency department, they generally only see patients admitted before midnight and do so independently of the resident-admitting team and without explicit order review. This absence of explicit order review overnight signifies an entrustment of autonomous clinical decision-making. An assessment of orders placed during a period of increased resident autonomy has not previously been performed. In this initial, exploratory study we sought to characterize (1) orders placed by overnight residents on newly admitted patients who were subsequently discontinued on morning rounds, and (2) new orders added on morning rounds for patients admitted the night before PHM teaching services. Our primary aims were to categorize the types of orders that are added or discontinued on rounds the morning after admission to a resident team and to understand the rationale for these order additions and discontinuations.

We used our hospital’s enterprise data warehouse (EDW) to generate a report of orders placed for patients admitted to the resident PHM service overnight between July 1, 2017, and June 29, 2018 that were either (1) placed overnight and then discontinued on rounds, or (2) added on rounds the next morning. Orders were classified as “discontinued” if they were placed by overnight residents (6 pm–8 am) and then discontinued during daytime rounding hours (8 am–12 pm). Orders were classified as “new” if they were placed during daytime rounding hours (8 am–12 pm). At least 1 member of the research team manually screened the patient charts containing the discontinued and new orders. We excluded the following categories of orders: (1) duplicate orders without consequential effect to the patient (eg, regular diet ordered twice), (2) administrative orders automatically generated by the electronic health record for laboratory or billing purposes, or (3) initial admission to a non-PHM service. All orders classified as discontinued were first screened for eligibility. Because of a significantly larger number of new orders, new orders were selected for screening via a random number generator to ensure a distribution of orders across the entire study period and to avoid temporal bias related to resident education and experience. Selected new orders were screened until the final sample size equaled that of discontinued orders. Authors EF and LC performed chart review on included orders to determine categorization by type of order and reason for discontinuation. After establishing interrater reliability for the categorization of order type and reason for order addition or discontinuation, chart review of included orders, including review of clinical documentation related to the admission, such as the admission note, daytime attending admission documentation, other physician and nursing notes, medication administration records, and laboratory and diagnostic results, was performed. Orders were categorized by type of order (medicine, diagnostics, diet, patient care, and other) and reason for addition or discontinuation (change in medical trajectory, medical error, and medical decision-making [MDM]). Orders were then subcategorized. Within the “change in medical trajectory” category, orders were subcategorized as “patient improved” or “patient worsened.” Within the “medical error” category, orders were subcategorized as “omitted intended order,” “omitted home medication,” “incorrect home medication,” “medication product of incorrect MDM,” “wrong medication dose,” “wrong medication frequency,” or “wrong order, other.” The subcategories of medical errors were based on an adapted, well-established categorization framework.⁴  Within the “MDM” category, orders were subcategorized as “consultant recommendation” or “primary team MDM.” The primary team MDM subcategory was created as a broad category to account for variation in provider documentation and as a means to designate orders that did not clearly fall into another category based on chart review or where the reason for order change was unclear. An example of primary team MDM is when the morning rounding team discontinued acyclovir ordered by the overnight team for a febrile neonate in whom no new relevant clinical or laboratory data were apparently available on morning rounds, and where both starting acyclovir and discontinuing it were thought to be reasonable management decisions by our research coding team. Orders with ambiguous categorization were discussed with the research team to reach consensus. Orders classified as medical errors were reviewed as a research team to ensure appropriate designation.

Our EDW report identified 819 discontinued orders and 5108 new orders across the study duration. Two hundred sixty-nine discontinued orders met inclusion criteria and subsequently new orders were screened until 269 met inclusion criteria (Fig 1). EF and LC reviewed the charts of 20 included orders independently with excellent interrater reliability (κ = 0.91), after which they reviewed the remaining orders in parallel. The reason for order discontinuation or addition was MDM (n = 357, 66.4%), change in patient trajectory (n = 151, 28.1%), and medical error (n = 30, 5.6%), further subcategorized in Table 1. The majority of discontinued orders were medications (n = 184, 68.4%). Diagnostics were the most common new order (n = 118, 43.8%). The most common medication changes were to antibiotics (n = 62, 21.9%). The most common change to diagnostics was laboratories (n = 103, 79.8%). The medical errors are described in Table 2. An example of a medication error that was the product of incorrect MDM is the use of benzocaine in a pediatric patient admitted for a brief, resolved, unexplained event without other indication.

In this study of orders placed during a period of increased resident autonomy, we found that new or discontinued orders often resulted from evolving patient management decisions or changes in patient trajectory and less commonly from medical errors. Because of the expected changes in clinical status during hospitalization, most of the changes captured reflect the anticipated course of patient care.

Of the medical errors, the majority were medication related and most were errors of omission, consistent with previous literature that errors of omission are the most common error type during home medication reconciliation on hospital admission.^5,6  When considering medical errors in periods of increased resident autonomy specifically, we hypothesize that clinical decision support systems may be more effective than increasing attending presence in mitigating errors, because attending presence alone has not been shown to decrease medical errors in other settings.⁷  In addition to pharmacy and nursing review of orders to identify orders placed incorrectly, we propose that clinical decision support systems be expanded to include a “what’s missing” lens, including alerting admitting providers of possible omitted home medications. The application of machine learning and collaborative filtering to address this problem has previously been described.⁸ 

Limitations of this study include its retrospective nature, including ambiguity in clinical documentation regarding reasons for order discontinuation or addition, inability of our EDW query to capture orders that were modified but not discontinued, and inclusion of a single institution limiting generalizability. This methodology only captures errors that were detected on rounds the next morning, likely underestimating total errors. Errors in too much or too little diagnostic testing cannot easily be captured given myriad inputs (history, physical exam, workup to date, provider risk-tolerance, etc) which inform diagnostic decision-making. The denominator of total orders placed by overnight teams (ie, those that do not go on to be discontinued) and rounding teams was not captured by our EDW report. We also were unable to control for possible confounding factors that might affect the outcome of orders placed during periods of increased resident autonomy, including overnight fatigue, outpatient pharmacy closures, or other changes in available resources overnight.

Our study highlights that the patterns of orders placed by residents autonomously can be harnessed electronically to inform opportunities for intervention, such as increased attention to prevention of errors of omission. Additional next steps include generating resident-specific automated reports of new and discontinued orders as a means to provide formative feedback and raise residents’ personal awareness of order changes so that these data can drive learning. Tracking adherence to existing evidence-based or clinical practice guidelines, such as de-escalation of monitoring in bronchiolitis,⁹  would be valuable in informing resident education and optimizing patient care. Finally, analyzing whether specific orders, such as acyclovir in the setting of neonatal fever, are frequently changed may inform areas for additional standardization with new evidence-based guidelines or areas for additional research.

We thank Dr Frederick Lovejoy, Elayne Fournier, Anne Vaccaro, and the Fred Lovejoy Research and Education Grant Committee for their generous support.