OBJECTIVES:

The Association of American Medical Colleges published a list of entrustable professional activities (EPAs) that graduating medical students should be able to perform on day 1 of residency without direct supervision. We sought to explore the perceptions of residents and pediatric hospitalists about the level of supervision new interns need in conducting these EPAs.

METHODS:

An electronic survey was sent to pediatric hospitalists who supervise interns in a large pediatric residency program in which they were asked to rate the amount of supervision they perceive new interns need when performing 11 EPAs. Another survey was sent to residents in which they were asked how much supervision they needed at the beginning of their intern year when performing the same EPAs. Hospitalist and resident responses were compared.

RESULTS:

The majority of hospitalists thought new interns could perform only 5 of the 11 EPAs without direct supervision. For 5 of the EPAs, residents’ perceptions of their own abilities to perform the EPA independently as interns were significantly greater than the hospitalists’ perceptions of interns’ abilities. For example, 91% of residents thought they were able to recommend and interpret common diagnostic and screening tests when they were interns without direct supervision, compared with only 30% of attending physicians (P < .001).

CONCLUSIONS:

Hospitalists and residents in a pediatric residency training program reported that new interns were not consistently able to perform the Association of American Medical Colleges’ core EPAs without supervision. Furthermore, residents and pediatric hospitalists perceived new interns’ abilities to perform the EPAs without direct supervision significantly differently.

Each July, trainees advance into new roles in hospitals nationwide. Given the variation in clinical instruction across medical schools15  and the limited pediatric-specific training in medical school, the transition from medical school to intern year poses a particular challenge for pediatric trainees and hospitalists alike. Newly minted doctors may begin residency with considerably different knowledge bases and skill sets, and pediatric hospitalists bear the responsibility of ensuring that trainees can safely provide patient care with an appropriate level of supervision when on inpatient pediatrics rotations. There are data to suggest that inadequate supervision of residents is associated with increased medical errors6,7  and enhanced supervision of trainees is associated with improved patient outcomes,8  although the role that supervision plays in the safety of pediatric care in particular has not been well established. Additionally, across specialties, it has been demonstrated in recent literature that many program directors perceive new residents to be underprepared for residency.9 

Accordingly, in May 2014, the Association of American Medical Colleges (AAMC) published a list of 13 core entrustable professional activities (EPAs) that all entering residents should be expected to perform on day 1 of residency without direct supervision, regardless of specialty.10  It is unknown, however, whether new residents are currently competent to perform the AAMC EPAs without direct supervision. To date, little work has been done examining how much supervision residents and pediatric hospitalists perceive new interns to need when performing the AAMC’s recommended EPAs Additionally, it is unknown whether pediatric hospitalists and residents have different perceptions on how much supervision is needed at the beginning of the year. We therefore sought to examine how much supervision pediatric hospitalists and residents perceive new interns to need when performing the AAMC’s core EPAs in a large pediatrics residency program.

This study was performed in a large, urban pediatric residency program. Our 4-person study group (composed of pediatric hospitalists, residency program leadership, and a chief resident) designed 2 surveys with which we intended to ascertain faculty perceptions and resident self-perceptions of the level of supervision required when performing the AAMC EPAs. We chose response options based on scales of proficiency commonly used in the literature and in similar educational studies.11  We solicited feedback from 6 content experts, including experts in competency-based education, residency education, survey design, and pediatric hospital medicine. The major revision made by this group was to exclude 2 of the 13 EPAs. We did not include the following: (1) obtaining informed consent for tests and/or procedures; and (2) performing general procedures of a physician. Although these 2 activities are considered important, interns in our pediatric residency program infrequently perform these tasks at the beginning of the intern year. Therefore, the study group worried that it would be difficult for residents and attending physicians to assess the amount of supervision needed when performing these EPAs. Before formal survey administration, we conducted a small pilot with 2 chief residents and 2 pediatric hospitalists, without major revisions. Formal cognitive interviewing was not performed. Our institutional review board reviewed the protocol and determined that it qualified as exempt.

For each of the 11 EPAs, we asked the hospitalists to answer the following question: “For each of the following activities, please rate how you would describe your level of entrustment of interns on average at the beginning of the intern year.” Answer options included the following: (1) Has knowledge; (2) May act under full supervision; (3) May act under moderate supervision; (4) May act independently; or (5) May act as a supervisor and instructor.

For each of the 11 EPAs, we asked residents the following similar question: “Rate how much supervision you feel that you needed at the beginning of the intern year when performing each of the following activities.” Answer options included the following: (1) I had knowledge about this but could not do it; (2) I was able to perform under full supervision; (3) I was able to perform under moderate supervision; (4) I was able to perform independently without supervision; or (5) I was able to function as a supervisor and instructor for this.

Approximately 4 months into the 2014–2015 academic year, the voluntary electronic survey was administered via e-mail to all pediatric hospitalists who supervise pediatric interns on inpatient general pediatric services and all current residents in our pediatrics program. Hospitalists supervise interns for a variable number of weeks per year and have a variable number of years in practice; their background and experience with EPAs was also variable. One reminder e-mail was sent 1 week after the original e-mail. Incentives were not used.

We calculated the number, percentage, and 95% confidence intervals (CIs) of pediatric hospitalists and residents who selected 1 of the top 3 options (options 3, 4, or 5 out of 5) for each activity; we determined that these 3 options best corresponded with the AAMC’s expectation that interns should be able to perform the EPAs “without direct supervision.” Fisher’s exact tests were used to test for differences in the percentages between residents on the basis of their postgraduate year (PGY) as well as differences between pediatric hospitalists and residents. A P value of <.05 was considered statistically significant. The 95% CIs were calculated by using the Wilson Method. SAS software (version 9.4; SAS Institute, Inc, Cary, NC) was used for the analysis. We performed a secondary analysis in which we limited the comparison to the top 2 options (options 4 and 5 out of 5) and excluded option 3 (performance with moderate supervision), representing an intern’s ability to perform an EPA independently without supervision.

Twenty-seven out of 50 hospitalists (54%) and 65 out of 152 possible residents (43%) responded to the survey. Of the hospitalist participants, 11 (42%) attended solely on inpatient general pediatrics wards, 7 (27%) attended on an intermediate care unit, 2 (8%) attended on a short stay unit, 6 (23%) attended on both general pediatrics and short stay units, and 1 (4%) left the question unanswered. Of the resident participants, 25 (39%) were postgraduate year 1 (PGY1), 20 (31%) were postgraduate year 2 (PGY2), 19 (29%) were postgraduate year 3 (PGY3), and 1 (2%) was postgraduate year 4 (PGY4).

Pediatric hospitalists reported that the average new intern’s ability to perform the core EPAs without direct supervision was limited (Table 1). There were only 5 EPAs that the majority of hospitalists believed new interns could perform without direct supervision, correlating to 1 of the top 3 options (options 3, 4 or 5 out of 5). In contrast, there were 9 EPAs that the majority of residents believed new interns could perform without direct supervision. On secondary analysis (using our more stringent definition in which we included only options 4 and 5), there were no activities that the majority of pediatric hospitalists thought new interns could perform independently, and there were only 3 activities that the majority of residents believed that new interns could perform independently (Table 2).

TABLE 1

Resident and Pediatric Hospitalist Perceptions of Interns’ Supervisory Needs at the Beginning of the Intern Year (Assessed by Using the Top 3 Boxes of a 5-Point Likert Scale)

Resident: Was Able to Perform Under Moderate Supervision or Independently or Function as a Supervisor and Instructor, n/N (%) (95% CI)aPediatric Hospitalist: Resident May Act Under Moderate Supervision or Independently or as a Supervisor and Instructor, n/N (%) (95% CI)aPb
Gather a history and perform a physical examination 64/65 (98) (92–100) 24/27 (89) (72–96) .07 
Prioritize a differential diagnosis after a clinical encounter 60/65 (92) (83–97) 12/27 (44) (28–63) <.001 
Recommend and interpret common diagnostic and screening tests 59/65 (91) (81–96) 8/27 (30) (16–48) <.001 
Enter and discuss orders and prescriptions 31/65 (48) (36–60) 11/27 (41) (25–59) .65 
Document a clinical encounter in the patient record 48/65 (74) (62–83) 22/27 (81) (63–92) .59 
Provide an oral presentation of a clinical encounter 62/64 (97) (89–99) 23/27 (85) (68–94) .06 
Form clinical questions and receive evidence to advance patient care 54/65 (83) (72–90) 21/27 (78) (59–89) .57 
Give or receive a patient handover to transition care responsibility 47/65 (72) (60–82) 12/27 (44) (28–63) .02 
Collaborate as a member of an interprofessional team 62/65 (95) (87–98) 25/26 (96) (81–99) .99 
Recognize a patient requiring urgent or emergent care and initiate evaluation and management 30/65 (46) (35–58) 6/27 (22) (11–41) .04 
Identify system failures and contribute to a culture of safety and improvement 34/63 (54) (42–66) 11/26 (42) (26–61) .36 
Resident: Was Able to Perform Under Moderate Supervision or Independently or Function as a Supervisor and Instructor, n/N (%) (95% CI)aPediatric Hospitalist: Resident May Act Under Moderate Supervision or Independently or as a Supervisor and Instructor, n/N (%) (95% CI)aPb
Gather a history and perform a physical examination 64/65 (98) (92–100) 24/27 (89) (72–96) .07 
Prioritize a differential diagnosis after a clinical encounter 60/65 (92) (83–97) 12/27 (44) (28–63) <.001 
Recommend and interpret common diagnostic and screening tests 59/65 (91) (81–96) 8/27 (30) (16–48) <.001 
Enter and discuss orders and prescriptions 31/65 (48) (36–60) 11/27 (41) (25–59) .65 
Document a clinical encounter in the patient record 48/65 (74) (62–83) 22/27 (81) (63–92) .59 
Provide an oral presentation of a clinical encounter 62/64 (97) (89–99) 23/27 (85) (68–94) .06 
Form clinical questions and receive evidence to advance patient care 54/65 (83) (72–90) 21/27 (78) (59–89) .57 
Give or receive a patient handover to transition care responsibility 47/65 (72) (60–82) 12/27 (44) (28–63) .02 
Collaborate as a member of an interprofessional team 62/65 (95) (87–98) 25/26 (96) (81–99) .99 
Recognize a patient requiring urgent or emergent care and initiate evaluation and management 30/65 (46) (35–58) 6/27 (22) (11–41) .04 
Identify system failures and contribute to a culture of safety and improvement 34/63 (54) (42–66) 11/26 (42) (26–61) .36 
a

Wilson 95% CIs.

b

Based on Fisher’s exact tests.

TABLE 2

Resident and Pediatric Hospitalist Perceptions of Interns’ Supervisory Needs at the Beginning of the Intern Year (Assessed by Using the Top 2 Boxes of a More Stringent Definition of a 5-Point Likert Scale)

Resident: Was Able to Perform Independently or Function as a Supervisor and Instructor, n/N (%) (95% CI)aHospitalist Physician: Resident May Act Independently or as a Supervisor and Instructor, n/N (%) (95% CI)aPb
Gather a history and perform a physical examination 57/65 (88) (78–94) 5/27 (19) (8–37) <.001 
Prioritize a differential diagnosis after a clinical encounter 19/65 (29) (20–41) 1/27 (4) (1–18) .004 
Recommend and interpret common diagnostic and screening tests 12/65 (18) (11–30) 0/27 (0–12) .02 
Enter and discuss orders and prescriptions 2/65 (3) (1–11) 2/27 (7) (2–23) .58 
Document a clinical encounter in the patient record 18/65 (28) (18–40) 4/27 (15) (6–32) .28 
Provide an oral presentation of a clinical encounter 39/64 (61) (49–72) 10/27 (37) (22–56) .04 
Form clinical questions and receive evidence to advance patient care 24/65 (37) (26–49) 6/27 (22) (11–41) .22 
Give or receive a patient handover to transition care responsibility 4/65 (6) (2–15) 2/27 (7) (2–23) .99 
Collaborate as a member of an interprofessional team 42/65 (65) (52–75) 9/26 (35) (19–54) .01 
Recognize a patient requiring urgent or emergent care and initiate evaluation and management 4/65 (6) (2–15) 0/27 (0–12) .32 
Identify system failures and contribute to a culture of safety and improvement 10/63 (16) (9–27) 1/26 (4) (1–19) .17 
Resident: Was Able to Perform Independently or Function as a Supervisor and Instructor, n/N (%) (95% CI)aHospitalist Physician: Resident May Act Independently or as a Supervisor and Instructor, n/N (%) (95% CI)aPb
Gather a history and perform a physical examination 57/65 (88) (78–94) 5/27 (19) (8–37) <.001 
Prioritize a differential diagnosis after a clinical encounter 19/65 (29) (20–41) 1/27 (4) (1–18) .004 
Recommend and interpret common diagnostic and screening tests 12/65 (18) (11–30) 0/27 (0–12) .02 
Enter and discuss orders and prescriptions 2/65 (3) (1–11) 2/27 (7) (2–23) .58 
Document a clinical encounter in the patient record 18/65 (28) (18–40) 4/27 (15) (6–32) .28 
Provide an oral presentation of a clinical encounter 39/64 (61) (49–72) 10/27 (37) (22–56) .04 
Form clinical questions and receive evidence to advance patient care 24/65 (37) (26–49) 6/27 (22) (11–41) .22 
Give or receive a patient handover to transition care responsibility 4/65 (6) (2–15) 2/27 (7) (2–23) .99 
Collaborate as a member of an interprofessional team 42/65 (65) (52–75) 9/26 (35) (19–54) .01 
Recognize a patient requiring urgent or emergent care and initiate evaluation and management 4/65 (6) (2–15) 0/27 (0–12) .32 
Identify system failures and contribute to a culture of safety and improvement 10/63 (16) (9–27) 1/26 (4) (1–19) .17 
a

Wilson 95% CIs.

b

Based on Fisher’s exact tests.

Residents perceived their own abilities to perform the EPAs without direct supervision when they were new interns significantly differently than did pediatric hospitalists (Table 1). Ninety-two percent of residents perceived that they could have prioritized a differential diagnosis after a clinical encounter without direct supervision versus only 44% of hospitalists (P < .001). In addition, 91% of residents believed they could have recommended and interpreted common diagnostic and screening tests without direct supervision versus 30% of hospitalists (P < .001). Furthermore, 72% of residents perceived that they could have given or received a patient handover to transition care responsibility without direct supervision versus 44% of hospitalists (P = .02). Lastly, 46% of residents perceived that they could have recognized a patient requiring urgent or emergent care and initiated evaluation and management versus 22% of hospitalists (P = .04). In secondary analyses in which we used the more stringent definition, the pattern of discrepancies between hospitalist and resident ratings of new interns’ abilities differed somewhat (Table 2).

For all but 1 EPA, residents across PGY levels had similar self-perceptions of their abilities to perform the activities without direct supervision at the beginning of their intern year (Table 3). However, 100% of PGY1 residents thought they could have performed the EPA “recommend and interpret common diagnostic and screening tests” at the beginning of their intern year without direct supervision, as compared with 90% of PGY2 residents and 80% of PGY3 and PGY4 residents (P = .04).

TABLE 3

Resident Perceptions of Their Supervisory Needs at the Beginning of the Intern Year (Assessed by Using the Top 3 Boxes of a 5-Point Likert Scale)

PGY1, n/N (%) (95% CI)aPGY2, n/N (%) (95% CI)PGY3 and PGY4, n/N (%) (95% CI)Pb
Gather a history and perform a physical examination 25/25 (100) (87–100) 19/20 (95) (76–99) 20/20 (100) (84–100) .62 
Prioritize a differential diagnosis after a clinical encounter 24/25 (96) (80–99) 18/20 (90) (70–97) 18/20 (90) (70–97) .61 
Recommend and interpret common diagnostic and screening tests 25/25 (100) (87–100) 18/20 (90) (70–97) 16/20 (80) (58–92) .04 
Enter and discuss orders and prescriptions 13/25 (52) (34–70) 8/20 (40) (22–61) 10/20 (50) (30–70) .74 
Document a clinical encounter in the patient record 21/25 (84) (65–94) 12/20 (60) (39–78) 15/20 (75) (53–89) .20 
Provide an oral presentation of a clinical encounter 24/25 (96) (80–99) 19/20 (95) (76–99) 19/19 (100) (83–100) .99 
Form clinical questions and receive evidence to advance patient care 24/25 (96) (80–99) 16/20 (80) (58–92) 14/20 (70) (48–85) .07 
Give or receive a patient handover to transition care responsibility 21/25 (84) (65–94) 14/20 (70) (48–85) 12/20 (60) (39–78) .20 
Collaborate as a member of an interprofessional team 25/25 (100) (87–100) 18/20 (90) (70–97) 19/20 (95) (76–99) .28 
Recognize a patient requiring urgent or emergent care and initiate evaluation and management 15/25 (60) (41–77) 6/20 (30) (15–52) 9/20 (45) (26–66) .15 
Identify system failures and contribute to a culture of safety and improvement 16/24 (67) (47–82) 10/20 (50) (30–70) 8/19 (42) (23–64) .26 
PGY1, n/N (%) (95% CI)aPGY2, n/N (%) (95% CI)PGY3 and PGY4, n/N (%) (95% CI)Pb
Gather a history and perform a physical examination 25/25 (100) (87–100) 19/20 (95) (76–99) 20/20 (100) (84–100) .62 
Prioritize a differential diagnosis after a clinical encounter 24/25 (96) (80–99) 18/20 (90) (70–97) 18/20 (90) (70–97) .61 
Recommend and interpret common diagnostic and screening tests 25/25 (100) (87–100) 18/20 (90) (70–97) 16/20 (80) (58–92) .04 
Enter and discuss orders and prescriptions 13/25 (52) (34–70) 8/20 (40) (22–61) 10/20 (50) (30–70) .74 
Document a clinical encounter in the patient record 21/25 (84) (65–94) 12/20 (60) (39–78) 15/20 (75) (53–89) .20 
Provide an oral presentation of a clinical encounter 24/25 (96) (80–99) 19/20 (95) (76–99) 19/19 (100) (83–100) .99 
Form clinical questions and receive evidence to advance patient care 24/25 (96) (80–99) 16/20 (80) (58–92) 14/20 (70) (48–85) .07 
Give or receive a patient handover to transition care responsibility 21/25 (84) (65–94) 14/20 (70) (48–85) 12/20 (60) (39–78) .20 
Collaborate as a member of an interprofessional team 25/25 (100) (87–100) 18/20 (90) (70–97) 19/20 (95) (76–99) .28 
Recognize a patient requiring urgent or emergent care and initiate evaluation and management 15/25 (60) (41–77) 6/20 (30) (15–52) 9/20 (45) (26–66) .15 
Identify system failures and contribute to a culture of safety and improvement 16/24 (67) (47–82) 10/20 (50) (30–70) 8/19 (42) (23–64) .26 
a

Wilson 95% CIs.

b

Based on Fisher’s exact tests.

We found that pediatric hospitalists and residents in a large pediatric residency training program reported that new interns were not consistently able to perform the core EPAs that the AAMC has established as a goal for all new residents without direct supervision. For 6 of the 11 EPAs that we studied, less than half of pediatric hospitalists thought new interns could perform the EPAs without direct supervision. Even residents (who consistently indicated less need for supervision) reported that they could not have performed 2 of the 11 activities without direct supervision at the beginning of their intern year. These data support the AAMC’s call for significant curriculum development and increased training of medical students.

Less than one-third of pediatric hospitalists thought new residents could perform the following 2 EPAs without full supervision: (1) recommending and interpreting common diagnostic and screening tests; and (2) recognizing a patient requiring urgent or emergent care and initiating evaluation and management. Medical errors by new interns performing these activities can lead to significant patient harm. Accordingly, it may be unsafe to expect new interns to perform these tasks without direct supervision, although fostering their ability to do so would certainly be of value.

Although we primarily examined new residents’ performance of the EPAs without direct or full supervision, we also analyzed their abilities to perform the EPAs independently, without any supervision, as a secondary analysis. We hypothesize that many new residents do, in fact, perform many of the EPAs independently, and we therefore thought it would be important to include this analysis. For each of the activities studied, most pediatric hospitalists doubted that new interns could function independently. Similarly, more than half of residents reported that they were unable to perform 8 of the 11 tasks independently at the beginning of their intern year. Therefore, we agree with the AAMC that it is critical to provide at least some supervision for all of the EPAs.

Our study also revealed that for many of the EPAs, residents and pediatric hospitalists had significantly different perceptions regarding the need for supervision of new interns. This secondary finding is in many respects not surprising and likely in part stems from the inaccuracies of intern self-assessment. There is a robust literature base to suggest that physicians inaccurately self-assess their knowledge,12  procedural skills,12,13  and clinical competence.12,14,15  Of further concern is that trainees who are the least competent are also the least able to self-assess competently.1214  Additionally, residents may mistakenly believe that less intern supervision is needed, perhaps because they may not fully appreciate the need for supervision or because they may have an interest in achieving autonomy as soon as possible.16,17  It is critical that pediatric hospitalists are aware of the inaccuracies of trainee self-assessment. Specifically, pediatric hospitalists should rely on direct observation in place of intern self-assessment when making entrustment decisions.

An alternative explanation for the discrepancies between hospitalist and resident ratings is that pediatric hospitalists may mistakenly believe that more intensive intern supervision is needed than is in fact the case, perhaps out of an inability to adequately evaluate trainees and a desire to ensure the safest possible care in the face of uncertainty.18  It should be noted that the EPA framework was designed with the intention of having an observer assess a single trainee in a clinical context and, on the basis of those observations, make a decision about entrustment.11  In contrast, in this survey we asked pediatric hospitalists to rate how much supervision the average trainee needs. Therefore, pediatric hospitalists are forced to reflect on all of their workplace observations of interns over many years. It is likely that specific intern interactions may unduly influence their perceptions, a form of availability bias. For example, if a pediatric hospitalist were to work with a struggling new intern in the week before completing this survey, they might mistakenly report that the average intern needs more supervision than they actually do need. Pediatric hospitalists should be aware of this potential bias and make decisions about entrustment based on direct observation rather than recent experiences.

Despite its limitations, the disparity between hospitalists’ and residents’ perceptions has some important implications for the organization of care systems on inpatient units. For example, 72% of residents self-reported that they could give or receive a patient handover to transition care responsibility at the beginning of their intern year without direct supervision, as compared with only 44% of hospitalists. Researchers in a previous study have demonstrated that implementation of a supervised group handover was associated with reduced medical errors.19  Therefore, if interns do, in fact, perform handoffs independently or with only moderate supervision, then adverse patient outcomes could result. Reassuringly, however, there is agreement between residents and pediatric hospitalists regarding some of the other high-stakes activities (eg, entering and discussing orders and prescriptions).

Interestingly, there was little difference in the perceived need for supervision of new interns between residents across PGY levels. One may have hypothesized that as residents progressed in training, they may have changed their self-perceptions on their need for supervision at the beginning of their intern year; however, this occurred to only a limited degree. The stability in answers across PGY levels also suggests that PGY3 residents, who are further away from the beginning of their intern year and could theoretically be at risk for recall bias, may in fact recall how much supervision they perceived they needed at the start of internship reasonably accurately.

There are several limitations of this study. First, the response rate was modest, and there may be a nonresponse bias. Additionally, we did not specify in the surveys who would be supervising the new interns (ie, residents, pediatric hospitalists, or both), and therefore the responses were based on the interpretation of the respondents. It is possible that residents might have assumed this supervision would be by residents, whereas hospitalists assumed it would be by attending physicians. We also did not define some of the terms used in the survey (eg, “average intern” and “moderate supervision”), and therefore these terms were subject to interpretation. In addition, pediatric hospitalists were asked to rate the amount of supervision that the average intern needs at the beginning of the intern year, whereas residents were asked to rate how much supervision they themselves needed at the beginning of their intern year. It is possible that residents might either under- or overestimate how much supervision they themselves needed as compared with their peers. Additionally, this study was performed in a single large pediatric program, and therefore its results may not be generalizable across other hospitals. Lastly, shifting expectations regarding intern competence and the evolution of hospitalists’ clinical expertise could affect how ratings were provided.

With further preparation at the end of medical school or the onset of internship, we believe that it will be possible to narrow the gap between the current and desired independence of graduating medical students and new interns. We are encouraged by new fourth-year medical school boot camps and other curricula that aim to better prepare new interns.20  In addition, residency programs could consider implementing intensive training programs for incoming interns on their arrival.21  Such training programs would permit residency program leadership and pediatric hospitalists to simultaneously assess new residents’ capabilities and adjust the levels of individual supervision accordingly. Given pediatric hospitalists’ unique opportunity to assess new interns and tailor their supervision on the basis of learner competency, we should offer faculty development regarding intern assessment and feedback. By better preparing our new interns to enter residency and by improving pediatric hospitalists’ abilities to tailor supervision, we can better align intern and attending physician expectations. It is essential that residents and hospitalist physicians openly discuss these issues so that interns can be supervised appropriately and patients can receive optimal care.

The authors thank all of the residents and faculty who worked closely with us and participated in this study.

FUNDING: Supported by a Boston Children’s Hospital residency funding grant titled the Fred Lovejoy Resident Research and Education Award.

Dr Winn conceptualized and designed the study, designed the data collection instruments, coordinated and supervised data collection at all sites, and drafted the initial manuscript; Dr Marcus helped conceptualize and design the study, helped design the data collection instruments, drafted portions of the initial manuscript, and reviewed and revised the manuscript; Drs Sectish and Landrigan helped conceptualize and design the study and reviewed and revised the manuscript; Ms Williams conducted the statistical analyses and reviewed and revised the manuscript; and all authors approved the final manuscript as submitted.

1
Barzansky
B
,
Etzel
SI
.
Medical schools in the United States, 2011-2012
.
JAMA
.
2012
;
308
(
21
):
2257
2263
2
Barzansky
B
,
Etzel
SI
.
Medical schools in the United States, 2012-2013
.
JAMA
.
2013
;
310
(
21
):
2319
2327
3
Barzansky
B
,
Etzel
SI
.
Medical schools in the United States, 2013-2014
.
JAMA
.
2014
;
312
(
22
):
2419
2426
4
Barzansky
B
,
Etzel
SI
.
Medical schools in the United States, 2014-2015
.
JAMA
.
2015
;
314
(
22
):
2426
2435
5
Barzansky
B
,
Etzel
SI
.
Medical schools in the United States, 2015-2016
.
JAMA
.
2016
;
316
(
21
):
2283
2290
6
De Oliveira
GS
 Jr
,
Rahmani
R
,
Fitzgerald
PC
,
Chang
R
,
McCarthy
RJ
.
The association between frequency of self-reported medical errors and anesthesia trainee supervision: a survey of United States anesthesiology residents-in-training
.
Anesth Analg
.
2013
;
116
(
4
):
892
897
7
Baldwin
DC
 Jr
,
Daugherty
SR
,
Ryan
PM
.
How residents view their clinical supervision: a reanalysis of classic national survey data
.
J Grad Med Educ
.
2010
;
2
(
1
):
37
45
8
Farnan
JM
,
Petty
LA
,
Georgitis
E
, et al
.
A systematic review: the effect of clinical supervision on patient and residency education outcomes
.
Acad Med
.
2012
;
87
(
4
):
428
442
9
Lyss-Lerman
P
,
Teherani
A
,
Aagaard
E
,
Loeser
H
,
Cooke
M
,
Harper
GM
.
What training is needed in the fourth year of medical school? Views of residency program directors
.
Acad Med
.
2009
;
84
(
7
):
823
829
10
Association of American Medical Colleges
.
Core entrustable professional activities for entering residency. 2013. https://www.mededportal.org/icollaborative/resource/887. Accessed January 15, 2016
.
11
ten Cate
O
,
Scheele
F
.
Competency-based postgraduate training: can we bridge the gap between theory and clinical practice?
Acad Med
.
2007
;
82
(
6
):
542
547
12
Davis
DA
,
Mazmanian
PE
,
Fordis
M
,
Van Harrison
R
,
Thorpe
KE
,
Perrier
L
.
Accuracy of physician self-assessment compared with observed measures of competence: a systematic review
.
JAMA
.
2006
;
296
(
9
):
1094
1102
13
Evans
AW
,
Aghabeigi
B
,
Leeson
R
,
O’Sullivan
C
,
Eliahoo
J
.
Are we really as good as we think we are?
Ann R Coll Surg Engl
.
2002
;
84
(
1
):
54
56
14
Woolliscroft
JO
,
TenHaken
J
,
Smith
J
,
Calhoun
JG
.
Medical students’ clinical self-assessments: comparisons with external measures of performance and the students’ self-assessments of overall performance and effort
.
Acad Med
.
1993
;
68
(
4
):
285
294
15
Antonelli
MA
.
Accuracy of second-year medical students’ self-assessment of clinical skills
.
Acad Med
.
1997
;
72
(
10
,
suppl 1
):
S63
S65
16
Biondi
EA
,
Varade
WS
,
Garfunkel
LC
, et al
.
Discordance between resident and faculty perceptions of resident autonomy: can self-determination theory help interpret differences and guide strategies for bridging the divide?
Acad Med
.
2015
;
90
(
4
):
462
471
17
Meyerson
SL
,
Teitelbaum
EN
,
George
BC
,
Schuller
MC
,
DaRosa
DA
,
Fryer
JP
.
Defining the autonomy gap: when expectations do not meet reality in the operating room
.
J Surg Educ
.
2014
;
71
(
6
):
e64
e72
18
Sterkenburg
A
,
Barach
P
,
Kalkman
C
,
Gielen
M
,
ten Cate
O
.
When do supervising physicians decide to entrust residents with unsupervised tasks?
Acad Med
.
2010
;
85
(
9
):
1408
1417
19
Starmer
AJ
,
Landrigan
CP
;
I-PASS Study Group
.
Changes in medical errors with a handoff program
.
N Engl J Med
.
2015
;
372
(
5
):
490
491
20
Englander
R
,
Flynn
T
,
Call
S
, et al
.
Toward defining the foundation of the MD degree: core entrustable professional activities for entering residency
.
Acad Med
.
2016
;
91
(
10
):
1352
1358
21
Winn
AS
,
Marcus
CH
,
Williams
K
, et al
.
Development, implementation and assessment of the Intensive Clinical Orientation for Residents (ICOR) curriculum: a pilot intervention to improve intern clinical preparedness [published online ahead of print November 24, 2017]
.
Acad Pediatr
.
doi:10.1016/j.acap.2017.11.004

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: Dr Landrigan has served as a paid consultant to Virgin Pulse to help develop a sleep and health program. He is supported in part by the Children’s Hospital Association for his work as an executive council member of the Pediatric Research in Inpatient Settings network. In addition, Dr Landrigan has received monetary awards, honoraria, and travel reimbursement from multiple academic and professional organizations for teaching and consulting on sleep deprivation, physician performance, handoffs, and safety, and he has served as an expert witness in cases regarding patient safety and sleep deprivation. Dr Sectish is a consultant to the I-PASS Institute and has equity in the company. He has also received monetary awards, honoraria, and travel reimbursement from multiple academic organizations for his work on patient safety research. We do not believe these activities to be conflicts of interest for this study. Drs Winn, Marcus, and Williams have indicated they have no financial relationships relevant to this article to disclose.