Feedback Culture Perceived by Trainees in an Academic Institution: A Mixed Methods Study

A mixed methods design was conducted using triangulation of findings from quantitative and qualitative methods to study feedback culture perceived by trainees.¹⁷  In this design, the study was deemed exempt by the university’s institutional review board.

All Pediatric fellows and residents at a tertiary-care academic institution in the northeastern United States were invited to anonymously complete a survey using Qualtrics. The survey consisted of questions on (1) demographics such as gender, race, ethnicity, and post graduate year (PGY) level; and (2) the modified FES. We included race and ethnicity data in our survey and analyses to understand whether perception of feedback culture may be different among minority trainees as they experience several challenges in negotiating their personal and professional identities in the medical setting while being seen as “other.”¹⁸  Race and ethnicity were self-reported.

The original FES has 32 total items with a 7-point Likert scale ranging from strongly disagree to strongly agree.¹⁶  It is scored by calculating the average score for the items in each construct. It measures 7 aspects (constructs) of the feedback environment, including source credibility, feedback quality, feedback delivery, reinforcing feedback, constructive feedback, source availability, and promotion of feedback seeking. In the industrial setting, validity is supported by internal consistency and relationship to other variables.¹⁶  Furthermore, a predetermined factor structure was confirmed on factor analysis.

We made minor modifications to the FES (Supplement 1), such as substituting “job performance” with “clinical performance” and “supervisor” with “attending” to enhance applicability. We also removed certain items from each construct to reduce redundancy and to make the survey more applicable in a medical setting. The number of constructs were kept the same (Supplement 1). Answers such as, “The feedback I receive from my supervisor helps me do my job,” were removed as we were concerned that the modifications that would be required to make the question applicable in a medical setting would substantially change their meaning. Modifications to the survey were made by our research team that consisted of faculty members with expertise in medical education, assessment, and educational research. The modified survey consisted of a total of 20 items distributed within 7 constructs. All constructs contained 3 to 4 items except for reinforcing feedback, which contained 1 item. The FES was used to assess and compare the feedback culture between our residency and fellowship programs. In addition, we used this instrument to understand the differences in perception of feedback culture among trainees of different gender, race, and ethnicity as such data have not been previously elucidated. The survey was piloted among 5 pediatric residents and revised based on their feedback. The pilot data were not included in the final study and no additional questions were removed between the pilot and the final modified FES.

Descriptive statistics, such as sample size and percentage, were used to describe demographics. Fisher’s exact test was used to compare demographic data of trainees who participated in the study versus nonrespondents. Fellow and resident survey ratings were compared using 2-sided Fisher’s exact tests. To adjust for confounding variables, multivariate regression analyses were performed for each construct using a linear regression model with backward selection and a significance level-to-stay of P value = .05. For each parsimonious model in multivariate analysis, initial variables included were self-identified gender, race, ethnicity, pediatric training program, and PGY-level. SAS 9.4 was used for all quantitative analyses.

Using a phenomenological design,¹⁹  semistructured, 1-on-1 interviews were conducted to explore resident experience, with a particular focus on trying to understand barriers to feedback. Residents were targeted for the qualitative study because of specific concerns about the feedback environment that both they and the residency leadership raised. Feedback was defined as comments given during a clinical learning experience for the purposes of ongoing learning and improvement. Interviews were conducted by 4 interviewers, all of whom received formal education in qualitative research techniques. Only1 of the interviewers was part of the research team. The remaining interviewers were independent from the study team and were students in the “Principles of Qualitative Research” course at the authors’ primary institution. The codes and transcripts were generated by these 4 interviewers and then shared anonymously with the rest of the research team. All participants provided verbal consent to participate in the study and were made aware that their responses may be shared with departmental faculty and published, in an anonymous manner.

Interviews were conducted using a semistructured interview guide and were audiotaped and professionally transcribed. Semistructured interview guides were created after discussion between study personnel to ensure that each participant was asked similar questions. Participants were chosen using purposive sampling along the axes of post graduate year (PGY) of training and self-identified gender to achieve a representative mix of residents. Invitations for participation were sent via e-mail.

Data were analyzed using a constant comparative method²⁰  in which essential concepts from interview data were coded and compared over successive interviews to extract recurrent themes. The transcripts were divided among pairs of investigators for review. In each pair, the reviewers independently reviewed and coded transcripts, then met together to reach consensus on codes. Then, the pairs met together to compare and contrast findings to generate final codes, with which transcripts were recoded as needed. The pairs also categorized codes to develop and reach consensus on themes. Saturation of themes was noted in our interviews. Dedoose© (https://www.dedoose.com/) was used to organize and manage transcript data and guide analysis.

Fifty-two of 80 residents and 21 of 45 fellows completed the survey, representing response rates of 65% and 47%, respectively. Trainee demographics of study participants versus study nonparticipants are shown in Table 1. African American trainees (P = .05) and residents in the internal medicine-pediatrics program (P = .02) were underrepresented when comparing those who completed the survey to nonrespondents. In the bivariate analyses, fellows scored significantly higher in the constructs of source credibility, feedback quality, reinforcing feedback, constructive feedback, source availability, and promotion of feedback seeking (Fig 1) but not feedback delivery (5.9 vs 5.9, P = .75). The significant differences between the residents and fellows persisted in the multivariable analyses (Table 2). In the models, the β coefficients show the linear relationships in the construct scores between the residents and fellows. In the “reinforcing feedback” construct, for example, fellows would be expected to score 0.64 points higher, on average, than the residents. In addition, Hispanic ethnicity and female gender were negatively associated with source credibility (P = .04) and constructive feedback (P = .03), respectively (Table 2). Race and PGY level of training were not associated with any construct. Statistically significant results are reported in Table 2.

Semistructured interviews were conducted with 8 resident physicians (7 pediatric residents, 1 combined medicine-pediatric resident). Two residents were in the third year, 2 in the second year, and 3 in the first year of their training in pediatrics. The combined internal medicine-pediatric resident was in the third year of training. There were 4 males and 4 females. Residents described several barriers to the process of feedback during their interviews. Two main themes emerged from these data:

The challenge of receiving and seeking feedback in a busy inpatient clinical environment was a prominent theme in all interviews. Participants described time given to feedback was insufficient because of the demands of clinical volume and the need to multitask in the clinical environment. Clinical patient care was identified as the priority and created several downstream effects on feedback. Participants described an environment where feedback and clinical care were regarded as distinct entities rather than feedback being an integral and essential component of patient care. All residents reported situations where efficient patient care often compromised feedback causing it to be either abbreviated or completely neglected (Table 3).

Furthermore, the prioritization of patient care magnified differences and barriers related to feedback between the pediatric subspecialists and hospitalists in an inpatient setting. Subspecialists were often perceived as being removed from the feedback environment because of competing clinical responsibilities and were unable to spend meaningful time with the residents. This resulted in difficulties in seeking feedback by the residents and credibility, acceptance, and receptiveness to feedback. Participants described how subspecialists gave nonspecific (eg, “great job”) feedback, which was perceived as far too general and, thus, less meaningful. Lack of longitudinal exposure also did not allow the residents to showcase their “growth” following feedback. This issue with feedback from subspecialists was, however, thought to be more a system issue rather than a personality or individual issue (Table 3). Residents explained that working on the overnight shift (“night float”) reinforced their practice of “signing-out” without getting feedback on clinical care decisions that they made overnight (Table 3).

Another theme that emerged through resident interviews was the barrier created from a culture of courtesy. Attending physicians were described as friendly and approachable but avoided constructive comments in their feedback. This was thought to be related to a fear of hurting the recipients’ feelings and maintaining a positive nurturing environment. This perceived culture created a barrier for feedback specificity and details, which the participants felt were important for their progress. It also resulted in several “you are doing great” type comments. This deficiency in constructive feedback was especially magnified when the culture of courtesy and patient care priority intersected (Table 3).

All participants felt that although constructive feedback was limited, it was crucial for their progress. A culture of courtesy prevailed among residents where seeking feedback from an attending was perceived as an imposition, especially when the attending was a busy subspecialist (Table 3).

This culture also seemed to impact the feedback residents gave to their peers. There was concern that constructive feedback would create a negative emotional state in the recipient (Table 3). A participant felt that having to “work day and night” and “work so heavily with coresidents” was a reason for avoiding constructive feedback as it could compromise their relationship. Another participant stated that rather than in-person feedback, written feedback might be a better way to give constructive comments (Table 3).

We have described the feedback culture perceived by our pediatric trainees in our department and have demonstrated a less favorable perception by residents as compared with fellows. Patient-care efficiency pressures and a culture that prioritized courtesy compromised the quality and quantity of feedback given to residents, including feedback seeking behaviors and constructive feedback. This highlighted the differences in feedback trainees received from the subspecialists as compared with the hospitalists.

Although our FES data identified perceived feedback culture between the fellows and residents, our mixed methods analysis was able to triangulate data to further identify specific actionable items to improve perceived feedback culture in the residency program. Feedback was affected when efficiency in patient care took priority. Time limitations in a demanding clinical environment are not surprising and have often been identified as a feedback barrier.^1,10  Our study demonstrated that time constraints especially compromised a feedback dialogue between subspecialists and residents. Residents also felt the effects of this priority in patient care during busy rotations during overnight shifts where they felt they did not receive adequate feedback. Faculty and trainee education in learning ways to efficiently provide and seek feedback, respectively, in busy clinical settings should be considered.^12,21,22 

Lack of time and the inability to form longitudinal relationships with subspecialists may be more of an issue for residents than fellows in our institution. Residents typically do not have consistent exposure to subspecialists, especially in an inpatient setting. Subspecialist schedules can sometimes result in different attending physicians providing patient care on a day-to-day basis. In addition, subspecialists may have conflicting demands in an outpatient location or with procedures t can abbreviate the time spent in the inpatient environment where feedback occurs. In contrast, pediatric fellows, fewer in number than residents, often have longitudinal exposure to subspecialists and work closely with them during the 3 years of their training. It could be hypothesized that this lack of longitudinal exposure between residents and subspecialists could play a role in our results. Recently, measures such as restructuring rotations to prevent frequent resident turnover and organizing teams to allow for continuity with subspecialists have been implemented in our program. Additionally, departmental leadership support to minimize conflicting clinical demands on the on-call subspecialist (eg, cancelling morning clinics or procedures) and to avoid frequent cross-coverage have been instituted. We plan to collect follow up data to understand whether such structural changes might improve the process of feedback between residents and subspecialists. Further studies to also explore the impact of whether senior learners, such as fellows, might be better at obtaining feedback or have more opportunities to do so would be beneficial.

Our study also revealed a feedback culture of courtesy over candor, where attending physicians and residents were thought to be approachable and friendly. This courteous environment, unfortunately, was perceived as a barrier for honest constructive feedback even though the residents viewed the attending physicians in a positive way. A similar “culture of niceness” was previously described by Ramani et al¹⁰  during focus groups conducted with residents in the internal medicine residency program. Our results are consistent with the politeness theory that defines politeness as social skills that ensure self-affirmation for those engaged in social interaction.^23–26  Further studies to understand this phenomenon of courtesy and the barriers to giving frank and helpful feedback might be helpful in identifying targeted areas for improvement. Interactive feedback sessions for faculty to practice difficult feedback conversations could be considered. Additionally, studies to learn the faculty perspective and whether concern over feedback seeming too harsh or whether fear of retaliation from the trainee could be playing a role in this culture of courtesy may be important. We found that female trainees felt that they received significantly less constructive comments when compared with their male counterparts. This aligns with previous qualitative data by Mueller et al that describes how female residents received less consistent constructive comments from faculty as compared with their male counterparts.¹¹  This inconsistency was particularly apparent across issues of autonomy and leadership, traits that are traditionally considered more masculine than feminine. Although our data did not allow for evaluation of the reasons for a difference in constructive feedback between males and females, these preliminary findings are noteworthy. Firstly, inadequate constructive feedback would mean inferior feedback for female trainees compared with male trainees with downstream impact on their performance. Secondly, our data raise the concern for gender bias in beliefs held by faculty. Further studies would be helpful in clarifying the significance of our findings and preventing any deleterious effects on the education of our female trainees.

Hispanic trainees had a less favorable perception of faculty expertise and trustworthiness. A recent qualitative study that was conducted across 21 residency programs across the Unites States showed that minority residents experience daily microaggressions, bias, and challenges negotiating their professional and personal identity while seen as “other.”¹⁸  It is possible that these additional burdens felt by minority trainees could impact the trust felt in their relationships with attending physicians. Minority trainees’ experience with the feedback may be different and exploring their perceptions of the feedback culture is critical in mitigating the adverse consequences. Faculty and trainee education on issues related to diversity, equity and inclusion, and implicit bias is crucial.

Our research has several limitations. We used the FES in a new setting and with new learners. Hence, it is unclear that the validity evidence collected in the industrial setting is applicable and can be extended to its use in a medical education setting. However, modifications were made in a deliberate process that involved faculty with expertise in survey design. We are presently conducting a study to gather validity evidence for this modified tool.

Trainees could have been thinking of specific attending physicians or rotations when responding to the survey, introducing the potential for recall bias. We tried to minimize this possibility by specifically choosing the FES as it prompts the trainees to think about their overall feedback experiences. We had a small sample size for our semistructured interviews, which may not have assured representation of the entire group of residents in our program. This limitation is somewhat offset by the fact that we selected a purposive sample based on PGY and gender to provide a level of representation in the sample along these axes. There may be other factors that would have assured further representation that we did not consider. The study was performed in a single institution, and we had a less than fifty percent response rate among fellows, limiting generalizability. In addition, African American trainees were underrepresented in our study, which may have contributed to the fact that we did not find an association between race and feedback constructs. Conversely, we noticed differences in feedback constructs based on ethnicity and gender and it is possible that these findings were type I errors given the small sample size. Further larger prospective studies would be helpful in understanding and confirming these findings.

In our study, the FES was used in a novel clinical setting. It was able to detect differences in the perceived feedback cultures of residents and fellows, as well as suggest differences based on gender and ethnicity, which would allow identification of actionable changes to be implemented and the ability to monitor their effect. Our work highlights the importance of educating both trainees and faculty about the importance of constructive feedback and the need for systematic changes in resident rotation structures to optimize continuity with subspecialists.