Defining Global Health Tracks for Pediatric Residencies

To arrive at a definition, we chose a Delphi process framework¹⁹ using consensus-building iterative surveys for 3 rounds.^20,21 Inclusion criteria for an expert panel included expertise in pediatric GH education demonstrated through leadership of a GH residency track, experience in GH curriculum development, and scholarly output, including peer-reviewed publications or presentations on pediatric GH education. Authors of this article were excluded from the expert panel. The authors (members of the Association of Pediatric Program Directors GH Learning Community) each sent a list of potential expert panelists to the principal investigator (PI), who compiled them into a pool of 32 potential participants, of whom 28 met inclusion criteria.

From this pool, the PI stratified potential panelists on the basis of variables we believed could significantly impact responses based on differences reported in the previous survey of self-defined GH track leadership.⁵ These variables included size of the residency program, size of the GH track, and whether the GH track was multidisciplinary. The PI then constructed a purposeful sample of 12 potential panelists representing distribution among the criteria of interest, with preference given to those who had authored peer-reviewed publications about GH education. The PI sent those individuals an e-mail invitation to voluntarily participate, including a detailed description of the study and notice that survey participation implied consent. Nine participants immediately accepted. For the 3 who declined, additional names were pulled from the same categories, and those 3 accepted.

Consensus research suggests that panels smaller than 6 or larger than 12 have more limited reliability.¹² The author group selected a panel size of 12 per those recommendations and for adequate inclusion of individuals representing residency programs and tracks with different criteria of interest.

The final panel included at least 5 panelists from large programs (≥60 residents), at least 5 panelists from small to medium programs (<60 residents), at least 3 panelists from multidisciplinary tracks versus tracks that were specific to pediatrics or medicine-pediatrics, and at least 3 panelists from small GH tracks (<10 total residents in the track). Panelists agreed to a confidentiality clause, thus ensuring independent responses.

The author team developed a literature-based survey focusing on GH track definitions,^17,18 minimal requirements for educational offerings,^{4,6,10,–12,18,22,–24} institutional supports,^3,18,22 and outcomes for evaluation^25,26 (Supplemental Information). The online survey was piloted by multiple author team members before dissemination.

We first proposed an adapted definition of a GH track (objective 1) for panelists to consider.¹⁷ For objective 2 (determining minimal educational offerings and supports required of a GH track), we provided common educational components (eg, predeparture orientation,^1,4,5,18,27 international electives^1,6,28,29) and structural supports (eg, GH track leadership,^1,3,5,18 coordinator support^3,5,18) from the literature as suggestions as well as free text options. Similarly, for objective 3 (outcomes to measure the effectiveness of a GH track), we provided possible outcomes of GH tracks from existing literature that were organized on the basis of Kirkpatrick’s levels (reaction, learning,³⁰ behaviors,³¹ and results^24,26,32).

The PI’s hospital institutional review board deemed the study exempt from review. The survey was distributed via e-mail and administered by using Qualtrics online surveys (Qualtrics, Provo, UT). For the track definition, we asked panelists to accept, reject, accept with changes, or suggest an alternative, with open text for comments to support their choice or recommendations for change. For the other 2 objectives, we asked panelists to rank the items as essential, very important, somewhat important, or not important (4-point Likert scale) and add open-text comments to justify their ratings or propose alternate wording. On the basis of consensus research literature, we defined consensus as at least 75% agreement for questions asking for agreement or inclusion or a mean score of <1.5 or >3.5 for the questions with 4-point Likert scales.^20,21 Items with mean scores >3.5 were defined as areas of positive certainty and ultimately were included as essential, whereas items with mean scores <1.5 were areas of negative certainty. Average rankings of 1.5 to 3.5 were considered areas lacking consensus and were further explored in subsequent questionnaires for the 3 rounds of the Delphi process. Round 1 took place between November 17 and December 6, 2017; round 2 was between January 17 and February 12, 2018; and round 3 was between February 28 and March 19, 2018.

The PI and research assistant analyzed results from each round and presented deidentified data to the coauthors. For areas that lacked consensus, we analyzed response comments and, where applicable, revised items to reflect modal suggestions for changes that would increase likelihood of consensus in the next round. If no modal modifications were identified, then respondent explanations as to why the option or item was acceptable or essential were summarized and shared in the following round. We presented overall response rate and quantitative results (mean and bar graph) and deidentified comments to the panelists for rounds 2 and 3, generating additional questions when applicable, on the basis of qualitative responses from the previous surveys. Panelists were asked to review each question and consider altering their responses on the basis of the modifications and/or shared data, although they were assured that they need not conform. Panelists who did not agree with the modal response category in rounds 2 and 3 were again asked to explain why, and these explanations were used to refine the response category and move toward consensus.

For data analysis, we calculated frequencies with means, medians, and ranges of Likert scale responses. Three reviewers with experience in qualitative analysis (H.H., A.B., and a nonauthor research assistant) independently reviewed all qualitative data, developed a coding scheme, and conducted independent coding. Using an inductive content analysis approach,³³ each reviewer analyzed the data to identify emergent themes and select representative quotations.

All of the invited panelists (n = 12) agreed to participate in the Delphi process, and we achieved 100% participation for all 3 rounds.

The Delphi process generated significant discussion around the term “global health” and required all 3 rounds to obtain consensus on terminology to define a GH rotation. Although some initially expressed that a GH rotation should involve crossing an international border, ultimately a broader definition was accepted. “Our local consensus is that GH is transnational by definition, and therefore [local-global] experiences are, if framed correctly, truly GH experiences.” On the basis of such comments, all items were amended in round 2 to use the term “global health” rather than “international,” unless referring specifically to a cross-border international experience or partnership.

After 2 rounds, the panelists achieved positive certainty on the definition of a GH track, provided in Fig 1.

Panelists identified with positive certainty 7 essential components of a GH track, with 2 offerings described in the literature deemed not essential but important (Fig 2). Panelists unanimously agreed that formal educational infrastructure to support GH rotations is essential, including predeparture preparation, preceptorship during GH electives, and postreturn debriefing. Panelists recognized scholarly output as an essential component but indicated a wide range of acceptable scholarly activity. Panelists also indicated that scholarly projects should align with host site priorities and highlighted a need for flexibility in scholarship requirements because some projects may need to be adapted once a resident is on site.

After 2 rounds, there were 2 items that did not achieve consensus for being essential components of a GH track: local or domestic activities in GH and a continuity clinic with an underserved local population. Panelists recognized that without clear definitions of “local-global” or underserved populations, these terms would need to be standardized before being required components of GH tracks. There was agreement that certain domestic experiences could fulfill the GH rotation requirement of the GH track, but panelists did not feel that all GH track residents must complete a domestic GH experience. Panelists also expressed that a continuity clinic in an underserved setting could be appealing to GH track residents but should not be a requirement.

The panel was asked to comment on whether the AAP’s suggested 4-week minimum for a GH elective⁶ was appropriate, too short, or too long. The panel agreed with positive certainty that the 4-week minimum was appropriate. One panelist described an “educational development process” that unfolds during a GH rotation and warned that if “shortened by a rotation of less than 4 weeks, [residents are] less likely to achieve clinical, cultural, and system proficiency.” Several panelists commented that a longer duration would be optimal but recognized scheduling constraints. As one panelist stated, “4 weeks should be the minimum ‘sandals on the ground’ time and not be cut into by leisure travel during or at the tail end of the rotation.”

The panelists reached consensus on a number of supports deemed necessary to ensure the success of a GH track (Table 1). After 2 rounds, panelists agreed that a dedicated GH track director with salary support or protected academic time is necessary “to grow and develop a program.” Consensus was not obtained, however, on the degree of support required in the percentage of full-time effort for the GH track director. In addition to dedicated time, panelists also endorsed financial support for faculty professional development, partnership development, and travel. Finally, administrative support for the GH track was noted to be “imperative,” although the size of the residency program would dictate the amount of support.

Resident salary support during GH electives; maintenance of existing malpractice, health, and disability insurances; and the provision of evacuation insurance were all unanimously agreed on as essential during round 1. Panelists believed that all were “important safeguards that should not be compromised.”

To maximize sustainability and collaboration, panelists believed that at least 1 established partnership should exist, either by the residency program’s institution or in collaboration with other institutions or organizations in which the resident is performing the GH elective. Although most agreed that some form of on-ground support is essential, the requirement was considered “too prescriptive” and potentially burdensome to the host site partners.

For round 1 responses for objective 3, the initial use of a Likert scale failed to adequately differentiate the most useful track outcomes. For subsequent rounds, the response choice was amended to binomial choices to include or exclude the item in a GH track evaluation tool, with the same definition of consensus as in objective 1.

Panelists agreed on several outcome measures that would effectively serve as quality indicators of a GH track within a residency program (Table 2). Many of these items measured resident perceptions about their experiences and personal impact of the track. Panelists agreed that partner perceptions about the benefits and challenges of hosting track residents at the local institution would be helpful so as to “help decrease burden and increase benefit to [the] host institution in order to grow a partnership.”

A number of learning outcomes were also selected as important quality indicators, including cumulative assessment of the track residents’ improvement in GH knowledge and attitudes, track resident self-evaluation of the ability to recognize GH-related diagnoses, and cumulative achievement of competency-based milestones. International faculty assessment of track residents participating in an international elective was unanimously selected as an important outcome.

All panelists believed cumulative scholarly work produced by track residents should be evaluated because this could serve as a “distinguishing characteristic” of the track. Long-term outcomes of residents, such as career choice after graduation, were deemed essential to evaluate the impact of GH tracks. International and domestic partner evaluation of local community and institutional impact of the GH track was also unanimously agreed on; as 1 panelist commented, “If partnerships are not mutually beneficial then they are doomed to failure.” Finally, to ensure that residents are safe during their GH rotations, panelists agreed that adverse health and safety events should routinely be reported and addressed, so as to mitigate risk to future residents.

Several themes emerged throughout the survey rounds, many of which influenced the panel of experts’ rank and rating of items. These themes included consideration of program size, partnership and bidirectionality, and standardization of various components.

Panelists were cognizant of the impact of recommendations on small programs. For example, although agreeing that mentorship was an important defining factor, they acknowledged that smaller programs might have difficulty identifying faculty members with adequate GH experience to serve as mentors. Panelists suggested that members from more resourced programs could mentor residents in smaller programs. Likewise, although panelists strongly believed that a GH director with salary and administrative support were “essential for ensuring a robust and thriving GH track,” many felt that this may not be a reality for smaller programs with fewer overall institutional supports and that given the number of residents participating in the track, “effort required may be less for the GH track director in these programs.” Finally, panelists conceded that although having an established partnership is ideal, this may be less feasible for small programs and that “reasonable alternatives include collaborating with other programs that have more robust GH tracks or having a group of training programs share international sites when possible.” Other suggestions included collaborating with other departments within the home institution.

Many panelists considered the impact on the partnering institution when endorsing certain recommendations. Many felt that while it is not yet feasible for all institutions, establishing “reciprocal educational arrangements where track residents also host visiting learners, preferably from the same longitudinal site,” is an ideal model for institutions. Panelists felt that this bidirectionality was important to avoid a sense of paternalism. Having an established partnership was thought to be beneficial for both domestic and international locations for “ease of communications, familiarity with processes, and long-lasting impacts.” When reflecting on the importance of partnership perception, one panelist summarized that it is important to “[couch] partnerships in terms of mutual benefit, equitability, and reciprocity.”

While recognizing this process as an opportunity to establish minimum educational requirements, panelists also articulated the importance of standardizing the suggested requirements. For example, to ensure high-quality predeparture training, “basic standards are needed to ensure [that] predeparture preparation is. . .adequate prior to international rotations.” Likewise, although a longitudinal curriculum is considered the backbone of the track, many advocated for a clear way to “achieve a standard of knowledge,” even suggesting a standard set of specific readings, modules to complete, or criteria to fulfill, established by various governing bodies (eg, American Board of Pediatrics, AAP, and Association of Pediatric Program Directors).

Although GH tracks have become common in pediatric residency programs, lack of agreed-on standardization or consensus definition has created challenges for educators, researchers, and trainees.^1,5 The results of this study offer an expert consensus definition (Fig 1) as well as several essential components, supports, and outcome measures (Fig 2; Tables 1 and 2) that will be useful for stakeholders in GH education. Although others have proposed models for GH curricula, which include suggested definitions based on their program description,^11,17 the Delphi methodology used here offers a more rigorous approach to determine expert consensus.

GH curriculum remains a “supplemental” component of residency training because there are not Accreditation Council for Graduate Medical Education mandates pertinent to GH training. However, trainee demand,^1,4,34 paired with the desire for residency programs to train globally competent pediatricians,^{3,23,29,35,–37} warrant scrutiny over GH educational offerings nationally. This study offers an “internal policing” of sorts for GH educators as an attempt to set standards within our own community that will offer a firm foundation on which programs can grow, share resources, and track outcomes. The definition and core components identified in this study offer residency programs a platform on which to build their GH education philosophies. Rather than serving as a checklist, the 7 core components of GH tracks identified in this study can serve as a springboard for discussion for building more robust programs. This study also highlights creative strategies that small residency programs might leverage to achieve the core components. Additionally, residency applicants can consider these components as they evaluate residency programs that feature GH tracks.

Several of the factors deemed essential for GH tracks affirmed suggested best practices published elsewhere, such as the need for comprehensive predeparture training,^4,18,22 mentorship,^18,38 and minimum duration of GH electives.⁶ The study also offered insights to the importance of building GH tracks around sound partnerships with institutions and organizations in resource-limited settings, in line with recent literature.^39,40 Postreturn debriefing was recognized by panelists as essential, not only to help residents process their experiences but also as a form of quality improvement for the rotation and the partnership with the host site. Three panelists commented on the importance of using the debrief as a way to also screen for posttraumatic stress disorder or other mental health symptoms and to refer for counseling as indicated. Although some have written about their programs’ expectations of a scholarly project as part of their GH tracks,^5,24 the panel reached consensus that this is an essential component of the definition.

There are inherent limitations to the Delphi methodology. The expert panel was limited to GH educators working in pediatrics in the United States and did not explore the generalizability of these results to other specialties or other countries. The author team chose a panel of 12 panelists on the basis of literature about Delphi processes,²¹ but the limited number of panelists and lack of face-to-face conversation may have limited discussion. Although care was taken that drafting of the initial survey was done on the basis of existing literature, and iterative feedback to respondents was done objectively, there remains risk that we unintentionally guided the panel in the survey design.

On the basis of consensus of expert opinion, we propose an accepted definition of pediatric GH tracks as “a longitudinal area of concentration dedicated to global child health, offered within a residency program, which includes a formal curriculum and mentorship with required scholarly output for a defined cohort of pediatric residents.” We recommend that residency programs who have GH tracks strive to offer at minimum the institutional supports and educational opportunities described here, including adequate faculty and institutional resources to support GH track residents through immersive rotations of at least a 4-week duration, with mentorship and partnerships to support scholarly work that is appropriate for their field site. Several areas for future study were identified by the panel, including a need for standard tools for assessments of trainees in GH-specific knowledge, skills, and attitudes and to evaluate the efficacy of GH tracks in preparing resident participants to make meaningful contributions to improve global child health.

We appreciate contributions to data collection and quantitative analysis by Yonina Frim, contributions to qualitative analysis by Kerry Coughlin-Wells, guidance on Delphi methods by David N. Williams, and the thoughtful participation of the panelists.

AAP

American Academy of Pediatrics

GH

global health

PI

principal investigator