Establishing a Pediatric Health Equity, Diversity, and Inclusion Research Review Process Free

In recent years, there has been an increasing interest in equity, diversity, and inclusivity (EDI) research,^1 ,2  which has facilitated welcome changes in the research ecosystem, such as modifications to funding mechanisms. EDI research can be defined as research that positions equity at the core of health outcomes, considers multiple interacting spheres of influence (systems of power, relationships and networks, individual factors, physiologic pathways), and incorporates historical and life-course perspectives.³  High-quality EDI research requires expertise, skill, and experience. However, researchers may lack expertise in applying a health equity framework^3 –10  to research design, conduct, analysis, and reporting, resulting in poorly designed studies that may lead to further harm to underrepresented and underserved communities.¹¹  Key components in high quality health equity research include the following: (1) a conceptual model to inform the study hypothesis,^8 ,9 ,12  (2) research question and design that do not perpetuate bias,^13 ,14  (3) input about study question and design from patients and community members with lived experience,^4 ,5 ,15  (4) participant engagement activities that include staff with lived experience,^5 ,8 ,15  (5) intentional recruitment of diverse populations,¹⁶  (6) analytic methods that appropriately consider subpopulations (eg, reference group selection, address missing data, small sample size, oversampling),^{4 ,13 ,15 ,17}  and (7) language use that conforms to current health equity standards.¹⁸  For example, many researchers have proposed study outcomes that are stratified by race and ethnicity or other EDI-related characteristics, or include these characteristics as primary predictors in multivariable regression models without identifying a framework or conceptual model to explain the relationship.¹²  The absence of such a framework may lead to problematic assumptions about biological differences. Without a proper grounding in theories relevant to EDI work, researchers may unknowingly perpetuate race-based medicine (the mischaracterization of race as an essential biological variable relevant to clinical decision-making) when their focus should be on race-conscious medicine (considering racism rather than race as the social determinant of health).^19 ,20 

In response to a marked increase in EDI research, the Boston Children’s Hospital Institutional Board Review (IRB) noted the need for adequate review of these research protocols and collaborated with the Boston Children’s Hospital Office of Health Equity and Inclusion (OHEI) to develop a supplemental EDI review process that was embedded within the existing IRB review process. The overall purpose of this review process was to improve the quality of EDI research conducted at Boston Children’s Hospital, including through assessment of research design, methods, recruitment and representativeness, and dissemination with a focus on improving representativeness by actively engaging, recruiting, and retaining historically underrepresented and marginalized individuals and communities. Below we describe the development, implementation, and initial evaluation of the EDI Research Review Process.

Starting in 2020, the OHEI began to receive a high volume of requests from multiple sources for ad-hoc review of EDI-related research, surveys, and quality improvement and educational projects. In response, the health equity research experts in the OHEI collaborated with the hospital’s quality improvement experts and researchers in the Program for Patient Safety and Quality, along with other researchers and subject matter experts from across the hospital, to form the Health Equity Quality Improvement and Research Committee. This committee reviewed EDI-related research, quality improvement, and educational projects to make certain that the projects aligned with the Boston Children’s Hospital Declaration on EDI.²¹ 

As awareness evolved about researchers’ barriers and needs, and the volume of projects submitted for review increased, the OHEI partnered with the Boston Children’s Hospital IRB to form the EDI Research Review Committee. Initially implemented in January 2022, this innovative EDI research review process was modeled operationally after other IRB-adjacent review processes, such as departmental scientific review before IRB submission or pharmacy ancillary review after initial IRB submission for studies that involve the use of medication (to ensure safety protocols are in place for patients and employees). Many studies at the hospital require at least 1 adjacent review because of the nature of the research conducted. EDI research review was felt to be an important addition to existing reviews to ensure that EDI research was conducted in ways that did not further existing health disparities and inequities. Although there was little existing literature available regarding a dedicated EDI research review process, a recent article by Meloney et al suggested IRBs have a responsibility to ensure inclusivity and diversity in research and recommended specific ways in which IRBs can modify current research review processes to support EDI in clinical trials.²² 

Given the long history of racism and bias in biomedical research,²  this EDI research review process was designed to translate ethical considerations specific to EDI into research methodology. For example, researchers were provided with guidance to support recruitment of diverse populations, which is both methodologically sound and, from an ethical standpoint, allows the benefits of research studies to be shared across a broader population. Specifically, the EDI research review process aimed to ensure (1) human subjects EDI-related research protocols use appropriate methodology and are aligned with the Boston Children’s Hospital Declaration on EDI²¹  and (2) clinical research at Boston Children’s Hospital considers health equity research principles.^{4 ,5 ,8 ,9 ,12 –15 ,17 ,18}  After it was formalized, the EDI research review process was communicated to the Boston Children’s Hospital research community via an e-mail announcement from the IRB.

The primary component of the EDI research review process was the EDI Research Review Committee (EDI RRC). The EDI RRC provided education and recommendations (factors suggested by the committee that could improve their protocol) and/or requirements (factors that must be addressed before final IRB review) for study teams before the IRB review. The committee was comprised of 11 faculty, staff, and employees with EDI expertise (eg, expertise in community-engaged research; health equity research; quality improvement; and research with underrepresented groups, such as sexual and gender minorities), including researchers, clinicians, data analysts, and legal representatives. Most committee members were previously part of the Health Equity Quality Improvement and Research Committee. All committee members were identified as representing different types of EDI expertise and appointed to the committee. Committee membership was intentionally diverse to ensure multiple perspectives, lived experiences, and types of professional expertise were represented, and expertise included health equity research, social determinants of health, biostatistics, adolescent medicine, sexual and gender minority health, EDI-related measurement, and legal considerations (Fig 1). We were especially intentional in creating a committee membership that was inclusive of individuals with diverse lived experiences, including from populations that have been historically underrepresented in biomedical research, medically underserved communities, and historically marginalized populations. As some types of research conducted at the hospital include employees as study participants, the inclusion of colleagues from the Boston Children’s Hospital Office of General Counsel was important for ensuring that employees (including trainees) were protected as a special population.

The EDI RRC worked closely with the IRB to establish criteria for EDI review (Table 1), which were used by the IRB to determine which human subjects research protocols came to the EDI RRC. Once a protocol was determined to meet these criteria, the IRB referred the study team to the OHEI, who communicated with the study team about the steps of the EDI research review process. This systematic approach to identifying protocols for review was critical to ensuring appropriate protocols were referred to EDI RRC. Principal investigators could also self-refer before IRB submission. The EDI review had 2 main workflows (Fig 2), which were determined by the type of protocol. EDI research protocols that included a primary data collection component (defined as the collection of original data) were reviewed by the full EDI RRC. An example of a primary data protocol that was reviewed by the full committee is one that examined the racial and socioeconomic inequities in the delayed diagnosis of a medical condition. Protocols that were limited to secondary data analysis (defined as analyzing data that were previously collected, including electronic medical record reviews) were reviewed by a subcommittee of 3 members from the full EDI RRC. One such protocol examined the association between the Childhood Opportunity Index and major adverse events in outcomes of a medical procedure. Research protocols for multi-institutional projects with BCH as the single IRB were reviewed by the committee; but when the primary IRB was from an institution outside of BCH, these studies were exempt from review. The full committee met virtually via Zoom 2 to 4 times monthly, and the subcommittee met virtually via Zoom weekly to review the protocols. Typically, 2 protocols were reviewed at each full committee or subcommittee meeting, and up to 16 protocols (4–8 primary data collection and 8 secondary data analyses) were reviewed each month. Protocols assigned to a meeting were reviewed by all committee members attending that meeting. To assist study teams in considering EDI during the study design, an EDI Research Checklist for Investigators was developed and later updated to reflect the committee’s experience (Supplemental Fig 3).

For all types of protocols, the study team completed a prereview form (also since updated, see Supplemental Fig 4) describing the study with an emphasis on the EDI-related aspects of the study design. For primary data collection protocols, at least 1 representative from the study team with knowledge about the EDI-related aspects of the project (Principal Investigator or another team member) was invited to attend a meeting of the full EDI RRC and present their project, focusing on the EDI-related aspects. This was followed by a question and answer period in which committee members’ questions focused on providing methodological feedback and review of alignment with the Boston Children’s Hospital EDI Declaration.²¹  After the question and answer period, the committee met without the study team present to discuss recommendations and requirements. Once the feedback was finalized, the study team was invited back to the virtual meeting, where the requirements and/or recommendations were shared with them. Within a week, the study team received a letter confirming the EDI RRC review and detailing the committee requirements and/or recommendations. For the secondary data analysis protocols, the study team was not required to attend a meeting. Instead, the subcommittee reviewed these protocols at weekly meetings and provided a similar letter to the study team with requirements and/or recommendations.

For all types of protocols that underwent an EDI review, all committee requirements needed to be incorporated into the protocol before IRB submission. For committee recommendations, the study team either needed to incorporate them into the study protocol or justify why they did not accept the recommendations. The EDI RRC letter and the study team’s responses were submitted to the IRB as part of the protocol submission materials. The IRB then reviewed whether requirements were incorporated and whether recommendations were included or there was an acceptable justification for why they were not. The IRB was responsible for the final determination.

The EDI research review process was evaluated after the first 20 months of implementation by examining the number and types of research protocols reviewed, and the types of requirements and recommendations provided to research teams. The analytical cohort for this study included all primary and secondary data collection research protocols with an EDI component submitted directly to the EDI RRC by the study team before IRB submission and/or referred by the IRB from January 2022 to August 2023. The OHEI maintained a dataset of all reviewed protocols, including the date of initial contact, Principal Investigator name, department or division, protocol title, review type (primary or secondary data review), date of review, date of the final letter sent to the Principal Investigator, and a multicolumn narrative of requirements and recommendations provided to the research team by the committee. Narrative requirements and recommendations were aggregated into common themes and described by the research stages they pertained to, including design, execution, analysis, and dissemination. Quantitative results are presented as frequencies and percentages for categorical variables and median (IQR) for the number of days from EDI review to recommendation letter dissemination. All analyses were performed using SAS version 9.4.

From January 2022 to the end of August 2023, 78 research protocols were reviewed by the EDI RRC (Table 2). Forty-five (57.7%) protocols involved primary data collection and went through a full committee review, whereas 33 (42.3%) protocols limited to secondary analysis of existing data went through a subcommittee review. The number of protocols reviewed fluctuated substantially over the study period, ranging from 1 to 14 in any given month, with an average of 4 protocols per month. The volume of EDI-related research protocols exceeded our initial expectations, necessitating more frequent review meetings to provide timely feedback to study teams.

The median number of days from the EDI RRC review meeting to the requirement or recommendation letter being sent out was 5 days (IQR 1.0–8.0) (Table 2). This turnaround time differed by review type, with primary data collection review protocols taking a median of 6 days (IQR 5.0–9.5) and secondary review protocols taking a median of 1 day (IQR1.0–2.0) (Table 2).

The protocols reviewed by the EDI RRC represented a heterogeneous group of departments and divisions across the hospital. One-third came from 3 departments and divisions: general pediatrics (n = 10), cardiology (n = 9), and adolescent medicine (n = 8) (data not shown). Eleven (14.1%) protocols that underwent review were approved as submitted with no requirements or recommendations provided by the EDI RRC. Seven primary data collection protocols received requirements, whereas none of the secondary data analysis protocols were given requirements (Table 2). Of the 67 protocols that received recommendations to improve EDI-related aspects of the research study, 36 involved primary data collection, whereas 29 involved secondary data collection (Table 2). Recommendations were made more frequently than required changes. Recurrent EDI RRC recommendation and requirement themes are presented in Table 3. One-third of the protocols received requirements or recommendations to ensure that the research was grounded in a conceptual model or theoretical framework to limit the possibility that the study design, execution, analysis, and/or dissemination of findings would inadvertently perpetuate existing bias (Table 3). Twenty percent of protocols were given requirements or recommendations to address missing EDI data or to follow appropriate guidelines on the collection and reporting of race, ethnicity, and language data (Table 3).

In our EDI review process, recommendations and/or requirements to improve the EDI-related aspects of research studies were made more than 85% of the time. As a result, standard recommendations for conducting EDI research were developed, many of which are summarized in the checklist for investigators (Supplemental Fig 3). One frequent recommendation was that the protocol should use an EDI conceptual model or framework that centers structural and systemic barriers (30%). Several health equity frameworks exist, although they may need to be adapted to support specific research approaches.^{3 –10 ,23 –26}  Equity-conscious research requires acknowledgment that structural and systemic barriers are key determinants of health-related outcomes. Without the context of structural and systemic barriers, hypotheses involving race or other EDI-related characteristics as primary predictors risk perpetuating bias,^{2 ,12 ,20 ,27}  or leading to erroneous conclusions that racial and ethnic inequities can be attributed to genetic differences.

We recommended enhanced community engagement for more than 10% of the reviewed protocols. Community members can be engaged in research in several ways, including, but not limited to, as members of the primary research team (eg, coinvestigator or research assistant), through partnerships with research teams where community members are key stakeholders, and through participation in community advisory boards. Community members can provide valuable input on many aspects of the research process, from recruitment strategies to interpretation of research findings. When community members are not involved, research can perpetuate harm by creating opportunities for research participation that are unaffirming or even offensive (eg, responding to a survey item with outdated language), furthering negative stereotypes about a particular community, or causing harm through the ways in which research findings are presented and ultimately disseminated.

The EDI RRC also provided recommendations to study teams to standardize the measures used to collect data on race and ethnicity, sexual orientation and gender identity, and other sociodemographic characteristics. Measures used to assess EDI-related characteristics are often outdated and contain language that may be perceived by study participants as unaffirming and/or even inaccurate in describing their identities and experiences. Thus, it is important to use current terminology and follow standard approaches that have been established for measuring EDI-related characteristics. Several health equity language and data collection standards exist, and researchers are encouraged to use them.^18 ,28 

Finally, missing data were a concern in one-third of the protocols reviewed by the EDI RRC. A high percentage of missing race and ethnicity, and other sociodemographic data will produce biased results, except when data are missing completely at random or when the chosen statistical model appropriately accounts for patterns of missingness. Additionally, we suggested that research teams should ensure sufficient statistical power to enable subgroup comparisons based on EDI characteristics.

Evaluation of this EDI review of human subjects research protocols at a pediatric academic medical center demonstrated that the majority of protocols (67 of 78) referred to the EDI RRC for review benefitted from EDI-related recommendations for improvement. Recommendations applied to various stages of the research process, such as identifying an EDI framework during the design phase, using best practices for measurement of EDI-related constructs during the execution phase, accounting for missingness among EDI-related variables during the analytic phase, and involving community members in the collection, interpretation, and dissemination of research findings. The results of this evaluation support the need for EDI research review as well as the need for investigators to receive guidance on best practices for conducting EDI research. The committee was able to offer both feedback and guidance on EDI research by translating ethical considerations into methodology and improving the quality of EDI research being done across the institution. The improvement of EDI research quality was demonstrated when research teams returned to the committee with another project that reflected feedback that the committee had provided on a previous project, such as using updated terminology to refer to specific populations, expanding the languages in a project beyond English, or consulting with a community advisory board.

Several challenges and limitations were identified during the development and implementation of the EDI research review. First, this new and innovative process was developed without the aid of existing literature to guide its development. Departmental scientific reviews, during which studies are evaluated for scientific rigor before IRB submission, may be considered an analogous process, but the EDI-focused research review is novel. A recently published paper provided IRBs with recommendations to promote diversity and inclusion in intervention studies.²²  The authors suggest a range of IRB activities to support inclusion, such as acknowledgment of IRB responsibility in promoting EDI, ensuring recruitment of diverse populations and community engagement, and supporting diverse IRB membership.²²  Although this paper was published after the EDI research review process was implemented, several of the authors’ recommendations were already included in the EDI review process.²² 

Second, because of the urgent need to begin this review process, the process was operationalized while still under development. This led to some reluctance from researchers because the expectations for EDI research review were initially unclear, underwent revisions during the development process, and created delays in obtaining approval to begin research projects. Early on, there was a backlog of research protocols requiring review, necessitating extra committee meetings to ensure the timeliness of review.

Third, EDI review requires both time and commitment from committee members, many of whom are uncompensated for this service. Although the importance of EDI work is increasingly acknowledged, this work may be undervalued.²⁹  In addition, EDI work is often done by historically marginalized or underrepresented individuals who are subject to the “minority tax,” such that they are disadvantaged by working on EDI initiatives that are uncompensated and not typically considered a metric of academic success.³⁰  Fortunately, the hospital provided protected time for some EDI RRC committee members, including the chairs. However, the inability to provide protected time for other committee members has created challenges in finding individuals who have available protected administrative or academic time to serve in this capacity.²⁸ 

Additionally, it was observed early in the process that the members of the EDI RRC who represented the legal department had important considerations that clinical and research members of the EDI RRC might be unaware of or may not have expertise in. For example, these members identified the need to provide extra protections and processes for studies proposing to use employees as study participants, underscoring the importance of legal representation on the committee to provide this expertise. This emphasizes the importance of recognizing the value of diverse perspectives and differing expertise within the committee, and in needing to take time not only to educate research and nonresearch committee members about one another’s approach, necessary priorities, and concerns but also to create space for ongoing crossdisciplinary dialogue.

Finally, it is important to recognize that this review process was developed at a pediatric academic medical center that has prioritized EDI initiatives.²¹  It may not work in the same way at other institutions with different resources, levels of EDI expertise, and/or prioritization of EDI initiatives as they relate to research.

For other institutions that hope to develop a similar EDI research review process, it is critical to obtain commitment from institutional leaders, who can prioritize and provide resources for this review process. Close partnerships between the IRB staff and EDI RRC are essential, as the process cannot proceed without IRB leadership. Communicating with the institution’s research community about EDI research review early in the process is recommended to allow research teams to plan for this supplemental review. As institutional leaders and some members of the research community may not have EDI expertise, it may be necessary to articulate why EDI research, as well as review of EDI research, are necessary. Recruiting committee members for the EDI RRC who represent a diversity of experiences and perspectives, including representation from the legal department and/or other related departments, is highly recommended. It may be necessary to provide additional explanation of research processes for committee members with less research experience. If possible, committee members and others involved in the EDI research review process should be compensated through protected administrative and/or academic time.

Assessing the EDI needs of the research community is recommended, as it will allow the alignment of training and resources to support research teams in developing EDI protocols. Research teams may also need coaching during the EDI research review, particularly researchers without formal EDI training, so that they understand why changes to their projects are needed and how they can improve their research. Finally, the development of an EDI research review is likely to be a learning process for all involved, as the level of researcher EDI knowledge is revealed and as the committee recommendations evolve to address the variety of EDI-related considerations that arise during the review process.

EDI research review is critically important to ensure the use of appropriate EDI methodology and address ethical considerations specific to EDI research. EDI research must be conducted with an awareness of the influence of potential biases and thoughtfulness about the methods used to ensure that this research does not contribute to health inequities. A supplemental EDI review can provide a process to constructively examine and provide feedback on EDI research to ensure that it is standardized and follows recommendations.

We thank Kali McCollister and Ashley B. Tartarilla, MPH for their administrative assistance.

AMC

Academic Medical Center

EDI

equity, diversity, and inclusion

IRB

Institutional Review Board

OHEI

Office of Health Equity and Inclusion

RRC

Research Review Committee