Common Data Element Collection in Underserved School Communities: Challenges and Recommendations Free

Using a de novo questionnaire with fixed-response and open-ended questions, we conducted a cross-sectional qualitative and quantitative evaluation of mandatory CDE collection among RADx-UP RTS project teams with differing priority populations and from varying geographic locations within the United States. Projects awarded under RADx-UP RTS were also required to collect the 153 CDE questions spanning 13 domains: (1) consent; (2) location; (3) sociodemographic information; (4) housing, employment, and insurance; (5) work personal protective equipment and distancing; (6) medical history; (7) health status; (8) vaccine acceptance; (9) testing (to be referred to as “previous testing experiences”); (10) COVID-19 test (to be referred to as “administered COVID-19 test”); (11) symptoms; (12) alcohol and tobacco; and (13) identity (Table 1 contains an overview of each of the CDE domains and Supplemental Table 4 contains the questions asked within each domain). Of these 13 domains, 1 domain (“consent”) was related to both testing and nontesting activities, 2 domains (“administered COVID-19 test” and “symptoms”) were related to the testing initiative for which each project was funded, whereas the other 10 required domains were not related to current testing administration.

We created a questionnaire to gather information on how nontesting CDE collection was conducted by RADx-UP RTS project teams. Information collected included projects’ target demographics, whether project teams applied for and obtained CDE exemptions from the NIH, community member involvement in CDE survey construction (if applicable), and the number of project study aims that included CDE collection. Furthermore, we asked whether CDE collection was fully coupled, fully uncoupled, or partially coupled from the RTS project’s SARS-CoV-2 testing initiative based on whether participants were required to complete CDE questions only at the time of enrollment and accessing SARS-CoV-2 testing (fully coupled), at a time separate from and/or independent from testing enrollment (fully uncoupled), or through a combination of both approaches (partially coupled).

Additionally, we collected data on whether incentives were offered to participants for completion of nontesting CDEs, the proportion of CDE questions completed by projects’ participants, the racial and ethnic distribution of those who completed CDE questions, and the percent of missing data by CDE domain. Participant race and Hispanic ethnicity were evaluated as separate and independent variables. Self-reported race for each participant was categorized as American Indian or Alaska Native, Asian, Black, Native Hawaiian or Other Pacific Islander, White, Other, Multiple, or Prefer Not to Answer. Hispanic ethnicity was categorized as participants who reported identifying as Hispanic, Latino, or Spanish origin (binary: “yes” or “no”). RTS project teams calculated the percent of missing data for each CDE domain as the number of unanswered domain questions across all project participants divided by the total number of domain-related questions asked to all project participants. Missing data reporting by RTS project teams was optional to accommodate project teams that had not yet started analysis and were therefore unable to report percent missingness. Last, we used open-ended questions to collect qualitative information on project-level experiences with CDE collection including challenges, lessons learned, and recommendations for future CDE collection.

Study teams for each of the 16 RADx-UP–funded RTS projects were invited to complete the questionnaire. Responses were recorded using research electronic data capture (REDCap).^9,10  No individual participant-level data from research participants within each project were shared across institutions and only aggregate (counts, proportions), deidentified data were collected via REDCap.

This study was approved by the Duke University Health System institutional review board under Pro00108129.

Descriptive statistics were used to analyze quantitative results. Counts, proportions, and trends were used to summarize categorical data, whereas 5-number summaries (median, quartiles, range) were used to summarize discrete, numerical data. To assess the racial and ethnic representativeness of participants completing CDE questions relative to participants enrolled in project-level testing initiatives, we compared the proportion of participants in each group stratified by self-reported race and ethnicity. To assess the distribution of missing data among the CDE domains, we determined the median percentage of missing data for each domain across RTS projects.

Next, we stratified and compared both participant racial/ethnic representativeness and missingness by whether each project’s study aim collecting CDE questions was fully coupled, partially coupled, or fully uncoupled from the project’s testing initiative. Then, to assess the role of monetary compensation on both participant racial/ethnic representativeness and data missingness, we stratified both analyses by whether an incentive was offered for CDE participation. Last, we examined data missingness stratified by whether study aims modified the standard language of CDE questions before providing them to participants (eg, modified language to fit specific populations such as the child participant). Statistical significance testing was deferred because of the small sample size and lack of power.

For qualitative analysis, content analysis was used to describe lessons learned and key challenges, as well as recommendations for future data collection across projects.¹¹  Because of the exploratory nature of the lessons learned, key challenges, and recommendations across projects, thematic categories were not defined a priori, and, instead, salient themes were identified and grouped, resulting in posteriori definitions/recommendations. Representative quotes were highlighted to provide context. Qualitative and quantitative data were analyzed separately.

Thirteen (13) of the 16 (81%) total RTS-awarded project teams participated and provided responses to the study questionnaire. These 13 teams were located in 11 states (Fig 1).¹²  The CDE collection process varied across projects. All project teams (13/13, 100%) reported use of an electronic platform (REDCap, Qualtrics, or KoBo) for CDE collection and 6 teams (6/13, 46%) offered additional paper-collection options. Eight (8/13, 62%) project teams offered CDE questions in languages other than English (8 in Spanish and 1 in Haitian Creole). Eleven (11/13, 85%) project teams collected self-reported data, 11 (11/13, 85%) collected caregiver-reported, 6 (6/13, 46%) collected teacher-reported, and 1 (1/13, 8%) collected school-reported data. Eleven projects completed reporting for mechanisms used to provide survey reminders: 5 project teams (5/11; 45%) used text messages reminders, 5 (5/11; 45%) used phone call reminders, 7 (7/11; 63%) used e-mail reminders, and 3 (3/11; 27%) used in-person reminders at testing visits. Eight (8/13; 62%) project teams collected nontesting CDEs from participants at 1 timepoint and 5 teams collected CDE data from participants more than once. Among those project teams that collected nontesting CDE data more than once, the number of times participants were surveyed ranged from 2 to 4 times with 2 project teams asking the same questions each time, 2 project teams asking different questions each time, and 1 project team not specifying whether the same or different questions were asked each time. Among the projects that collected CDE at multiple times, 1 project team specified that it collected a “mandatory” initial survey and filtered more sensitive questions to subsequent “optional” survey(s).

There were 15 study aims across the 13 participating RTS projects. Of the 15 aims, 7 (47%) were structured such that CDEs were fully uncoupled from the testing initiative, 4 (27%) were fully coupled with the testing initiative, and 4 (27%) had data collections that were partially coupled with testing. In 9 (60%) study aims, participant incentives were provided in the form of monetary compensation for participation in CDE collection. Compensation amounts ranged from $10 to $50 with differing payment approaches (e.g., 1-time payment versus tiered payment approach). Eight (8, 53%) study aims reported modifying CDE questions before use.

Project teams completed optional “missing data” reporting for 10 study aims (10/15; 66%); of these 10, 2 (20%) aims were fully coupled, 2 (20%) were partially coupled, and 6 (60%) were fully uncoupled. Seven (7, 70%) of the aims reporting missing data provided financial incentives to participants and 5 (50%) modified the CDE questions before use. For specific information on question exemptions and modifications, please see the Supplemental Results.

Across all participating projects, there were no notable differences in the racial or ethnic distribution of those participating in the testing initiatives versus CDE collection (Table 2). Overall, White participants comprised 47% of both CDE participants and testing initiative participants; Black participants comprised 23% of CDE participants and 25% of testing participants. The remaining racial categories (American Indian or Alaska Native, Asian, Native Hawaiian or Other Pacific Islander, Other, Multiple, or Prefer Not to Answer) comprised 1% to 3% of CDE and testing participants. Ethnically, Hispanic participants accounted for 11% of both CDE participants and testing initiative participants. See the Supplemental Results for specific information on the role of coupling status and compensation on racial and ethnic distribution.

Of those who reported missing data within study aims, the median percent of missing data ranged from 4% to 21% across nontesting CDE domains (Table 3). CDE domains with the greatest median percentages of missing data were: location (21%; interquartile range [IQR], 3% to 40%); identity (18%; IQR, 3% to 80%); housing, employment, and insurance (15%; IQR, 3% to 79%); and health status (15%; IQR, 4% to 49%). Domains with the least missing data were alcohol and tobacco (4%; IQR, 3% to 10%); previous testing experiences (7%; IQR, 2% to 55%); medical history (8%; IQR, 4% to 31%); and work personal protective equipment and distancing (8%; IQR, 1% to 51%). See the Supplemental Results for specific information on the role of coupling status, compensation, and question modification on missing data.

Project teams identified several lessons learned from their CDE collection experiences. First, teams qualitatively noted that among participants who were offered the CDE questions, CDE completion rates were low, given that many individuals elected not to participate in CDE collection and those who did participate often did not complete their CDE questionnaires. Project teams hypothesized that possible reasons for low CDE completion rates were (1) the amount of time needed to complete the CDE questions and (2) participant skepticism stemming from unclear understanding of how and why nontesting data were being collected and used.

Second, among the project teams who provided incentives for CDE collection, completion was variable, and project teams reported a range of participant engagement in cases in which incentives were offered. Some projects noted that incentives were “essential” or “instrumental in achieving a [given] response rate,” whereas others noted that “not everyone will complete the CDEs despite an associated incentive.”

Finally, project teams that fully uncoupled CDE collection from testing noted that they felt the distinction was a necessary step to prevent CDE questions from being a barrier to increasing access to testing. For the 4 RTS projects that engaged community partners in selecting and editing CDE questions before broad distribution, feedback from community partners reinforced the challenges and concerns about data privacy and invasiveness identified by researchers.

By requiring funded projects to collect more than 150 CDE questions per participant as part of project execution, RADx-UP envisioned creating a large data repository representative of perspectives from historically underrepresented research populations as a way to potentially resolve questions that could inform health disparities reduction efforts via community-engaged approaches. Nevertheless, our examination of CDE collection among RTS projects suggests that additional steps are needed to better operationalize this goal. Our results showed that each RTS project facilitated data collection using a variety of strategies to promote community comfort and reduce perceived barriers to testing access. There was minimal variation in the racial and ethnic distribution of CDE survey participants from those who participated in testing, but fully uncoupling CDE questions from testing increased the proportion of Black and Hispanic individuals participating in both initiatives. Furthermore, across all projects, the percent of missing data was greatest among nontesting CDE domains that sought to collect potentially personal, sensitive, or identifiable information; however, the amount of missing data for these domains was more limited when financial incentives were offered.

RADx-UP RTS projects identified highly heterogeneous methods for implementation of CDE collection. Reasons for this heterogeneity may be related to the principles of community-engaged research¹³  in which RTS projects were responsive to local context and used approaches to CDE collection that were informed by community experiences.¹⁴  The tailored data collection strategies essential for community-partnered research precludes a unified, 1-size-fits-all approach to CDE collection, but also highlights the importance of uniform questions and answers for collected data elements to ensure meaningful data aggregation across projects.¹⁵  A 2018 retrospective analysis¹⁶  of a real-world dataset developed as part of a CDE investigation found that of the 1414 CDEs collected across 426 clinical studies with a genomics focus, only 32 “truly common” CDEs were identified, demonstrating limited data harmonization. Although this study focused primarily on identifying CDEs across genetic data repositories, resulting in an abundance of heterogeneity across nearly 25 000 variables, the majority of “truly common” CDEs identified included sociodemographic (age, race, gender) or anthropometric (height, weight) data rather than novel data collected between topic-related projects.

At the project level, there was minimal variation in the racial and ethnic distribution of CDE survey participants from those who participated in testing. Nonetheless, when stratifying project study aims by coupling status, the skew in racial representativeness toward White participants among aims that fully or partially coupled CDE questions with testing highlights the possibility that coupling CDE collection to testing initiatives inadvertently served as a barrier to SARS-CoV-2 testing participation. Possible reasons for this finding may be related to distrust within communities that have been historically underrepresented or harmed by historical research efforts. Distrust is a commonly cited barrier to research engagement among participants from underrepresented backgrounds^17,18 ; linking testing initiative enrollment to mandatory data collection could further perceptions of exposure to unnecessary risk¹⁸  and contribute to decisions to forgo involvement all together.

Alternatively, it is possible that our findings related to racial and ethnic distribution and coupling status are reflective of project teams’ proactive decision-making to uncouple testing and CDE collection because of anticipated community hesitation or recommendations provided by consulting community members. Given that most of the sites were in partnership with community-based schools, the level of detail required through the CDEs could have been out of character for typical information shared in these settings and perhaps made participants feel that research was being conducted “on” them, rather than “with” and “for” them.

There were also concerns about data amount, quality, and utility. Participants enrolled in most of the projects were required to complete 153 questions. One previous study demonstrated that survey completion rates substantially decrease (from 63% to 37%) when survey lengths increase from 13 to 72 questions, respectively, suggesting that limiting the number of survey questions is associated with increased response rate and reliability.¹⁹  Researchers investigating respondent preferences across 700+ participants found that the ideal length of time to complete a survey is roughly 10 minutes, whereas 20 minutes is the maximum recommended duration.¹⁶  The large quantities of missing CDE data also introduce worries about systematic nonresponse and the potential for biased conclusions, thereby potentially harming underserved populations that often battle stereotypes and systematic discrimination based on false narratives. Furthermore, the large quantities of missing data may limit the ability to meaningfully combine data across projects and raises questions about the utility of such efforts. Finally, one should use caution when interpreting data from adult-oriented CDEs in predominantly pediatric populations. Given that the original CDEs were adult focused, issues around parent-proxy respondents, mismatches between question content and developmental age, and the practicalities of data management with redundant variables for parent and child CDE data when collected as dyads, all could have contributed to our findings around data completion, exemption requests, decisions around coupling, and ultimately data quality.

Monetary compensation did not impact participant willingness to complete CDE questions among the majority of RTS participants. We did not identify major differences in the total amount of missing data between study aims that included and did not include incentives, but when stratifying missing data by CDE domain within each aim, the percent of missing data was profoundly lower in study aims where project teams used incentives compared with those that did not. This could indicate that participants felt adequately compensated for their time. In contrast to previous work that demonstrated increased racial diversity among survey participants when compensation was offered,¹⁹  we found no major differences in racial representativeness of participants at either level of incentive offering; however, we found increased representation of those identifying as Hispanic when an incentive was offered.

Our study has several limitations. First, the sample size of 13 participating project teams is small; therefore, views and recommendations presented here may not be representative of the range of RTS projects or of other RADx-UP CDE collection efforts outside of the subset of participating RTS projects. Second, given time restraints and ethical considerations, the questionnaire used to survey RTS project teams collected aggregate data that were self-reported by each RTS project team rather than individual-level data across all projects. Furthermore, project teams were only able to comment qualitatively on their achieved CDE completion rates. Data collection strategies varied greatly by teams and within project aims, and, as a result, not all teams were able to specify the total number of individuals who received their CDE questions. Therefore, we lacked the denominator needed to calculate completion rates. Finally, this study describes only a subset of community members willing to participate in the RTS projects and may not be representative of each community as a whole. Despite these limitations, we were able to survey the majority of RTS project teams, summarize CDE collection efforts within K-12 school communities across a wide geographic area, and aggregate data across heterogenous projects to describe overarching trends.

RTS project teams recommended the following for future data collection: (1) provide clear communication about data collection goals; (2) separate nontesting and testing CDE collection to decrease testing access barriers; (3) connect data collection to the project objective in a meaningful way if it is essential that data be collected in conjunction with a community-engaged project; (4) prioritize questions so fewer, higher quality questions are asked; and (5) increase community involvement at all stages of question development.

We must continue to critically examine and improve CDE collection efforts within programs designed to increase representation of underrepresented populations. Failing to improve CDE collection efforts and engage community stakeholders in an intentional and meaningful way may be harmful to individuals from underrepresented groups^20,21  and may risk extrapolating findings that incorrectly inform future research efforts. Therefore, collaboration with underrepresented populations from the early study design process may improve interest and participation in CDE collection efforts and may help address ongoing mistrust issues related to health research data collection in marginalized communities.

The authors thank the families, school employees, Tribal Nations, and other community members for taking the time to participate in this study. The authors acknowledge the sacrifices, stress, and difficult decisions that our participants and many others have had to make to support the safe growth and development of children in the midst of the global pandemic. The project would like to acknowledge the investigators, staff, students, and trainees who worked tirelessly to complete these projects. The research team is grateful for the time and support of our community partners and research participants. Erin Campbell, MS, provided editorial review and submission of this manuscript but did not receive compensation for her contributions, apart from her employment at the institution where this study was conducted.

CDE

common data element

IQR

interquartile range

preK-12

prekindergarten through grade 12

NIH

National Institutes of Health

RADx-UP

Rapid Acceleration of Diagnostics- Underserved Populations

REDCap

research electronic data capture

RTS

Return to School (initiative)