Health System Research Priorities for Children and Youth With Special Health Care Needs

We adapted RAM to set priorities for a CYSHCN research agenda (Fig 1). We used the following terms and processes. First, we developed a set of research topics from responses to an open-ended survey and published literature. Next, stakeholders evaluated every research topic according to 3 different domains, that is, the research topic’s perceived (1) need and urgency, (2) research impact, and (3) family centeredness. During a moderated expert panel, stakeholders gave every research topic 3 ratings (ie, 1 score in each domain). Finally, by using the median ratings of the 3 domains, each research topic was assigned into a rank order reflecting the research topic’s priority from 1 (high) to 4 (low). The Colorado Multiple Institutional Review Board considered this work exempt from human subjects review.

Between February 2018 and March 2018, we sought input from professionally, demographically, and geographically diverse stakeholders. Members of stakeholder organizations identified by the CYSHCNet Executive Committee (Supplemental Information) were invited to complete an open-ended electronic questionnaire (Qualtrics, Provo, UT) developed and pilot tested with CYSHCNet family leaders. In the questionnaire, respondents identified “one or two things that they would most like to see improved in the approach to health for CYSHCN,” and “what research with or about CYSHCN they would like to see in a national research agenda.” Participants had 2 weeks to reply, after which nonresponders received up to 2 reminders at 1-week intervals. Although an individual could be a member of >1 stakeholder organization, they were asked to complete the survey only once.

Surveys from 269 respondents yielded 2835 responses, which 3 team members coded, edited, and combined into 86 unique research topics through an iterative, consensus-seeking approach. We used conventional inductive content analysis²²  to identify key themes emerging during synthesis and aggregation of the CYSHCN research topics. Study team members reviewed the final list of research topics and themes to ensure representation of all submitted ideas.

Although a systematic literature review was not feasible because of the breadth of the topic of health systems for CYSHCN, stakeholder organizations were asked to submit articles perceived to guide future CYSHCN research directions. The research team updated the final list of research topics on the basis of a review of the submitted articles.

Members of the CYSHCNet Executive, Steering, and Advisory Committees (each of which included family members of CYSHCN), the network’s MCHB Division of Research project officers, and members of the MCHB services branch for CYSHCN were invited to participate in the RAM expert panel. The personal and professional characteristics of the 24 RAM panelists are summarized in Table 1. Three RAM participants were coauthors (D.Z.K., K.K., and C.G.H.) to ensure that the interpretation and presentation of the results was consistent and balanced from clinical, research, and family participant perspectives. Panelists received rating materials and a written summary of the research topic development that included the qualitative analysis of survey responses and the articles submitted by stakeholders.

During 2 rounds, in March 2018 and April 2018, panelists rated the degree to which each research topic represented 3 domains: need and urgency, research impact, and family centeredness. The need and urgency domain was defined as the influence the research topic had on the health and well-being of CYSHCN, the severity of consequences (eg, number of CYSHCN affected by the issue, mortality risk, severity of symptoms, or functional loss), and the presence of critical gaps in knowledge to solve the problem. Research impact was defined as the ability of the knowledge gained from this research topic to translate to meaningful progress in 5 to 10 years and the potential to lead to improvements in health status, quality of care, public health, or policy over the longer-term. Family centeredness was defined as how well the research topic reflected the experiences and priorities of most CYSHCN and their families.

Using a 9-point Likert scale, panelists gave each research topic 3 ratings (1 rating in each of the 3 domains). A rating of 1 to 3 indicated low representation, a rating of 4 to 6 indicated uncertain representation, and a rating of 7 to 9 indicated high representation of a domain. Consistent with RAM convention, disagreement was defined if one-third or more of panelists rated an item in the low range (ie, 1–3), whereas one-third or more of panelists rated the same item in the high range (ie, 7–9). Panelists could include comments to improve clarity of each research topic and to suggest additional research topics.

During round 1, panelists completed the ratings electronically over 2 weeks, with 2 weekly reminders sent to nonresponders. During round 2, panelists participated in an in-person 2-day meeting to discuss the round 1 ratings. Panelists received a copy of their ratings and de-identified summary tables with group distributions for each research topic. Meeting moderators (R.J.C. and C.S.) did not rate topics. The facilitated group discussion goal was to ensure panelists had a consistent understanding of terms and concepts. The aim was to identify legitimate areas of disagreement and not to build rating consensus.¹⁷  After adjustments to rating materials (eg, wording changes for clarity) panelists rerated the research topics.

After round 2, we summarized rating distributions, areas of agreement and disagreement, and frequencies of each rating for research topics. Anticipating that the ratings would be skewed toward higher numbers, we ranked research topics into groups to convey their relative prioritization. The following criteria were used to define each rank group.

• Rank 1: topics had a median in the high range (≥7) for all 3 domains (need and urgency, research impact, and family centeredness) and a median of >7 in at least 2 of 3 domains;

• Rank 2: median of ≥7 in all 3 domains and >7 in 1 of 3 domains;

• Rank 3: median < 7 in 1 or 2 of 3 domains, or median = 7 for all domains; and

• Rank 4: median < 7 in all 3 domains or any disagreement.

Finally, to determine if family stakeholder priorities were distinct from those obtained through the RAM process, we reanalyzed the results from the 8 family caregivers separately. Differences in the highest-ranked research topics between family caregivers and the whole group were summarized.

Survey respondents (N = 269) reflected multidisciplinary stakeholders from across the United States, with at least 1 respondent from 41 of 50 states (Table 1). Nearly one-quarter (24%) were CYSHCN or family members. Approximately three-quarters (73%) self-identified as health professionals, including primary (28%) and specialty (26%) care providers, behavioral health providers and therapists (10%), administrators and policy experts (13%), and researchers (15%). Because reliable denominators were unavailable for surveyed groups and respondents could identify with multiple groups, response rates were not calculated. Moreover, our goal was to achieve thematic saturation and a broad list of candidate research topics from stakeholders rather than to achieve a high response rate.

As described previously, surveys generated 2835 individual free-text responses, which the research team synthesized and combined into 86 research topics. Nine themes emerged by consensus discussion: (1) access to services and supports (health system and community); (2) financing, payment models, and insurance; (3) caregiving at home and caregiving experience; (4) family health; (5) child health, vulnerability, and complexity; (6) care planning and coordination; (7) medical home and comprehensive care models; (8) technology and care; and (9) CYSHCN research methods innovations. Themes and representative text submissions are summarized in Table 2. Although research topics were sorted into a single theme, they did reveal overlap, with some research topics plausibly belonging to multiple themes.

After round 1, 24 of 86 research topics met rank 1 or rank 2 criteria. Experts proposed 10 additional research topics during round 1, leading to 96 total research topics. During round 2, as they reranked topics, panelists discussed tensions around breadth of research topic scope from 3 perspectives: (1) CYSHCN versus children and youth with medical complexity, (2) applicability to larger versus smaller populations, and (3) children versus youth versus adults. Guidance from MCHB team members was to focus on broadly applicable topics. Panelists were therefore encouraged to rate need and urgency lower if a research topic’s scope was felt to be narrow.

The results reveal the varying degrees of each research topic’s perceived urgency, impact, and family centeredness. Overall, median ratings for the family centeredness domain were >7 for the largest number of research topics (47 of 96 topics, Table 3). The domain research impact had the lowest number of topics with high ratings, with only 10 of the 96 having median ratings >7. Although research topics focused on financing were most often rated as having high impact, they were simultaneously the least family centered. No research topics were rated by experts as low (ie, no medians were ≤3) in either need and urgency, research impact, or family centeredness.

The rankings of research topics after round 2 are summarized in Table 4. The rank 1 group comprised 15 research topics emphasizing social determinants of health (SDH), family caregiving, refining clinical models, value, and youth-adult transitions. Although research topics related to these concepts were present across all ranks, the rank 1 topics highlight specific priorities within each. For example, SDH topics in rank 1 centered on research to effectively integrate SDH screening into care, understand the unique influence of rurality on CYSHCN, and develop innovative technology to reduce inequities.

Rank 1 caregiving topics were centered on families’ experience of caring for CYSHCN at home. These topics centered on research to optimize care delivered by families to CYSHCN at home, use a whole-family perspective, and examine caregiving by using a strength-based approach (eg, understanding and improving family resilience as a means to achieving CYSHCN and family health). Rank 1 clinical-model refinement topics emphasized research to integrate systems, uncover key elements of effective care models, and divide “labor” (routine care and coordination tasks) efficiently among team members, including families. Value-oriented rank 1 research topics centered on research used to determine what value means to different stakeholders and then to focus on return on investment in programs, personnel, and state-based systems. Finally, the rank 1 youth-adult transition topics emphasized interventions to effectively plan for transition and to ensure ongoing insurance and community supports into adulthood.

As illustrated in Table 5, rank 2 research topics had lower need and urgency ratings than rank 1 topics. Rank 3 research topics had lower research impact and family centeredness ratings than rank 2 topics. Rank 4 research topics (ie, those with medians <7 for all domains) were more technology focused (eg, family use of social media and application- or cloud-based care plans), were more provider centered (eg, workforce and compensation), or were more related to research methods (eg, standardizing CYSHCN subpopulations). Complete rating details are available in the Supplemental Information.

Notably, the RAM ratings obtained from the complete panel were consistent with the ratings generated by only the family caregivers on the panel. In particular, both analyses placed 10 of the same research topics in rank 1. Ranks 1 and 2, when combined, were nearly identical in both analyses. For example, none of the rank 1 topics from the complete analysis were ranked below rank 2 in the family caregiver analysis. Only 1 rank 1 research topic from the family caregiver analysis was below rank 2 in the complete analysis (“How do delays from insurance denials affect CYSHCN health, and how can they be minimized?”).

We adapted RAM with a professionally broad and nationally diverse group of CYSHCN experts, including family caregivers, to begin developing a national research agenda for CYSHCN systems of care. Our process resulted in a taxonomy of CYSHCN research topics depicting their varying levels of stakeholder-perceived need and urgency, research impact, and family centeredness. The highest-ranked research topics were focused on SDH, family resilience, value, and youth-adult transitions. These findings simultaneously reinforce current CYSHCN research trends and point toward important new directions that support achieving high-functioning systems of care.

Rank 1 topics reveal how our results could promote CYSHCN research advances. For example, directly supporting CYSHCN caregiving at home emerged as a top research priority. From topic development to discussion during RAM, our process framed caregiver challenges in terms of resilience and adaptability, providing a more positive orientation than studies that are focused on caregiving stresses and burdens.^23–30  This frameshift also aligns with recent research suggesting that strategies to support family caregivers may improve CYSHCN health. For example, emerging data support the theory that child and caregiver health are tightly interlinked^31–33  and that increasing resilience may mitigate negative effects of toxic stress.^34–36  Resilience in childhood disability appears to be dynamic,³⁷  and panelists described resilience as a natural intervention target for future work. The latter echoes recent calls to apply real-time resilience measurement into routine care^5,34,38  and is consistent with the American Academy of Pediatrics (AAP) clinical report on psychosocial factors faced by CYSHCN and families.³⁸ 

At present, effective interventions to support family caregiving are limited,^39,40  and how best to achieve the rank 1 caregiving priorities across the vast array of CYSHCN conditions and family structures remains a key unknown. Promising approaches that emphasize family needs and strengths, build family capacity, and promote sustainable goal attainment have direct relevance to CYSHCN families.⁴¹  For example, the Family System-Illness Model⁴²  is 1 framework for increasing family resilience in clinical settings for children with chronic illness. Strengths of the model are its linking of biological and psychosocial demands of chronic illness and its accounting for variation in patterns of illness onset, course, outcome, functional loss, and uncertainty about illness trajectory. As effective CYSHCN family resilience interventions materialize, adaptation to the different circumstances experienced by CYSHCN could lead to broadly generalizable and potentially transformative strategies.

Rank 1 research topics also extend current value-oriented CYSHCN research, such as return on investment in payments, staffing, and interventions.^43–47  Although value-oriented research could have major implications for financing and organization of systems of care for CYSHCN, the area remains poorly understood.⁴³  Consistent with our findings, recent research reveals the central importance of defining value for CYSHCN care through incorporation of an array of stakeholder perspectives.⁴⁵  Closing key value-oriented knowledge gaps could begin with mixed-methods studies to establish stakeholder-centered value measures and applying these with different stakeholders. Comparative effectiveness study designs could be used to evaluate the effects of various alternative payment models on CYSHCN health outcomes.

Rank 1 and rank 2 topics also highlighted integration across care sectors (home, school, community, and health care) as a CYSHCN research agenda priority. Particularly notable were numerous topics on behavioral health and the transition from childhood to adult systems. RAM participants rated more than half of all topics in these 2 areas as family centered, likely reflecting the central role of families in CYSHCN care and in building robust systems for their care. Conversely, relatively few topics in these areas were rated high in research impact. Although the significance of this finding is unclear, it may represent concerns about being able to translate research findings from these topics to meaningful impact in the short-term.

Our explicit motivation to develop research priorities and relative rankings distinguishes these results from related efforts. For example, the MCHB 6 Core Outcomes¹⁵  and the AMCHP Standards for Systems of Care¹⁶  provide health system metrics and were developed with multidisciplinary and family input; however, neither was designed to reflect specific research priorities. Although some aspects of AMCHP and MCHB frameworks may benefit from research, others may require different initiatives (eg, policy or advocacy). As an example of this distinction, the RAM research topics in the theme of access to services and supports have substantial overlap with AMCHP and MCHB frameworks (as well as high family-centered ratings); however, none rated high in research impact. One potential interpretation of this finding is that although these topics have substantial relevance to CYSHCN, panelists were skeptical that conducting research was the best way to address the underlying challenges.

Although panelists felt that lower-ranked research topics had some uncertainty in need and urgency, research impact, and/or family centeredness, this does not imply that those topics are unimportant. RAM does not readily facilitate interpreting why panelists gave specific ratings and is not intended to force consensus. Technology-centered research topics, despite being a focus area in the AMCHP Standards for Systems of Care, were consistently rated low or uncertain. On the basis of conversations during the RAM meeting, we speculate this was due to perceptions that technologies face important feasibility and generalizability barriers. Panelists also expressed that although technology might facilitate solutions, it might not inherently solve problems. Provider-centered research topics (eg, workforce, compensation, satisfaction, and efficiency) also tended to be lower ranked, presumably because they are less family centered, knowledge gaps are less clear, and the topics are seen as less urgent.

This work has limitations. The volume and content developed through this process precludes a comprehensive discussion of each research topic. Throughout RAM, it was challenging to be as inclusive as possible while simultaneously developing a well-defined, generalizable, and actionable set of high-priority research topics. Similarly, although we integrated solicited literature from stakeholder organizations into RAM, we did not conduct a formal systematic literature review. Whether a systematic literature review would have uncovered additional topics or influenced the panelist ratings is not known. Because we anticipated ratings would be skewed toward the high end of the scale, we established rank priority cutoffs, which might be overly sensitive. RAM also relies on experts having the necessary knowledge to make valid judgements, which we could not verify. Despite in-person discussion, experts may have had distinct CYSHCN contexts in mind, which could differentially influence some ratings. For example, thinking about children with autism versus asthma might affect ratings about caregiving research topics. Additionally, our RAM participants were less demographically diverse than we would have liked. We also recognize that priorities will change over time on the basis of new knowledge, practice patterns, payment models, and policies.

To translate RAM results into an actionable research agenda for CYSHCN health systems, additional steps are needed. First, comprehensive reviews should be used to synthesize current knowledge and distinguish key gaps for each research topic. Second, researchers should determine key barriers to progress. Do challenges result from limited knowledge about a topic or from implementation difficulties? Third, specific research initiatives that will close knowledge gaps and overcome barriers should be articulated. For some well-studied research topics, work may need to be focused on dissemination and implementation science,⁴⁸  whereas for more emergent topics, basic discovery⁴⁹  may be most needed.

Despite these limitations, this work uncovers research topics with high perceived need, research impact, and family centeredness and may offer a pragmatic guide for families, researchers, policymakers, and health system leaders interested in improving CYSHCN outcomes. The consistency of our results, whether evaluated across the complete stakeholder group or among family stakeholders alone, suggests that they accurately reflect priorities of CYSHCN families. The synthesis of opinions of a national multidisciplinary group of CYSHCN experts, including family caregivers, represents a fundamental first step toward the development of a national research agenda to improve CYSHCN health systems. The work prioritizes the most important systems-level questions, facilitates setting research goals related to these questions, and contributes to a blueprint to guide investigators to work cohesively toward achieving an idealized vision of comprehensive coordinated systems for CYSHCN.

We thank the expert stakeholders, youth, and families who contributed their invaluable knowledge during the various phases of this work. We also thank Jeffrey Sleeth, MD, for his invaluable review of an earlier draft of this article.

AAP

American Academy of Pediatrics

AMCHP

Association of Maternal and Child Health Programs

CYSHCN

children and youth with special health care needs

CYSHCNet

Children and Youth with Special Health Care Needs National Research Network

MCHB

Maternal and Child Health Bureau

RAM

RAND/UCLA Appropriateness Method

SDH

social determinants of health