BACKGROUND AND OBJECTIVES:

Defining and measuring health for children with medical complexity (CMC) is poorly understood. We engaged a diverse national sample of stakeholder experts to generate and then synthesize a comprehensive list of health outcomes for CMC.

METHODS:

With national snowball sampling of CMC caregiver, advocate, provider, researcher, and policy or health systems experts, we identified 182 invitees for group concept mapping (GCM), a rigorous mixed-methods approach. Respondents (n = 125) first completed Internet-based idea generation by providing unlimited short, free-text responses to the focus prompt, “A healthy life for a child or youth with medical complexity includes: ___.” The resulting 707 statements were reduced to 77 unique ideas. Participants sorted the ideas into clusters based on conceptual similarity and rated items on perceived importance and measurement feasibility. Responses were analyzed and mapped via GCM software.

RESULTS:

The cluster map best fitting the data had 10 outcome domains: (1) basic needs, (2) inclusive education, (3) child social integration, (4) current child health-related quality of life, (5) long-term child and family self-sufficiency, (6) family social integration, (7) community system supports, (8) health care system supports, (9) a high-quality patient-centered medical home, and (10) family-centered care. Seventeen outcomes representing 8 of the 10 domains were rated as both important and feasible to measure (“go zone”).

CONCLUSIONS:

GCM identified a rich set of CMC outcome domains. Go-zone items provide an opportunity to test and implement measures that align with a broad view of health for CMC and potentially all children.

What’s Known on This Subject:

Despite significant research on children with medical complexity (CMC) and interest in adopting a population health framework, a conceptualization of their health and potential health outcomes is underdescribed.

What This Study Adds:

Group concept mapping with experts on CMC resulted in a more comprehensive list of 10 outcome domains, leading to a population health framework for CMC.

Children with medical complexity (CMC) generate 40% of child Medicaid expenditures despite being only 3% of the pediatric population.1,2 Although the number of clinical programs for CMC has increased rapidly to meet care coordination needs of CMC and their families, there is no consensus on which CMC health outcomes are feasible to measure and important to pursue.3 To date, the CMC research and clinical fields have suffered from an absence of theoretically grounded conceptualization of shared health outcomes for this diverse population as well as a lack of understanding about which outcomes are measurable in clinical settings. The lack of conceptually driven health measures for CMC has contributed to an ad hoc reliance on hospital-related metrics and a relative lack of attention to nonhospital-related measures and needs for CMC, such as long-term wellness and social and/or developmental measures.3 

While previously developing candidate domains of health outcomes for CMC from a systematic literature review (E.S.B., R.C., and B.B.N.; unpublished observations) and in-depth interviews with stakeholder experts on CMC,3 we noted that CMC experts tended to prioritize family functioning and wellbeing as well as child mental health, wellbeing, and social integration more than was reflected in existing literature (E.S.B., R.C., and B.B.N.; unpublished observations).3 Instead, researchers in the literature strongly emphasized traditional measures of health care use, which most experts also acknowledged as important (E.S.B., R.C., and B.B.N.; unpublished observations).3 We decided to engage a much larger, national group of stakeholder experts to attempt to develop a consensus set of core outcome domains that could be used to guide the selection of specific CMC health outcomes that are foreseeably measurable in real-world settings.

Group concept mapping (GCM) is a rigorous, structured, mixed-methods approach designed specifically for such a purpose.4 This participatory research methodology combines qualitative and quantitative data to produce a conceptual model representing how a group views a particular topic.4 GCM has been used by the Robert Wood Johnson Foundation to define its “culture of health.”5 It has also been successfully applied by researchers across disciplines to gather and synthesize collective input in a dynamic and systematic fashion to achieve a structured conceptualization of an idea or issue.4 

Our objective was to use GCM to (1) generate a framework depicting how CMC health is conceptualized by a broadly representative and diverse group of experts and (2) develop a set of candidate outcomes prioritized by importance and measurement feasibility. A comprehensive and meaningful set of CMC health outcomes could be used to guide health systems, policy makers, and community leaders in more effectively monitoring health and delivering needed services to CMC.

GCM was conducted as the final phase of a larger project in which we combined a systematic literature review (phase 1; E.S.B., R.C., and B.B.N.; unpublished observations) and an in-depth qualitative interview analysis (phase 2)3 with GCM (phase 3). GCM was used to generate and then synthesize a comprehensive list of potential health outcomes for CMC from a diverse sample of experts.

Through snowball sampling starting with our previous in-depth interview participants (who themselves represented CMC caregivers, advocates, providers, researchers, and policy or health systems experts), we identified a list of diverse, nationally representative CMC experts. Inclusion criteria were having been nominated as a CMC expert, being competent to consent for study participation, and being at least 18 years of age. Potential participants received e-mail invitations to participate in the Web-based concept mapping activity. In the invitations, we explained that participation would take ∼1 hour divided over 2 phases: (1) the idea generation activity and (2) the sorting and rating activity occurring several weeks later. Of the 182 potential participants contacted, 125 individuals across the United States and Canada agreed to participate (69% response rate).

The GCM process involves 4 steps: (1) preparation, (2) idea generation (brainstorming), (3) sorting and rating generated ideas, and (4) empirically creating idea maps.4 We used Concept System Global Max (CS Global) software, which provided a Web-based platform for the concept mapping method (https://www.conceptsystemsglobal.com/). CS Global allowed us to collect, analyze, and represent data from diverse stakeholders at geographically disparate locations at the timing of participants’ convenience. Data collection and analytic procedures undertaken during the GCM phase were based on previous GCM studies4 and were performed under the guidance of Concept Systems consultants. Figure 1 diagrams an overview of this process.

FIGURE 1

Concept mapping consort diagram.

FIGURE 1

Concept mapping consort diagram.

Close modal

During multiple team meetings, the research team developed a focus prompt for the brainstorming activity. The focus prompt was intended to elicit a wide array of short, free-text responses regarding potential health outcomes for CMC. Using an iterative consensus-seeking approach, the research team developed 3 potential brainstorming prompts and then pilot tested them with CMC experts who were independent of the research team. Responses to the pilot prompts were reviewed. The team then selected the following prompt: “A healthy life for a child or youth with medical complexity includes: ____.” During GCM preparation, the research team also identified the dimensions on which participants would rate each of the brainstormed statements during the GCM rating phase. After group discussion, the research team established the dimensions as (1) feasibility of measurement and (2) perceived importance, each to be rated on a 6-point Likert scale. The team prepared written instructions and webinars to teach GCM participants how to complete the brainstorming and sorting and rating activities.

For the idea generation (ie, brainstorming) step of GCM, each respondent was asked to provide, via the CS Global Web portal, an unlimited number of short, free-text responses to the focus prompt, “A healthy life for a child or youth with medical complexity includes: ____.” Because our purpose was to create as many unique statements as possible without identifying the most popular ones, generated statements were visible to all participants in real time, which reduced redundancy by allowing participants to view what others had already submitted. Participant responses were not linked to names, contact information, or any other personal identifiers.

During the brainstorming step, participants generated 707 statements that completed the focus prompt provided. These statements included 879 individual ideas (ie, some statements had multiple ideas). Because many of these ideas were highly similar, the research team performed coding and editing selection using an iterative consensus-seeking approach to aggregate brainstormed statements into a final list of 77 unique statements with only 1 idea in each statement. The final list was reviewed by research team members to ensure that all 879 individual ideas were represented.

After the completion of the brainstorming activity, participants were then recontacted via e-mail and invited to participate in the sorting and rating activity. During this phase, participants were first asked demographic questions, including their geographic location, sex, and self-identified primary and (if any) secondary role related to CMC (choices were CMC parent and/or caregiver, advocate, provider, researcher, policy expert, or health systems expert). On the basis of how participants answered these questions, we categorized participants into 4 mutually exclusive subgroups for analysis: (1) CMC parents and/or caregivers and advocates, (2) CMC providers, (3) CMC researchers, and (4) CMC policy or health systems experts.

For the sorting component of the activity, participants were asked to enter the CS Global Web portal and sort the reduced list of 77 unique ideas. Participants were instructed to group statements according to how similar in meaning or theme they were to one another without a restriction on the number of groups. We explicitly asked participants not to create groups according to priority or value, such as “important” or “hard to measure,” but rather to group statements that they viewed as conceptually similar. Participants sorted the 77 unique ideas into an average of 8 groups.

For the rating component of the sorting and rating activity, participants were asked to rate each of the collectively generated 77 statements in which participants describe outcomes for CMC based on (1) the importance of that outcome and (2) participants’ perceived feasibility of measurement. The following instructions were given: “By measurable, we mean that you can envision this outcome being widely measured across a health system serving children with medical complexity within the next 10 years. By important, we mean how important the outcome is in comparison to all the other outcomes using the full range of the scale.” Responses were recorded on a 6-point Likert scale, with most important and easiest to measure being scaled highest, respectively.

Once sorting and rating were complete, we used the CS Global software to perform multidimensional scaling and hierarchical cluster analyses, creating visual three-dimensional concept maps that represented the ideas of the participants (ie, cluster representation).4 Generation of the concept maps involved the creation of a point map, a cluster map, and a go-zone plot. The point map acted as the foundation for all other maps, with each statement represented as 1 point on the map. Distances between all points were determined by the frequency with which any 2 statements were sorted into the same group by participants.4 

After the point map was created, we then created cluster maps with CS Global by bounding points on the point map into polygons to create nonoverlapping clusters. For the clusters, we empirically grouped the original statements into domains using dissimilarity thresholds calculated from pairwise distances between all points (ie, shorter distances equal less dissimilarity). Multiple maps were automatically generated by using different thresholds, resulting in different numbers of clusters. The research team was then tasked with determining the optimal number of clusters for the final cluster map on the basis of which map was used to qualitatively best identify conceptually meaningful and distinct domains. GCM methodology researchers advise that there is no single or “correct” number of clusters that should be represented on a cluster map but rather that the researchers must determine the appropriate number of clusters to aid in the optimal interpretation of their concept map.4 Regardless of cluster number, the underlying point map remains constant. After analyses of the upper and lower limits of 5 and 20 clusters, respectively, for cluster concept mapping analysis, the research team determined that 10 clusters best fit the data (Fig 1). The research team then created domain names for each cluster that best represented the group of statements encompassed in each cluster.

We also represented our rating data with what is called a go-zone map.4 The go-zone map was used to compare statements across our 2 rating criteria: perceived importance and feasibility of measurement. A go-zone map is a bivariate plot with a point for each statement based on its average participant rating for both variables.4 The map is divided into quadrants, with the horizontal line showing the mean of the rating values for the rating criteria on the x-axis (importance) and the vertical line showing the mean of the rating values for the rating criteria on the y-axis (measurability).4 The top-right quadrant shows statements that were rated as above average for both criteria, termed the go zone. The go zone represents CMC outcomes that have both above-average importance and above-average measurement feasibility according to the stakeholder groups.

Finally, the GCM creation process was repeated for each of the 4 participant subgroups individually to compare and contrast the ideas of participant subgroups.

The University of California, Los Angeles Institutional Review Board approved the study protocol.

In total, 110 participants completed brainstorming and 62 participants completed the sorting activity, with 125 individuals participating in total. The highest percentage of participants (32%) identified their role as a CMC researcher, 26% were CMC providers, 21% were CMC caregivers or advocates, and 21% were policy or health systems experts. Participants were generally diverse in terms of geography, race and/or ethnicity, and sex; participants represented 23 different states and the District of Columbia, and 1 participant resided outside of the United States.

As described above, the brainstorming activity resulted in 707 statements, which included 879 individual ideas. Statements were synthesized and reduced by the research teams to 77 unique ideas (Fig 1). Participants then sorted and rated the 77 unique statements, and these results were used to generate a cluster map. The cluster map that best fit the data had 10 clusters (Fig 2). The team assigned a domain name to each of the 10 mapped clusters. The 10 domains for CMC health outcomes were (1) basic needs, (2) inclusive education, (3) child social integration, (4) current child health-related quality of life, (5) long-term child and family self-sufficiency, (6) family social integration, (7) community system supports, (8) health care system supports, (9) a high-quality patient-centered medical home, and (10) family-centered care. Domain names are described and sample prompts are provided in Table 1.

FIGURE 2

Cluster map of health outcome domains.

FIGURE 2

Cluster map of health outcome domains.

Close modal
TABLE 1

Health Outcome Domain Descriptions and Selected Health Outcomes

GCM DomainDescription of DomainExample Prompta Response Within Domain (ie, CMC Health Outcome)
1: basic needs CMC having their basic needs met 76: having basic needs met, such as housing, food, clothing, and safety 
2: inclusive education Access to an education system that fully supports CMC and allows the opportunity for maximal participation in school 20: an individualized education plan that optimizes learning and development 
3: child social integration The opportunity for full social immersion and acceptance by a community that empowers CMC 15: freedom from bullying, discrimination, abuse, or neglect because of their disability 
4: current child health-related quality of life Physical, emotional, and social aspects related to the health and developmental status of CMC 2: feeling loved and valued 
5: long-term child and self-sufficiency The presence of confident and self-reliant management of care for CMC 29: parents and/or caregivers who understand the condition well and have the skills to manage the child’s health 
6: family social integration Access to family social supports that allow the family to fully participate in the child’s life and remain active in the community that they live in 66: parents who are able to fully participate in their children’s care without fear of losing their jobs or income 
7: community system supports Access to social and physical supports that allow CMC to navigate their homes and communities 23: adequate benefits that cover the children’s needs and provide them the services and supports they need to remain in the community 
8: health care system supports Access to supports that allow CMC to obtain all needed health care services 40: comprehensive and uninterrupted health insurance that covers all equipment, service, or care needs 
9: high-quality patient-centered medical home Access to and use of high-quality comprehensive and specialty health care 11: comprehensive care that includes case management, an education system, medical providers, and the child and family 
10: family-centered care The presence of a beneficial partnership between providers, patients, and families that places families at the center of planning and decision-making related to the child or youth’s health 70: the child or youth being included in decision-making regarding his or her medical care whenever appropriate and possible 
GCM DomainDescription of DomainExample Prompta Response Within Domain (ie, CMC Health Outcome)
1: basic needs CMC having their basic needs met 76: having basic needs met, such as housing, food, clothing, and safety 
2: inclusive education Access to an education system that fully supports CMC and allows the opportunity for maximal participation in school 20: an individualized education plan that optimizes learning and development 
3: child social integration The opportunity for full social immersion and acceptance by a community that empowers CMC 15: freedom from bullying, discrimination, abuse, or neglect because of their disability 
4: current child health-related quality of life Physical, emotional, and social aspects related to the health and developmental status of CMC 2: feeling loved and valued 
5: long-term child and self-sufficiency The presence of confident and self-reliant management of care for CMC 29: parents and/or caregivers who understand the condition well and have the skills to manage the child’s health 
6: family social integration Access to family social supports that allow the family to fully participate in the child’s life and remain active in the community that they live in 66: parents who are able to fully participate in their children’s care without fear of losing their jobs or income 
7: community system supports Access to social and physical supports that allow CMC to navigate their homes and communities 23: adequate benefits that cover the children’s needs and provide them the services and supports they need to remain in the community 
8: health care system supports Access to supports that allow CMC to obtain all needed health care services 40: comprehensive and uninterrupted health insurance that covers all equipment, service, or care needs 
9: high-quality patient-centered medical home Access to and use of high-quality comprehensive and specialty health care 11: comprehensive care that includes case management, an education system, medical providers, and the child and family 
10: family-centered care The presence of a beneficial partnership between providers, patients, and families that places families at the center of planning and decision-making related to the child or youth’s health 70: the child or youth being included in decision-making regarding his or her medical care whenever appropriate and possible 
a

A healthy life for a child or youth with medical complexity includes: _________.

Among the 10 outcomes for CMC deemed most important to measure, the perceived measurability of these outcomes varied (Table 2). Nearly all domains had at least 1 outcome represented. In the Supplemental Information, we provide a comprehensive list of the domain descriptions and statements listed in order of importance as identified by participants.

TABLE 2

Top 10 Outcomes Rated as Most Important and Their Measurability Ratings

Importance RatingaMeasurability RatingbHealth Outcome
“A Healthy Life for a Child or Youth With Medical Complexity Includes: _________.”
5.69 4.29 76: having basic needs met, such as housing, food, clothing, and safety (domain 1) 
5.68 2.94 2: feeling loved and valued (domain 4) 
5.64 4.65 40: comprehensive and uninterrupted health insurance that covers all equipment, service, or care needs (domain 8) 
5.42 3.88 29: parents and/or caregivers who understand the condition well and have the skills to manage the child’s health (domain 5) 
5.41 4.12 23: adequate benefits that cover the children’s needs and provide them the services and supports they need to remain in the community (domain 7) 
5.37 4.3 69: access to necessary and high-quality specialty medical care (domain 8) 
5.27 3.73 11: comprehensive care that includes case management, an education system, medical providers, and the child and family (domain 9) 
5.25 3.39 66: parents who are able to fully participate in their children’s care without fear of losing their jobs or income (domain 6) 
5.25 3.66 70: the child or youth being included in decision-making regarding his or her medical care whenever appropriate and possible (domain 10) 
5.22 28: hope (domain 4) 
Importance RatingaMeasurability RatingbHealth Outcome
“A Healthy Life for a Child or Youth With Medical Complexity Includes: _________.”
5.69 4.29 76: having basic needs met, such as housing, food, clothing, and safety (domain 1) 
5.68 2.94 2: feeling loved and valued (domain 4) 
5.64 4.65 40: comprehensive and uninterrupted health insurance that covers all equipment, service, or care needs (domain 8) 
5.42 3.88 29: parents and/or caregivers who understand the condition well and have the skills to manage the child’s health (domain 5) 
5.41 4.12 23: adequate benefits that cover the children’s needs and provide them the services and supports they need to remain in the community (domain 7) 
5.37 4.3 69: access to necessary and high-quality specialty medical care (domain 8) 
5.27 3.73 11: comprehensive care that includes case management, an education system, medical providers, and the child and family (domain 9) 
5.25 3.39 66: parents who are able to fully participate in their children’s care without fear of losing their jobs or income (domain 6) 
5.25 3.66 70: the child or youth being included in decision-making regarding his or her medical care whenever appropriate and possible (domain 10) 
5.22 28: hope (domain 4) 
a

On a scale from 1 to 6, please rate how relatively important you think each outcome is to the overall health of CMC and their families: (1) relatively unimportant, (2) slightly important, (3) somewhat important, (4) important, (5) very important, and (6) extremely important.

b

On a scale from 1 to 6, please rate how measurable each of the following ideas is (by measurable, we mean that you can envision this outcome being widely measured across a health system serving CMC within the next 10 y): (1) very difficult to measure, (2) difficult to measure, (3) somewhat difficult to measure, (4) somewhat easy to measure, (5) easy to measure, and (6) very easy to measure.

Across domains, a go-zone chart revealed 17 outcomes to be both important and feasible to measure (Fig 3). These 17 outcomes included items from 8 of the 10 domains. Example go-zone outcomes included “minimizing medical errors,” “timely access to medical equipment and supplies,” “absence of pain,” “living in a home with a family,” and “an individualized education plan that optimizes learning and development.” Pattern match testing groups visually revealed the perceived importance and feasibility of each domain, with domains being analyzed as a unit (Supplemental Fig 4). The domains basic needs and health care system supports were ranked high in importance and measurability, whereas the domain child social integration was ranked lowest. Subanalyses of pattern match testing by each stakeholder group type revealed consistency in the go-zone items across stakeholder groups (results not shown).

FIGURE 3

Health outcome go-zone map. a Participants were asked to rate the measurability of each outcome for CMC; measurability was defined as envisioning the outcome being widely measured across a health system serving CMC within the next 10 years. Items were rated on a scale from 1 to 6 (1 = relatively unimportant and 6 = extremely important). b Participants were asked to rate the importance of each outcome to the overall health of CMC and their families. Items were rated on a scale from 1 to 6 (1 = relatively unimportant and 6 = extremely important). Go-zone outcomes (green quadrant): 7) Minimizing medical errors. 8) A single, immediately accessible point of contact for care coordination. 20) An individualized education plan that optimizes learning and development. 23) Adequate benefits that cover the child’s needs and provide them the services and supports they need to remain in the community. 26) Absence of pain. 29) Parents and/or caregivers who understand the condition well and have the skills to self-manage the child’s health. 30) Living in a home with a family. 36) Timely access to medical equipment and supplies. 40) Comprehensive and uninterrupted health insurance that covers all equipment, service, or care needs. 42) Easy immediate access to language interpretation, including sign language. 45) Timely access to comprehensive coordinated health care including subspecialty, behavioral and/or mental, dental, habilitative, and home services. 46) Respite care. 51) Access to behavioral and/or mental health professionals trained to provide care to these children and families. 57) Goals for his or her future. 69) Access to necessary and high-quality specialty medical care. 71) Consistent access to physical, occupational, and speech therapy needed to support functioning. 76) Having basic needs met, such as housing, food, clothing, and safety. Important, but not feasible to measure (yellow quadrant): 2) Feeling loved and valued. 3) Frequent positive emotions and infrequent stress or emotional distress. 4) A system that is able to be rapidly flexible to meet the needs of the children whose needs can change quickly. 11) Comprehensive care that includes case management, an education system, medical providers, and the child and family. 12) Smooth transitions between primary care, urgent and emergency department care, hospitalizations, and home. 15) Freedom from bullying, discrimination, abuse, or neglect because of their disability. 19) Consistently high-quality and restful sleep. 21) Focusing on wellness and/or prevention rather than just the condition. 22) Being able to articulate family needs and having comfort talking to medical providers. 27) Successful transition to maximum levels of physical, social, and work independence. 28) Hope. 35) Adequately trained home health providers that can meet complex medical needs. 38) Living in a family that understands how to access services for their children. 41) Culturally and linguistically competent care. 43) A medical home that is family centered, continuous, comprehensive, coordinated, compassionate, and culturally competent. 50) Care that can adapt quickly to the changing needs of the family in periods of stability, rapid decline, or end-of-life. 60) Participating in school, family, and other social or recreational activities that bring pleasure or enrichment. 61) Prioritizing services based on youth and family goals. 65) Maximal access to learning and educational and/or social participation in school. 66) Parents who are able to fully participate in their children’s care without fear of losing their jobs or losing income. 67) Shared patient-centered goal-setting with care teams that know the child and family’s needs. 70) The child or youth being included in decision-making regarding his or her medical care whenever appropriate and possible. 72) Having a resilient well-functioning family. 77) Having providers who act respectfully towards all types of families (eg, lesbian, gay, bisexual, or transgender families; grandparents as primary caregivers; adoptive or foster care families). Feasible to measure but not important (orange quadrant): 1) Minimal absences from school due to outpatient and inpatient health services. 5) Access to technology that optimizes functional status. 10) School and/or district liaisons for parents that have CMC. 13) Minimizing time in health care settings as much as possible. 14) Freedom from transportation barriers to care. 18) Large spans of time without or “vacations” from medical appointments or procedures. 25) A planned and coordinated transition to appropriate care services in adulthood. 31) A system that includes continual home care and follow-up through the periods of exacerbation but also during the periods of stability. 34) An accessible shared plan of care that members of the care team, family, and others can access and update regularly. 39) Having venues for the child and family to have their voices heard such as through family and/or youth advisory councils. 44) Easy and timely access to legal help as needed. 47) Access to data about center-specific quality and outcomes compared to other centers. 48) Minimizing caregiver burden by scheduling medical visits together or on evenings and weekends. 49) Screening and identification of conditions that benefit from early intervention. 52) Funding to help families find Americans with Disabilities Act accessible housing and transportation. 53) Access to reproductive and/or sexual health care and developmentally appropriate sex education. 55) Time outdoors. 56) Advance planning for end of life care. 58) Minimal absences from school due to child’s health. 62) Comprehensive and up-to-date documentation of baseline status. Neither feasible to measure nor important (grey quadrant): 6) Living in a community that works to accommodate their needs. 9) A strength-based approach to care. 16) Being part of a community of shared values and beliefs. 17) Celebrating milestones. 24) Consideration of family health literacy. 32) A home that looks and feels like a home, not overwhelmed by medical equipment. 33) Counseling and other support for health care and educational professionals to prevent burnout so that they can better provide care. 37) Self-confidence. 54) Technology that efficiently enhances access to health care providers, services, and data exchange. 59) The family not being limited to where they can live geographically because services are only available in certain cities or regions. 63) Support from foundations and organizations. 64) Education for the school on how to care for the child. 68) Access to role models who have thrived with similar conditions. 73) Supporting siblings through programs such as sibling support groups, actively participating in social and/or recreational activities, and 1-on-1 time with parents. 74) Self-identification as just a “normal” kid or family. 75) Nutrition that is both culturally and medically appropriate.

FIGURE 3

Health outcome go-zone map. a Participants were asked to rate the measurability of each outcome for CMC; measurability was defined as envisioning the outcome being widely measured across a health system serving CMC within the next 10 years. Items were rated on a scale from 1 to 6 (1 = relatively unimportant and 6 = extremely important). b Participants were asked to rate the importance of each outcome to the overall health of CMC and their families. Items were rated on a scale from 1 to 6 (1 = relatively unimportant and 6 = extremely important). Go-zone outcomes (green quadrant): 7) Minimizing medical errors. 8) A single, immediately accessible point of contact for care coordination. 20) An individualized education plan that optimizes learning and development. 23) Adequate benefits that cover the child’s needs and provide them the services and supports they need to remain in the community. 26) Absence of pain. 29) Parents and/or caregivers who understand the condition well and have the skills to self-manage the child’s health. 30) Living in a home with a family. 36) Timely access to medical equipment and supplies. 40) Comprehensive and uninterrupted health insurance that covers all equipment, service, or care needs. 42) Easy immediate access to language interpretation, including sign language. 45) Timely access to comprehensive coordinated health care including subspecialty, behavioral and/or mental, dental, habilitative, and home services. 46) Respite care. 51) Access to behavioral and/or mental health professionals trained to provide care to these children and families. 57) Goals for his or her future. 69) Access to necessary and high-quality specialty medical care. 71) Consistent access to physical, occupational, and speech therapy needed to support functioning. 76) Having basic needs met, such as housing, food, clothing, and safety. Important, but not feasible to measure (yellow quadrant): 2) Feeling loved and valued. 3) Frequent positive emotions and infrequent stress or emotional distress. 4) A system that is able to be rapidly flexible to meet the needs of the children whose needs can change quickly. 11) Comprehensive care that includes case management, an education system, medical providers, and the child and family. 12) Smooth transitions between primary care, urgent and emergency department care, hospitalizations, and home. 15) Freedom from bullying, discrimination, abuse, or neglect because of their disability. 19) Consistently high-quality and restful sleep. 21) Focusing on wellness and/or prevention rather than just the condition. 22) Being able to articulate family needs and having comfort talking to medical providers. 27) Successful transition to maximum levels of physical, social, and work independence. 28) Hope. 35) Adequately trained home health providers that can meet complex medical needs. 38) Living in a family that understands how to access services for their children. 41) Culturally and linguistically competent care. 43) A medical home that is family centered, continuous, comprehensive, coordinated, compassionate, and culturally competent. 50) Care that can adapt quickly to the changing needs of the family in periods of stability, rapid decline, or end-of-life. 60) Participating in school, family, and other social or recreational activities that bring pleasure or enrichment. 61) Prioritizing services based on youth and family goals. 65) Maximal access to learning and educational and/or social participation in school. 66) Parents who are able to fully participate in their children’s care without fear of losing their jobs or losing income. 67) Shared patient-centered goal-setting with care teams that know the child and family’s needs. 70) The child or youth being included in decision-making regarding his or her medical care whenever appropriate and possible. 72) Having a resilient well-functioning family. 77) Having providers who act respectfully towards all types of families (eg, lesbian, gay, bisexual, or transgender families; grandparents as primary caregivers; adoptive or foster care families). Feasible to measure but not important (orange quadrant): 1) Minimal absences from school due to outpatient and inpatient health services. 5) Access to technology that optimizes functional status. 10) School and/or district liaisons for parents that have CMC. 13) Minimizing time in health care settings as much as possible. 14) Freedom from transportation barriers to care. 18) Large spans of time without or “vacations” from medical appointments or procedures. 25) A planned and coordinated transition to appropriate care services in adulthood. 31) A system that includes continual home care and follow-up through the periods of exacerbation but also during the periods of stability. 34) An accessible shared plan of care that members of the care team, family, and others can access and update regularly. 39) Having venues for the child and family to have their voices heard such as through family and/or youth advisory councils. 44) Easy and timely access to legal help as needed. 47) Access to data about center-specific quality and outcomes compared to other centers. 48) Minimizing caregiver burden by scheduling medical visits together or on evenings and weekends. 49) Screening and identification of conditions that benefit from early intervention. 52) Funding to help families find Americans with Disabilities Act accessible housing and transportation. 53) Access to reproductive and/or sexual health care and developmentally appropriate sex education. 55) Time outdoors. 56) Advance planning for end of life care. 58) Minimal absences from school due to child’s health. 62) Comprehensive and up-to-date documentation of baseline status. Neither feasible to measure nor important (grey quadrant): 6) Living in a community that works to accommodate their needs. 9) A strength-based approach to care. 16) Being part of a community of shared values and beliefs. 17) Celebrating milestones. 24) Consideration of family health literacy. 32) A home that looks and feels like a home, not overwhelmed by medical equipment. 33) Counseling and other support for health care and educational professionals to prevent burnout so that they can better provide care. 37) Self-confidence. 54) Technology that efficiently enhances access to health care providers, services, and data exchange. 59) The family not being limited to where they can live geographically because services are only available in certain cities or regions. 63) Support from foundations and organizations. 64) Education for the school on how to care for the child. 68) Access to role models who have thrived with similar conditions. 73) Supporting siblings through programs such as sibling support groups, actively participating in social and/or recreational activities, and 1-on-1 time with parents. 74) Self-identification as just a “normal” kid or family. 75) Nutrition that is both culturally and medically appropriate.

Close modal

GCM revealed outcomes for CMC that are largely missing from existing work. Furthermore, with it, we efficiently incorporated input from a diverse group of stakeholders (all with expertise on CMC) to combine their perspectives into a shared view of health for CMC. This approach yielded a rich set of domains from which to develop a more strategic approach to examining and measuring health outcomes for CMC and reshaping what we mean by health for this population.

In our preceding studies, we found that researchers in existing CMC literature do not consistently address several of the domains that are ranked as important and feasible, such as basic needs (E.S.B., R.C., and B.B.N.; unpublished observations).3 This represents a large and disturbing gap in our field. Consistent with our 1-on-1 interviews with experts on CMC, these outcomes extend beyond traditional conceptualizations of health, which are often focused on health care use measures that are directly fiscally relevant to health systems.3 With GCM, we elucidate core domains that extend beyond this to give a fuller view of health for CMC, 1 that perhaps more accurately represents both the day-to-day and the long-term challenges and opportunities that CMC and their families face. Our results reveal that more attention to psychiatric, social, and developmental needs may be needed. It will be up to health systems, and the policy makers whose rules govern them, to make these domains fiscally relevant and redesign their care delivery structures and processes accordingly.

To more comprehensively measure health for CMC, the implementation of additional measurement tools beyond what are commonly used in the CMC field is needed. We conducted an informal review of the biomedical literature to explore existing measurement tools for outcomes in the go zone (ie, ranked important and feasible to measure). We found that the richest existing measurement tools were primarily within research surveys such as the National Survey of Children with Special Health Care Needs.6 Moreover, for several go-zone items, tools that might be adaptable to clinical settings do not seem to exist at all. To better align with the outcomes identified by this group conceptualization process, a shift toward a more clinically appropriate measurement framework is needed. To achieve this, health system leaders and policy makers will need to be engaged alongside researchers in the process of setting measurement tool development agendas. The field will actively need to balance perceived importance and measurability. Purposeful alignment with other sectors, such as education and social services, may expedite the adaptation and implementation of a more comprehensive set of measures; for example, educational attainment (as a proposed inclusive education measure) and food security (as a proposed basic needs measure) were developed in nonmedical sectors. In fact, in this study, basic needs was rated as the outcome that was most important to measure. Seizing opportunities to align with existing measures in nonmedical domains and examine correlations with CMC may prove to be strategic. It remains an open question as to whether clinical settings are the best setting in which to collect these data. For instance, if data-sharing systems across sectors were enhanced, then one could potentially imagine multiple sources of data collection that could be used to increase the efficiency of outcome measurement.

As the field of pediatric complex care evolves, the eventual goal would be to incorporate measures of health for CMC that align with the broader view of health that emerged from this study and others like it. Ultimately, measures aligning with a broader view of health can become measures of health system quality as monitored and regulated by health systems. Additionally, policy makers and medical educators alike may draw from this conceptualization of health to cultivate communities and workforces of providers who are able to optimize health for CMC. Doing so, however, will require a collaborative multisector financing approach that currently eludes most health systems. Throughout this process, attentiveness to issues of equity (related not only to disability but also to factors such as race, ethnicity, and socioeconomic status) is crucial.

Several limitations of our approach exist. First, the inductive content analysis inherent to GCM is subject to investigator biases. Second, although the participant sample was large and diverse, the study sample may lack some important perspectives, including those of the CMC themselves. Finally, the feasibility of measuring many of the identified concepts is not yet clear, and perceived measurability is fairly subjective. Participants may have had differing ability to assess feasibility, and factors that influence measurability can change over time. Additionally, in our rating question on the perceived feasibility of measurement for each outcome, there may have been conflation between feasibility in developing a measurement tool versus actual logistic feasibility of measurement in a clinical setting. Nevertheless, we believe we provide valuable guidance on priorities for both developing and implementing the measurement of a broader set of key CMC health outcome measures.

The extent to which the health outcomes identified though the GCM approach extend beyond CMC to other pediatric populations is unclear. For CMC, the focal point for the domain of access to a high-quality patient-centered medical home may be centered on access to a subspecialist as a medical home provider. How would outcomes and prioritization differ for the larger group of children with special health care needs, for all pediatric populations, or infants versus school-aged children versus adolescents? Because CMC are an extreme example of a pediatric population with health needs (and the high family burden, reward, and involvement that result from these needs), lessons learned from CMC may elucidate important outcomes that are more easily obscured in other pediatric populations. Pediatrics might benefit from judiciously applying lessons from CMC to other populations, as was done historically in the conceptualization of the pediatric medical home from CMC and subsequent extension to the larger pediatric population.7 

Inclusive, systematic research approaches such as GCM may lead to a better, more comprehensive approach for identifying common health outcomes for CMC. The conceptualization achieved through GCM can be used to push the field closer to a more comprehensive population health framework for CMC. This framework can be used to inform policies and programs to better serve CMC and their families. Furthermore, lessons learned from CMC may have implications for the conceptualization and measurement of health for other special needs populations and potentially for all children.

     
  • CMC

    children with medical complexity

  •  
  • CS Global

    Concept System Global Max

  •  
  • GCM

    group concept mapping

Dr Barnert conceptualized the data collection approach and instruments, supervised the data analyses and interpretation, and drafted the manuscript and revisions; Drs Coller and Nelson, Ms Thompson, and Ms Breck assisted with the study design, interpretation of data analyses, and revision of the manuscript; Drs Szilagyi and Klitzner provided expert consultation on the study design, data analysis and interpretation, and manuscript revision; Dr Chung supervised all aspects of the study, from conceptualization and study design to data analyses and interpretation and writing of the manuscript; and all authors approved of the final manuscript as submitted.

FUNDING: Supported by a grant from the Lucile Packard Foundation for Children’s Health and by the University of California, Los Angeles Children’s Discovery and Innovation Institute.

COMPANION PAPER: A companion to this article can be found online at www.pediatrics.org/cgi/doi/10.1542/peds.2018-1925.

COMPANION PAPER: A companion to this article can be found online at www.pediatrics.org/cgi/doi/10.1542/peds.2018-1925.

We thank Dr Edward Schor for his contributions.

1
Cohen
E
,
Kuo
DZ
,
Agrawal
R
, et al
.
Children with medical complexity: an emerging population for clinical and research initiatives.
Pediatrics
.
2011
;
127
(
3
):
529
538
[PubMed]
2
Berry
J
,
Agrawal
R
,
Cohen
E
,
Kuo
K
.
The Landscape of Medical Care for Children With Medical Complexity
.
Overland Park, KS
:
Children’s Hospital Association
;
2013
3
Barnert
ES
,
Coller
RJ
,
Nelson
BB
, et al
.
Experts’ perspectives toward a population health approach for children with medical complexity.
Acad Pediatr
.
2017
;
17
(
6
):
672
677
[PubMed]
4
Trochim
W
.
An introduction to concept mapping for planning and evaluation.
Eval Program Plann
.
1989
;
12
(
1
):
1
16
5
Plough
AL
.
Building a culture of health: a critical role for public health services and systems research.
Am J Public Health
.
2015
;
105
(
suppl 2
):
S150
S152
[PubMed]
6
National Survey of Children with Special Health Care Needs
. NS-CSHCN 2011/12. Data query from the Child and Adolescent Health Measurement Initiative, Data Resource Center for Child and Adolescent Health Web site. Available at: http://childhealthdata.org/browse/survey/results?q=2625&r=1&g=448. Accessed June 7, 2018
7
Lerner
CF
,
Klitzner
TS
.
The medical home at 50: are children with medical complexity the key to proving its value?
Acad Pediatr
.
2017
;
17
(
6
):
581
588
[PubMed]

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

Supplementary data