Ethical Framework for Risk Stratification and Mitigation Programs for Children With Medical Complexity

Transparency is 1 of the cornerstones of good governance. The concept of transparency broadly encompasses an attitude of openness and clear communication on the part of the host organization regarding the program’s goals, scope, processes, finances, and all other aspects of the program.⁵ Transparency enables persons served by and otherwise affected by the program to understand and support, or question and challenge, its operation. For these reasons, transparency is a prerequisite of ethical program development and an essential element of trustworthiness.⁶ 

For the sake of transparency, any evaluation of a program should be clear about the perspective from which the evaluation is performed.³ For risk stratification–based programs for CMC, possible perspectives are those of the child, the child’s family, clinicians, health care systems, government agencies, or payers.

In what follows, we strive to highlight issues that would particularly matter from the perspective of a child and his or her family, and we will point out situations in which other perspectives might differ.

Risk stratification can serve several potential purposes. One (which is the chief focus of this article) is to enable case identification, which is to say identifying particular CMC who could potentially benefit from a programmatic intervention. Other purposes of risk stratification include enabling risk-adjusted analyses of outcomes or costs,^7,8 or to identify a group whose predicted medical costs are substantially different from the average to target for different insurance premium levels^7,9 or for exclusion from coverage.⁷ 

Whenever risk is discussed, the clarifying question “the risk of what?” should be asked and answered. There are many possible answers (Table 2). Any risk prediction model and risk mitigation program should specify the event or outcome that the program aims to prevent or minimize. For example, is the specific undesired event an emergency department (ED) visit or a hospital readmission or an outcome such as the onset of aspiration pneumonia or mortality? Is the focus any readmission or just preventable readmission, all-cause mortality or just preventable mortality? These are different from each other and from other potential events or outcomes, such as prolonged hospital stays or receiving health care with total costs >95th percentile for the population. If a program aspires to provide additional services to the children and families who need it most, then the focus would be on predicting which children and families are likely to experience excessive levels of parental work of care,¹⁰ unmet urgent or chronic care needs, preventable or remediable poorer health, or lower quality of life.

What about a program that sets out simply to identify CMC without focusing on a particular risk event or outcome? Aside from the problems that can arise from the circular reasoning of using a case identification process to both define and identify a group of patients (problems that we address below, such as how to determine the accuracy of such a scheme or determine the fairness of the identification process), this “cut to the chase” approach does not clarify what the program is chasing after. What is the reason or justification for identifying CMC? If the reason is to help meet the unmet care needs of CMC, curb the costs of care, or prevent hospitalization, the program should make these underlying reasons clear. Furthermore, the program should acknowledge that there is likely to be a less-than-perfect correlation between being identified as CMC by the program’s method of case identification and an elevated risk of the underlying problem that the program is supposed to be addressing, thereby threatening the ethical justifications for the program (as we discuss below).

The choice of which specific risk event or outcome to make the target of a program is a decision based on values that warrant clarification and examination. Care must be taken to explore how the value-based assessments of possible target risk events or outcomes might differ across stakeholder groups. For example, from the perspective of a family, a hospitalization may be viewed as an undesired event that imposes nosocomial risks on the child and disruption to family routines or, alternatively, as a desired event that provides respite or a desired level of therapy that could not be obtained at home, or a bit of both. From the perspective of a payer, the benefits and harms of a hospitalization (which could include its costs) may be viewed differently from the family’s view.

With the justification for the risk stratification–based program being to improve outcomes for CMC and their families, much hinges on whether the intervention effectively lowers the targeted risk event or outcome.

To reiterate, the target risk might involve the child’s health, functioning, development, or quality of life; the function and quality of life of the parents and family; unmet health care needs; or excess health care use and cost. Depending on which target was chosen, the program’s effectiveness should be evaluated specifically regarding lowering or otherwise improving that target risk.

Currently, few data exist regarding the effectiveness, or lack of effectiveness, of programs focused on CMC; whereas researchers in some studies have presented encouraging findings,¹¹ others have not.¹² Furthermore, most programs will have too few patients to have sufficient power to detect statistically significant differences in dichotomous outcomes (such as hospitalization events) and may equally lack power regarding continuous outcomes (such as costs) when the outcomes are distributed in a highly skewed manner. Those who administer programs may also lack the knowledge or resources to conduct and analyze rigorously designed studies regarding program effectiveness.

However, what those who administer programs should not do is rely on simple but potentially misleading evaluations in which they examine the outcomes and costs of patients before and then after entry into the program. We know that children who experience high levels of hospitalization in 1 year are the most likely to experience dramatically reduced levels of hospitalization the next year.¹³ This regression to the mean underscores the need for a comparator group that is similar in terms of the risk of hospitalization or other outcomes but does not receive the program’s risk mitigation intervention.

Given these challenges, one might ask whether we should avoid getting all tangled up in difficult evaluations about effectiveness and simply do the right thing, so to speak. Although this might seem tempting, we should regard what has occurred in adult care as a cautionary tale. For more than a decade, disease-management programs for adults have often been promoted by payers, hospitals, and other stakeholders on the basis of potential cost savings through greater uptake and adherence to relatively inexpensive preventative care, thereby (as the theory goes) reducing costs because of hospitalization and especially ED use. Although these programs have been shown to improve quality of care,¹⁴ researchers have not consistently concluded that disease-management programs reduce costs,¹⁵ and some have noted that standardized programs are a poor fit for complex patients with comorbid conditions, increasing burden but without adding benefit for some of the patients with the most severe cases.¹⁶ 

If a program identifies a particular patient as being suitable for the risk mitigation intervention, will the child and family willingly (ie, voluntarily) agree to enter into the program and receive the intervention? Once enrolled and receiving the intervention, will they willingly remain enrolled?

The answers to these questions about acceptability relate to factors that we have considered above: do the patient and family place value on preventing the targeted risk event or outcome, and is the program effective at minimizing that risk? The answer will also depend on other factors, some pertaining to the program itself (eg, the burden the intervention places on the family in terms of additional clinic visits or phone calls) and non-program–specific issues (eg, a family’s level of trust in the health care system or psychosocial factors that interfere with intervention adherence).

From an ethical point of view, the degree of acceptability of the program directly touches on issues of autonomy, voluntariness, and the role of informed consent when enrolling (and remaining enrolled) in the program. If the program is viewed by well-informed patients and families as highly acceptable, then no concerns on these fronts would arise. On the other hand, if the level of acceptability is lower for most families or is low for particular families, programs would need to ensure that participation is not encouraged via inappropriate inducements (eg, presenting only the potential benefits of the program while omitting the mention of burdens) or subtle forms of coercion (eg, negative financial repercussions for not participating).

A full evaluation of a program must also consider the burdens that the program places on patients and families (as mentioned briefly above) as well as the broader costs of running the program.

From the family perspective, the burdens of participating in a program could include expenditures of time (eg, going to care-coordination clinic visits, engaging in phone calls, or completing paperwork), bearing certain expenses (eg, transportation costs for additional clinic visits or taking time off from work to attend these visits), or having more work to do (eg, care-coordination or other direct-care activities to mitigate risk). Although a family may view these burdens as being offset by the benefits accrued by participating in the program, nevertheless, the burdens need to be accounted for to ensure that they are indeed outweighed by the benefits.

From the perspectives of individual providers, risk mitigation programs will almost inevitably cost a great deal of time, with potential implications for their performance evaluations, compensation, and career advancement. From the perspective of a health system or payer, running the risk mitigation intervention will also have costs that need to be accounted for (eg, staff salaries and other resources to house and run the program, education for staff and leadership, and lower reimbursement rates associated with more time-intensive patient care), although the reduction in other costs (eg, hospital stays) might offset the program’s costs.¹⁷ 

Other important drivers of program cost are the creation, implementation, and updates to the risk stratification model itself. These include the ongoing costs of acquiring and managing the necessary data, the costs of initial model development and validation, and periodic readjustments of the model (necessitated by changes in modes and the effectiveness of medical care).

As discussed above, the evaluation of a program’s burdens and costs can be performed from the perspectives of different stakeholders, including patients, families, health systems, payers, and even society. Given the possibility of cost shifting (whereby, for example, the payer costs associated with hospitalization are reduced but at the expense of greater costs for families), the evaluation needs to be clear about the perspective or perspectives taken and be able to justify the choice of perspective.

In practice, program evaluations employ different standards to determine if the program benefits are worth the program costs. Each of the various possible standards embodies a different stance regarding the value propositions of these programs, and these differences have ethical implications. For example, if the standard is that a program needs to be cost-saving or cost-neutral to be deemed worthy of continuation, regardless of the profile or magnitude of benefits, this stance would warrant ethical justification; in almost any other realm of health care, decreases in morbidity or improved outcomes justify some increase in spending.

Now, we must address the accuracy of the method used to identify children who are at heightened risk.

Within any risk stratification program, some model serves to estimate the risk level for every individual. These models can employ different methods. For example, some programs have clinicians nominate patients for participation in a high-risk disease-management program, whereas other programs use computer software to estimate a patient’s risk level on the basis of that patient’s diagnoses and other characteristics.¹⁸ 

Models should be assessed regarding their predictive accuracy because inaccurate or systematically biased models raise many concerns, ethically and otherwise. When assessing accuracy, the task is to determine how closely the model estimates the probability that a given individual will, in the future, experience the specific event or outcome. The word "future" is underscored because the model can only be evaluated by using longitudinal data, whereby data from 1 time period is used to predict events in some subsequent time period. Cross-sectional data should not be used to assess the accuracy of or validate predictive models because this type of data can only confirm cross-sectional associations.

To evaluate the accuracy of a predictive model, several statistical measures could be used. The area under the receiver operator curve, which is calculated as the C-statistic, provides a summary measure of how well a predictive model works (with a built-in tradeoff of sensitivity and specificity, weighing each equally). Because this measure evaluates the entire sample of individuals, it is not from the perspective of a given group of CMC, but rather from an analyst or program manager perspective.

The program’s analysts and manager would also want to know how well the risk prediction model is calibrated; across the continuous range of predicted probabilities from 0 to 1, how closely do the predicted probabilities match the observed probabilities, moving across groups of children from those with the lowest to those with the highest predicted probabilities? The calibration can be formally tested by using the Hosmer-Lemeshow test.

From the child or family perspective, important questions about the fairness of the risk prediction model would be better answered by knowing the model’s sensitivity (“If I truly should qualify for the program, how likely is this model to correctly identify me as eligible?”), false-negative rate (“If the model says that I do not have an elevated risk, how likely is it that the model made a mistake?”), or positive predictive value (“If I am identified as needing this program, how likely am I to really be at an elevated risk of the event or outcome?”).

Accuracy should be evaluated with an eye on the overall purpose of the program and a sense of whether the identification of patients will lead to a fair distribution of additional resources. For example, if a risk stratification algorithm based on diagnoses identifies a group of patients with cancer as being at risk for higher levels of resource use, costs, and mortality, but this group is already well known to have these risks and is already provided high levels of care coordination, should the algorithm be praised for this accomplishment? Conversely, if an algorithm is designed such that a specific condition (such as type 2 diabetes) is not included among the high-risk conditions that confer eligibility for program services, yet a particular patient with that condition would benefit from the program despite being ineligible on the basis of the algorithm score, how should the algorithm’s performance be judged?

Now, we move on to consider, from a population perspective, whether the program would fairly distribute its benefits and burdens across the population of CMC (and more broadly, across all other children) because fairness and justice are important ethical considerations.

Risk stratification requires dividing the underlying probability (ranging from 0 to 1) of an undesired event happening into categories or groups, such as low, medium, and high risk. A program must define cut-points by dividing the range of probabilities from 0 to 1 into 1, 3, or more subranges and then place individuals into risk groups depending on where their estimated probability falls within those subranges. For example, among children with neurologic impairment and swallowing dysfunction, the risk of microaspiration with feeding, the development of aspiration pneumonia, and being hospitalized all can, during a given period of time, range from 0 to 1. A program might classify children with risk in the 0 to 0.1 range as low risk, from 0.1 to 0.3 as medium risk, and from 0.3 to 1 as high risk.

The fairness of this scheme will depend in part on how, across the population of patients, the level of probabilistic risk is distributed; if the risk is distributed in 2 dominant modes, with most of the people having either a risk probability of close to 0 or close to 1, then dividing the population into 2 groups will seem sensible, whereas if the risk is distributed more evenly, drawing any given cut-point may seem arbitrary, raising the potential for perceived or real unfairness.

Fairness will also depend on how the distribution of risk matches up to, for each and every individual, the distribution of potential benefits and burdens of participating in the program. Let’s clarify what this means: one might suppose that the CMC who are at the greatest risk of having an undesired event or outcome are also the same CMC who would be the most likely to benefit from the program and have the greatest benefit-to-burden ratio, but these suppositions may not hold. For example, in a program intended to reduce hospitalizations among CMC with neurologic impairment, a particular child who is at high risk for aspiration pneumonia might nonetheless face an unfavorable benefit-to-burden ratio because of the potentially refractory process of microaspiration in a child with severely impaired airway protective reflexes, and thus limiting the potential benefit.

Fairness is a consideration not only about how benefits and burdens are distributed but also about how programs are administered and the degree of justice afforded by any appeal process.

We know from studies of other child-oriented programs (specifically regarding educational programs) that parents differ substantially regarding the amount of financial resources or other forms of cultural capital at their disposal to advocate for their children’s inclusion or exclusion from a program.¹⁹ Programs need to consider how they will ensure against providing, wittingly or unwittingly, preferential treatment to more empowered families.

For similar reasons, to ensure equity of treatment, programs should have a formal and auditable process in place for handling complaints and appeals from patients or families.

As a cornerstone of medical ethics, obtaining informed consent before performing any procedure is a fundamental method for respecting a person’s autonomy. Programs should scrutinize how they are informing and educating potential participants in the program about exactly what will be done and why and how they are ensuring that voluntary consent to participate has been granted. Similar to other interventions that unfold over time, programs should recognize that participants have the right to withdraw their consent and should have in place steps to enable participants to stop being part of the program if they so choose.

Finally, the ethical evaluation of any intervention or program (and in particular, a risk stratification–based program) must consider potential unintended consequences.² 

In the arena of adult health care, payers have employed a variety of tactics to reduce the expenses associated with complex medical conditions. A common scenario involves efforts to create a pool of relatively healthy, insured individuals by refusing coverage directly or indirectly to those who are sick or intentionally recruiting healthier policyholders. For example, after the passage of the Patient Protection and Affordable Care Act, the majority of insurers who offered policies through exchanges were found to be using adverse tiering to discourage certain groups of patients from enrolling in any of their plans.²⁰ Insurers have also been found to target advertising toward wealthier and healthier potential policyholders by, for example, sponsoring advertisements in health clubs.

Risk mitigation programs may not be equipped (financially, structurally, politically, or otherwise) to operate indefinitely. Some risk mitigation programs may bring considerable benefit to CMC and their families, but the prioritization of other outcomes (most commonly, cost reduction) that are not achieved might mean that the programs are considered failures and ultimately terminated by the host organization. Other programs may never fulfill their initial promises because of mismanagement, funding shortfalls, or political challenges and may disenroll CMC haphazardly or precipitously.

Disenrollment from programs may induce harms that would not have arisen had some CMC or their families never been enrolled in the first place because families may make important choices (eg, where they live, how their work is structured, or where they seek health care) on the basis of ongoing enrollment and then struggle to find similar services after they are terminated. Even when families are able to manage the end of a program with relative ease, they may feel disappointed and lose trust in their providers or the health care system. Some of these potential harms might be avoided with careful planning on the part of the host organization or intervention providers, who may owe participant families appropriate support or referral.

CMC and their families deserve the best health care possible, as do all children and families. To the degree that risk stratification and risk mitigation programs can optimize the care of and outcomes for CMC in a manner that is fair to all, we support these programs. Yet as we have highlighted here, these seemingly straightforward programs are complex and must be designed and evaluated regarding key ethical considerations to ensure that they indeed are fair to all and that they do not wind up serving the best the interests of health care organizations or payers as opposed to children and families.

CMC

children with medical complexity

ED

emergency department

We thank the Lucile Packard Foundation for Children’s Health for organizing and hosting the Designing Systems for Complex Care symposium in December 2015, which served as the impetus for writing this article.