Toward Transparent Clinical Policies

The mission of the American Academy of Pediatrics (AAP) is to promote the attainment of optimal physical, mental, and social health and well-being for all children. To aid in the accomplishment of this mission, the AAP develops clinical policies that are valued highly by members who provide direct health care to children, members of other organizations that share similar goals, and by parents, payers, and legislators. The utility of a policy depends, in large part, on the degree to which its purpose and basis are clear to policy users. This attribute is referred to as the policy's transparency. The purpose of this policy statement is to describe the critical importance and special value of transparency in clinical policies, guidelines, and recommendations; to identify obstacles to achieving transparency; and to suggest several approaches to overcome these obstacles. The term “policy” is used to refer generally to policies, guidelines, recommendations, and other similar statements.

The purpose of creating most clinical policies is to improve processes and outcomes of care by decreasing inappropriate variation in practice and increasing the implementation of effective strategies of health promotion and disease management. Such policies serve to guide clinical practice by summarizing the accumulated scientific evidence and combining it with the opinions of expert clinicians to define courses of action that are appropriate for patient care. Policies may provide guidance about:

• educating or counseling patients about their health and health care;

• implementing effective strategies of health promotion and disease management;

• using tests appropriately;

• monitoring patient status;

• defining criteria for diagnosis;

• performing procedures;

• prescribing medications and devices;

• referring for specialized care;

• documenting in the medical record;

• outlining ethical behaviors;

• defining an appropriate setting for care;

• promoting patient well-being;

• preparing clinicians and facilities to provide safe and effective care; and/or

• advocating on behalf of patients¹  and pediatricians.

Policies that are intended to influence the clinical actions of health care professionals should be based on the best available evidence and should include guidance regarding the application of such evidence to the individual patient.

Application of policy recommendations, like all clinical decision-making, is attended by some degree of uncertainty. Will this treatment be effective for this patient at this time? Will the patient suffer from an adverse effect that will interfere with successful treatment? Will the cost of the regimen make it impossible for the family to adhere to the prescribed intervention? Astute clinicians weigh the anticipated benefits of a policy recommendation against potential risks, harms, and costs viewed in the context of the patient's individual situation and preferences. Users' confidence that a given policy will result in particular benefits, risks, harms, and costs is enhanced when the policy is based on the best available clinical research and is free from bias and when the evidence and reasoning that support the policy are explicitly stated.

Transparency requires explicit statements regarding the reasons for developing a policy and explaining how published scientific evidence, pathophysiologic reasoning, clinicians' experiences, and perceptions of society's and patients' values are weighed. It also requires disclosure of potential biases of policy authors. Transparency allows potential users to judge the credibility of a policy by observing how the policy authors arrived at a result; thus, it has the potential to promote acceptance of the judgments and choices that have been made.

Policies will decrease inappropriate variability in practice only when they are accepted and adhered to by clinicians. Obstacles to the successful acceptance and implementation of a policy include concern about lack of credibility and bias on the part of policy writers; poor understanding of the process of policy formulation; apparent or actual lack of scientific evidence to support policies; and poorly articulated or vague policy statements. Each of these obstacles has the potential to diminish the user's confidence in the policy and, thereby, the authority and utility of a statement.

Ultimately, the authority accorded to a policy depends on the policy authors' credibility as child health experts. It is valuable for policies to include a concise summary of the breadth of skills and experience represented on the writing team. In general, national specialty societies, such as the AAP, and their policy-writing committees are accorded a high level of credibility by members and other users, because policy authors are considered to have considerable scientific knowledge and clinical expertise and to share and represent the values of stakeholders in the policy.² 

Concerns may also be raised about policy authors' potential biases. Despite the best intentions, policy authors may be influenced—consciously or unconsciously—by financial, personal, and intellectual conflicts in the development of policy. Potential conflicts of interest of all members of the formulating body must be declared. Disclosure of conflicts allows the reader to interpret the policy in light of those potential conflicts and permits other members of the policy-writing team to decide how to interpret contributions from a potentially conflicted team member. Public recognition of potential conflicts may also make policy authors more cognizant of otherwise-unrecognized biases.

An explicit statement of the purpose of the policy can help users to understand the values applied by the policy authors. For example, if the goal of a policy is to decrease inappropriate practice variation when scientific evidence supports a particular clinical practice, it may be interpreted differently from a policy with a goal of diminishing cost or influencing funding decisions.

The process of policy formulation should include a complete review of the available scientific evidence and formulation of guidance based on a combination of evidence and expert consensus. Because the validity of the guidance depends on these processes, an explicit statement of how evidence, expertise, and values were weighed by policy authors can help policy users to understand how best to apply the policy recommendations. The approved process of the AAP for creation of recommendations in evidence-based practice guidelines, for example, calls on policy authors to appraise evidence quality and make an explicit judgment regarding anticipated benefits, harms, risks, and costs.³  These declarations are summarized in a statement of evidence quality and strength of recommendation for each recommendation in a guideline.

For many situations in pediatric health care, high-quality evidence is not yet available.⁴  Because evidence is often absent or conflicting, many statements will inevitably be based largely on expert opinion. This is entirely appropriate, provided the basis is readily apparent to the critical reader. Indeed, it is when evidence is lacking, scant, or conflicting that expert guidance is most often sought. In these situations, policy authors must rely on lower-quality evidence, such as reasoning based on basic principles or expert consensus, to formulate coherent recommendations.

It is particularly important that users of a policy be aware if the policy relies on lower-quality evidence so they may be alert to the publication of new information and so those to whom the policy is applied are aware of the relatively tenuous state of the supporting evidence. Moreover, an understanding of the quality of supporting evidence should influence the expectations of payers and those who define legal standards of care. When policies are written in a spirit of full disclosure, policy users will be aware of the potential for change when new evidence becomes available and will be more likely to understand, and accept, changes in policy. Moreover, expectations of adherence should be lower when evidence quality is limited or there is a balance between anticipated benefits versus harms, risks, and costs.

Thoughtfully crafted statements that reflect hours of travail by policy authors may be difficult to put into practice consistently because of lack of clarity.^5,6  Ambiguous policy statements are those that are capable of being interpreted in more than one way. It seems obvious that policy statements that are intended to improve the consistency of clinical care should not be ambiguous. Yet, policy implementers regularly complain about the lack of clarity of published policies. A related problem is that authors often deliberately introduce vagueness into policy by using terms with meanings that lack precise boundaries.⁷ 

Reasons for intentionally creating vague recommendations include:

• insufficient evidence (commonly, the scientific literature in pediatrics has not addressed critical topics, or the conclusions of published studies are suspect because of methodologic flaws);

• inability to achieve consensus among the authors regarding evidence quality, anticipated benefits and harms, or interpretation of the published literature;

• legal considerations (ie, unwillingness to create a potential legal “standard of care”);

• economic reasons (one approach is clearly best but may not be affordable or cost-effective); and

• ethical/religious issues (such as attitudes about the “burden” or “futility” of care, premarital sex, or the use of blood products).

An explicit statement of the reasons for writing deliberately vague recommendations can help users interpret and apply them.

The Steering Committee on Quality Improvement and Management makes these recommendations to improve transparency and credibility of policy documents while recognizing the challenges involved in their full implementation.

1. Enhance the credibility of policy-making groups. Policy-writing panels should seek input from major stakeholders who are likely to apply the policy themselves or be influenced by it and from experts in the area of interest. Policy authors should disclose potential and actual conflicts of interest that might affect their policy writing and describe how they are addressed.

2. Make the process of policy formulation clear to users. A statement should include an explicit statement of the reason it was decided to make policy in this area and the intended goals of the statement. Authors should describe how the evidence was selected and assessed and what evidence exists to support the policy. Authors should clearly note when consensus and expert opinion have been required to formulate policy.

3. Improve the clarity of recommendations to facilitate implementation. Policy authors should be explicit about the exact circumstances under which a recommended action should be performed (decidability) and should describe precisely how that action should be performed when those circumstances exist (executability). By improving the decidability and executability of policies, such precision would improve consistent application of recommendations. If vagueness is introduced intentionally, the rationale should be documented.

4. Train skilled policy authors. Effectively implementing these recommendations will require training of policy authors in critical appraisal of evidence; differentiating questions of evidence and expertise from questions of value; consensus building; and understanding the policy-formation process. Opportunities for such training should be developed and sustained.

Professional societies, government agencies, and other organizations have created a vast array of policies that are widely used and highly valued. When such policies advise actions that have significant clinical and resource implications, it is particularly important that the policy-writing process explicitly recognize uncertainty and include careful literature review; systematic appraisal of evidence quality; weighing of anticipated benefits, harms, and costs; and documentation of the process. Every policy should be subject to a scheduled periodic review with reaffirmation, revision, or retirement as possible outcomes.

The solutions proposed in this statement may be applied as new policies are created or when policies are revised. Such gradual implementation will also help to ensure both continuity and accurate transformation to more transparent formats. Ensuring a high degree of transparency by standardizing the process for assessing the quality of evidence and strength of recommendations and defining reasons for deliberate vagueness will facilitate the work of policy developers, achieve better methodologic consistency across the broad range of clinical and policy statements, and thereby sustain and enhance the credibility of an organization's policies.

Richard N. Shiffman, MD (general academic pediatrics, medical informatics)

Edgar K. Marcuse, MD, MPH (general academic pediatrics, quality improvement)

Virginia A. Moyer, MD, MPH (general academic pediatrics, member US Preventive Services Task Force [evidence-based medicine])

Daniel R. Neuspiel, MD, MPH (general academic pediatrics, epidemiology)

Elizabeth Susan Hodgson, MD, Chairperson

Gordon Glade, MD

Norman Harbaugh, Jr, MD

Marlene R. Miller, MD, MSc

Xavier Sevilla, MD

Lisa Simpson, MB, BCh, MPH

Glenn Takata, MD

Gregg Lund, DO

Council on Clinical Information Technology

Denise Dougherty, PhD

Agency for Healthcare Research and Quality

Ellen Schwalenstocker, MBA, PhD

National Association of Children's Hospitals and Related Institutions

Caryn M. Davidson, MA