Video Abstract
In 2005, the American Academy of Pediatrics founded the Partnership for Policy Implementation (PPI). The PPI has collaborated with authors to improve the quality of clinical guidelines, technical reports, and policies that standardize care delivery, improve care quality and patient outcomes, and reduce variation and costs.
In this article, we describe how the PPI trained informaticians apply a variety of tools and techniques to these guidance documents, eliminating ambiguity in clinical recommendations and allowing guideline recommendations to be implemented by practicing clinicians and electronic health record (EHR) developers more easily.
Since its inception, the PPI has participated in the development of 45 published and 27 in-progress clinical practice guidelines, policy statements, technical and clinical reports, and other projects endorsed by the American Academy of Pediatrics. The partnership has trained informaticians to apply a variety of tools and techniques to eliminate ambiguity or lack of decidability and can be implemented by practicing clinicians and EHR developers.
With the increasing use of EHRs in pediatrics, the need for medical societies to improve the clarity, decidability, and actionability of their guidelines has become more important than ever.
What’s Known on This Subject:
Medical associations have focused attention on the systematic and structured development of clinical reports and practice guidelines. Some make guidelines policies public and may solicit public feedback. Others limit input to organizational members. In 2005, the American Academy of Pediatrics launched the Partnership for Policy Implementation to improve on the quality of its recommendations.
What This Study Adds:
In this article, we describe how the PPI applies a variety of tools and techniques to eliminating ambiguity in clinical recommendations, allowing guideline recommendations to be implemented by practicing clinicians and electronic health record developers more easily.
Medical associations, including the American Academy of Pediatrics (AAP), have placed increasing attention on the systematic development and implementation of clinical reports and practice guidelines over recent decades. Approximately 30 years ago, the US government formed what is now called the Agency for Healthcare Research and Quality, and the Institute of Medicine published “Clinical Practice Guidelines: Directions for a New Program” in 1990. The National Academy of Medicine (formerly the Institute of Medicine) subsequently produced reports on guidelines in 1992, 1995, and 2008. Early tools for assessing guideline quality and addressing standardization and implementation challenges were developed.1–3 The Council of Medical Specialty Societies and the Institute of Medicine published reports on best practices in guideline development, including the Principles for the Development of Specialty Society Clinical Guidelines4 and the Clinical Practice Guidelines We Can Trust.5 These reports emphasize the need for a transparent, systematic review of evidence and recommendations articulated in an actionable, standardized format.
Other societies described their process for systematic literature review, evidence assessment, and stating recommendations. Some organizations are more prescriptive (eg, all statements start with “[organization] recommends” or “[organization] suggests”). The American Academy of Otolaryngology—Head and Neck Surgery (AAO-HNS) published 2 comprehensive, pragmatic “how-to” manuals6,7 exploring the theory behind guideline elements and the specific steps for creating one. Within AAO-HNSF guidelines, key action statements (KAS) are action-oriented prescriptions of specific behavior from a clinician.”8 The AAO-HNS’ KAS are followed by action statement profiles, which are structured reports on the guideline authors’ decision-making, including evidence quality, benefit-harm assessment, intentional vagueness, policy level, and differences of opinion. The AAP similarly promotes the use of KAS and expects a KAS to indicate when (under which circumstances) who (responsible actor) ought (with what level of obligation) to do what (action to be taken) to whom (recipient of the action), how, and why.9 The American College of Emergency Physicians’ clinical policies center on “critical questions” and produce concise recommendations by level of recommendation.10 The American College of Cardiology and the American Heart Association recently developed the concept of “modular knowledge chunks,” discrete packages of information “on a disease-specific topic or management issue.”11 The American College of Gastroenterology developed a public guideline dashboard showing the status of guidelines and links to summaries, decision support tools, and podcasts.12 Most organizations make their guidelines publicly available, and some solicit public feedback. Others limit input to organizational members or the guideline development committee and leadership.
Clinical guidelines synthesize existing evidence and provide evidence-based recommendations13 to standardize care delivery, improve quality, reduce harmful variation, and reduce costs. Unfortunately, not all guidelines are well-developed and actionable. An actionable recommendation meets the criteria of decidability and executability. Clinical recommendations may lack decidability when they fail to specify under which circumstances to take action. For example, a reader of the recommendation must be able to discern who will perform an action under which condition(s) and to whom. Guidelines may lack executability if recommended actions cannot be executed because basic requirements are lacking. For example, the person required to perform an action must have the resources, training, and skills to perform the action. A lack of decidability and executability leads to unintended variation in care because clinicians may interpret or execute recommendations differently.
However, guideline development practices by specialty societies reveal significant heterogeneity in policies and products. The AAP’s “Bright Futures: Guidelines for Health Supervision of Infants, Children and Adolescents, 3rd edition,”14 considered the “gold standard for preventive pediatric care,”15 is an example with only 21% (52/245) actionable recommendations.8 In 2023, the AAP’s leadership conference acknowledging the problem passed a motion to add informatics representatives to the Bright Futures team.
Although a lack of evidence or disagreements within the guideline-writing committees may introduce intentional vagueness (“should consider” or “may order”), more commonly, recommendations lack decidability and executability because of flaws in logic, conflicts between recommendations, incomplete conditions for recommendations, or lack of clarity. To address the need for transparency when evidence is lacking, the AAP recommends communicating that recommendations were consensus-based.16,17
Designing Good Guidelines
Adherence to recommendations from well-written clinical practice guidelines (CPG) improves clinical outcomes,18 decreases unexplained variation in care,19 and improves care efficiency.20–23 However, guideline recommendations are frequently not followed24–26 because of intrinsic guideline factors or workflow or implementation challenges.27,28
Barriers intrinsic to guidelines are common and involve ambiguity, vagueness, and underspecification.3,30 The guideline recommendation wording affects the perceived level of obligation for a clinician to adhere and the ability to evaluate and measure adherence effectively.29 Although guideline developers may include vagueness intentionally because of author disagreements or to allow for clinical flexibility, more often, vagueness is unintentional.30 Any vagueness can lead to different and potentially problematic interpretations among guideline consumers.8 For example, a now retired guideline for screening and management of retinopathy of prematurity (ROP) stated “[Stop screening if] postmenstrual age of 45 weeks and no pre-threshold disease (defined as stage 3 ROP in zone II, any ROP in zone I) or worse ROP is present.”31 Two content reviewers had divergent interpretations of “worse ROP”. The first believed it applied to patients with disease more severe than represented in the earlier part of the sentence. The second interpreted it as disease progression compared with a previous examination (ie, getting worse).32 The implementation of either interpretation could lead to unintentional variations in screenings.
Equally relevant are internal and external contradictions among recommendations in different guidelines. Contradictions can be intentional due to guideline authoring groups reaching different conclusions from the evidence (eg, different screening recommendations for cognitive impairment in older adults between the Canadian Task Force on Preventive Care33 and the US Preventive Services Task Force34). Contradictions can be a function of local or regional differences, such as differences in lead screening programs across the United States.35
Guideline recommendations can be internally contradictory. For example, an older guideline for breast cancer treatment from the American Society of Clinical Oncology contained the recommendation “In patients given PMRT [Post-Mastectomy Radiation Therapy], we suggest that adequately treating the chest wall is mandatory.”36,37 The language makes it difficult to discern if the recommendation is strong (“mandatory”) or weak (“suggest”). Updated versions resolved this issue after a Partnership for Policy Implementation (PPI) member suggested alterations.38
Not only do ambiguous recommendations potentially endanger patients and frustrate clinicians, but with 79% of office-based pediatricians in 201239 and 94% in 201640 using electronic health records (EHRs), decision support in EHRs is difficult to design for poorly specified recommendations. To create templates, order sets, alerts, and other EHR functionality to implement guidelines at the point of care,41 developers must be able to precisely interpret guideline recommendations and have clear conditions for when to apply recommendations (eg, patients <2 years of age with a hematocrit <35%). Ambiguous recommendations could be translated into incorrect decision support, making it faster and easier to practice “incorrect” medicine.
Even when guideline recommendations are well-written, they can be ineffective if they cannot be implemented (eg, a cleft lip guideline draft recommended involving a multidisciplinary cleft lip team,42 which is unavailable in many areas of the country) or are not aligned with clinical workflow (eg, a procedure in an outpatient setting that requires intensive care monitoring).
Partnership for Policy Implementation
Recognizing the need to make guideline recommendations decidable and executable in EHRs, in 2005, the AAP created the PPI, a group of clinical informaticians who collaborate with guideline authoring groups to ensure that clinical recommendations are actionable and decidable.9,43,44
The group started with 10 members and had expanded to 16 members as of 2022. PPI members engage with policy and guideline writing teams early in the development process to ensure that policies contain clearly defined logic that can be incorporated accurately into clinical practice and EHR decision support systems. PPI tracks the guidelines and policies with which its members are involved and engages AAP staff to ensure early and frequent communication. Since its inception, the PPI has participated in the development of 45 published and 27 in-progress CPG, policy statements, technical and clinical reports, and other projects endorsed by the AAP.
PPI Member Expertise
To ensure that PPI involvement generates predictable outcomes regardless of the PPI member involved, the PPI developed membership eligibility criteria that include:
Existing AAP member, in good standing, with current membership in the Council on Clinical Information Technology or the Council on Quality Improvement and Patient Safety preferred.
More than 3 years of clinical experience beyond residency or fellowship.
Clinical Informatics fellowship training or comparable experience, including the ability to perform the following tasks:
Translate evidence into clinical recommendations.
Identify ambiguity and vague or underspecified language.
Explain strategies for successful clinical decision support (CDS) implementation.
Ensure standardization and consistency of tools and other work products.
Master quality improvement techniques.
Clinical Informatics research or applied experience in any form, notably:
Applied experience, such as CDS implementation
Publications
Grants
Excellent verbal and written communication skills. Willing and able to participate in quarterly meetings and support 2 to 3 projects once completing an apprenticeship period.
Interest in or experience with the implementation or development of clinical guidelines and policies.
Interaction and communication skills congruent with PPI efforts, including a friendly, inclusive, supportive attitude.
PPI Methodology
The PPI uses a variety of tools to support guideline authorship (Table 1). Each of the tools supports a particular phase of guideline authoring or implementation. Depending on guideline authors’ needs, the PPI member selects relevant tools for statement development.
Guideline Authoring Support Tools
| Tool/Application | Purpose | Output |
|---|---|---|
| BRIDGE-Wiz1 | BRIDGE-Wiz uses a systematic and replicable approach with a constrained set of words and phrases to specify types of recommended actions with clarity and decidability.1 | Carefully crafted recommendation statements that align include an assessment of evidence quality, risk/harm balance, and recommendation strength statement. |
| eGLIA1 | Improve the implementability of guideline recommendations. | Documented analysis of factors that are potential barriers and facilitators to implementation of a guideline’s recommendations within clinical care. |
| Guideline Elements Model1 | Mark up completed guideline into machine-digestible chunks to support the isolated analysis of each clinical concept and identify ambiguity and vagueness surrounding narrative text statements. | Separately identifiable clinical concepts contained within the guideline recommendations for patient characteristics (decision variables) and clinical activities (actions). XML Representation of a guideline (L2 pseudocode) to support CDS development. |
| Algorithms | Support appropriate algorithm development and organizations that adhere to algorithm standards. | Algorithms representing the flow of guideline recommendations through the course of clinical care for a patient or system. |
| Tool/Application | Purpose | Output |
|---|---|---|
| BRIDGE-Wiz1 | BRIDGE-Wiz uses a systematic and replicable approach with a constrained set of words and phrases to specify types of recommended actions with clarity and decidability.1 | Carefully crafted recommendation statements that align include an assessment of evidence quality, risk/harm balance, and recommendation strength statement. |
| eGLIA1 | Improve the implementability of guideline recommendations. | Documented analysis of factors that are potential barriers and facilitators to implementation of a guideline’s recommendations within clinical care. |
| Guideline Elements Model1 | Mark up completed guideline into machine-digestible chunks to support the isolated analysis of each clinical concept and identify ambiguity and vagueness surrounding narrative text statements. | Separately identifiable clinical concepts contained within the guideline recommendations for patient characteristics (decision variables) and clinical activities (actions). XML Representation of a guideline (L2 pseudocode) to support CDS development. |
| Algorithms | Support appropriate algorithm development and organizations that adhere to algorithm standards. | Algorithms representing the flow of guideline recommendations through the course of clinical care for a patient or system. |
XML, Extensible Markup Language.
BRIDGE-Wiz software, developed at Yale University, facilitates the authoring of actionable, evidence-supported recommendations. The software encourages authors to use unambiguous terms describing actions (ie, prescribe, monitor) and avoid less measurable terms (ie, consider, offer). Each action is linked to several action verbs. For example, with the “prescribe” action, the verb “taper” is an option. The system walks authors through an assessment of benefits and harms to inform recommendation strength. Recommendations produced with BRIDGE-Wiz adhere to National Academy of Medicine standards for trustworthiness.45 Developed by a founding member of the PPI, BRIDGE-Wiz has been used by at least 6 professional societies and is now an online application.46
When recommendations are already authored, the PPI offers refinements. The PPI can guide a Guideline Implementability Assessment using the electronic GuideLine Implementability Appraisal (eGLIA). With eGLIA, authors and the PPI assess each recommendation to evaluate key domains, including decidability, actionability, and barriers and facilitators to implementation. The system also asks participants to consider resource use and the availability of data within the EHRs. eGLIA has been used internationally and by numerous clinical societies to improve guideline implementability.47
Sometimes, an analysis includes an evaluation of discrete clinical concepts as a precursor to quality measures or CDS. The Guideline Elements Model allows authors, supported by the PPI, to evaluate each clinical concept independently. Consider the term “ill appearing.” From a clinical standpoint, a provider may know what an author wants to convey, but to a machine, this is a hard concept to represent, synthesizing multiple discrete elements (eg, vital signs, activity, etc). The Guideline Elements Model can break this concept into its component parts for expression in pseudocode statements (Fig 1). Maintaining original recommendation logic, it allows clinical concepts from guidelines to be organized and machine interpretable.
Content of a guideline representation that was GEMcut and translated to pseudocode to support a CDS application development. Colors indicate the related source and GEMified text.
Content of a guideline representation that was GEMcut and translated to pseudocode to support a CDS application development. Colors indicate the related source and GEMified text.
Evidence Evaluation
The AAP Evidence-Based Clinical Practice Guidelines Development and Implementation Manual48 describes participants (including the PPI), who contribute to the AAP evidence-based clinical guidelines. Clinical guidelines are based on clinical questions identified by the AAP’s sections and councils. After a librarian- or epidemiologist-assisted systematic literature review, articles are synthesized into a table. Frequently, the results of the literature review will be summarized in a technical report. In the future, large language models may be useful to summarize the evidence for the guideline authors.
Subsequently, the writing team will formulate clinical recommendations based on the evidence or from consensus if there is minimal or low-quality evidence. The AAP reports the evidence in KAS profiles. A KAS includes:
Recommendation
Evidence quality supporting the recommendation
Recommendation strength
Anticipated benefits
Anticipated risks, harms, and costs
Value statement (eg, “The CPG Subcommittee believes that the benefits listed above outweigh the risks, harms, and costs described.”)
Deliberate vagueness (if applicable)
Specific exclusions (if applicable)
Key Action Statement Example
| Clinicians may choose not to administer supplemental oxygen if the oxyhemoglobin saturation exceeds 90% in infants and children with a diagnosis of bronchiolitis. Evidence Quality: D; Recommendation Strength: Weak Recommendation (based on low-level evidence and reasoning from first principles) | |
|---|---|
| Benefits | Decreased hospitalizations, decreased length of stay (LOS) |
| Risk, harm, cost | Hypoxemia, physiologic stress, prolonged LOS, increased hospitalizations, increased LOS, cost |
| Benefit-harm assessment | Benefits outweigh harms |
| Benefits | Decreased hospitalizations, decreased LOS |
| Value judgments | Oxyhemoglobin saturation >89% is adequate to oxygenate tissues; the risk of hypoxemia with oxyhemoglobin saturation >89% is minimal |
| Intentional vagueness | None |
| Role of patient preferences | Limited |
| Exclusions | Children with acidosis or fever |
| Strength | Weak recommendation (based on low-level evidence/reasoning from first principles) |
| Differences of opinion | None |
| Clinicians may choose not to administer supplemental oxygen if the oxyhemoglobin saturation exceeds 90% in infants and children with a diagnosis of bronchiolitis. Evidence Quality: D; Recommendation Strength: Weak Recommendation (based on low-level evidence and reasoning from first principles) | |
|---|---|
| Benefits | Decreased hospitalizations, decreased length of stay (LOS) |
| Risk, harm, cost | Hypoxemia, physiologic stress, prolonged LOS, increased hospitalizations, increased LOS, cost |
| Benefit-harm assessment | Benefits outweigh harms |
| Benefits | Decreased hospitalizations, decreased LOS |
| Value judgments | Oxyhemoglobin saturation >89% is adequate to oxygenate tissues; the risk of hypoxemia with oxyhemoglobin saturation >89% is minimal |
| Intentional vagueness | None |
| Role of patient preferences | Limited |
| Exclusions | Children with acidosis or fever |
| Strength | Weak recommendation (based on low-level evidence/reasoning from first principles) |
| Differences of opinion | None |
To ensure that recommendations are sufficiently specified, the PPI may use BRIDGE-Wiz, which takes the individual parts of a recommendation (such as role to perform, conditions, action, benefits, risks/harms/costs, evidence quality) and proposes how the guideline author might formulate a sentence and evidence and recommendation strength. Using the recommendation in Table 2, BRIDGE-Wiz, with inputs of role, clinicians, and conditions, would suggest statements such as (please note [1] all items in parenthesis must be true, and [2] capitalized “AND” and “OR” represent Boolean operators):
If (oxyhemoglobin saturation exceeds 90%) AND (infant OR child) AND (diagnosis of bronchiolitis), then clinicians need not offer supplemental oxygen (Evidence quality: D; Recommendation strength: Weak)
Clinicians need not offer supplemental oxygen if/when/whenever (oxyhemoglobin saturation exceeds 90%) AND (infant OR child) AND (diagnosis of bronchiolitis) (Evidence quality: D; Recommendation strength: Weak)
The AAP suggests that if (oxyhemoglobin saturation exceeds 90%) AND (infant OR child) AND (diagnosis of bronchiolitis), then clinicians need not offer supplemental oxygen (Evidence quality: D; Recommendation strength: Weak)
The AAP suggests that clinicians need not offer supplemental oxygen if/when/whenever (oxyhemoglobin saturation exceeds 90%) AND (infant OR child) AND (diagnosis of bronchiolitis) (Evidence quality: D; Recommendation strength: Weak)
The committee is then able to select and refine the text to create the final published version.
Algorithm Example
An algorithm describes a step-by-step process to be followed.50 Algorithms, given the same input, will consistently create the same results or output. The PPI has been using algorithms because they are easily parsed, understood, and followed. Graphical representations often take up less space and cognitive effort than text descriptions of the same rules and can be displayed where they are easily accessible. Figures 2 to 4 are algorithms from the PPI.
Algorithm for the evaluation and management of well-appearing febrile infants 8 to 60 days old. Diamonds indicate steps in the process in which information input is required. Depending on the information provided, the user is directed to the next steps. Ellipses represent endpoints. Arrows indicate the flow of actions and the instantiation options for a decision node. KAS are shown in parenthesis. A full description of the algorithm can be found in Pantell et al.60
Algorithm for the evaluation and management of well-appearing febrile infants 8 to 60 days old. Diamonds indicate steps in the process in which information input is required. Depending on the information provided, the user is directed to the next steps. Ellipses represent endpoints. Arrows indicate the flow of actions and the instantiation options for a decision node. KAS are shown in parenthesis. A full description of the algorithm can be found in Pantell et al.60
This algorithm indicates the steps in the process of screening, diagnosis, evaluation, and treatment of pediatric overweight and obesity.58 Each gray square shows one part of the assessment (screening, diagnosis, evaluation or treatment). Blue diamonds indicate steps in each individual process the process in which information input is required. Depending on the information provided, the user is directed to the next steps. Arrows indicate the flow of actions and the instantiation options for a decision node. KAS are shown in parentheses. The box under the figure provides a legend to aid in the interpretation.
This algorithm indicates the steps in the process of screening, diagnosis, evaluation, and treatment of pediatric overweight and obesity.58 Each gray square shows one part of the assessment (screening, diagnosis, evaluation or treatment). Blue diamonds indicate steps in each individual process the process in which information input is required. Depending on the information provided, the user is directed to the next steps. Arrows indicate the flow of actions and the instantiation options for a decision node. KAS are shown in parentheses. The box under the figure provides a legend to aid in the interpretation.
The algorithm depicts a neurodevelopmental stratification framework for infants born prematurely. Primary Care Physicians (PCPs) can estimate the patient’s individual degree of risk for developmental disabilities based on patient’s degree of prematurity59 or presence of a neonatal complication. The degrees of risk (very high, high, moderate-low) are shown for each condition. PCPs can then use the color-coded risk-satisfaction information to discuss care plans with family, document clinical decision-making, and/or determine the need for additional subspecialty providers or multidisciplinary care.
The algorithm depicts a neurodevelopmental stratification framework for infants born prematurely. Primary Care Physicians (PCPs) can estimate the patient’s individual degree of risk for developmental disabilities based on patient’s degree of prematurity59 or presence of a neonatal complication. The degrees of risk (very high, high, moderate-low) are shown for each condition. PCPs can then use the color-coded risk-satisfaction information to discuss care plans with family, document clinical decision-making, and/or determine the need for additional subspecialty providers or multidisciplinary care.
Pseudocode Example
In 2011, the AAP published a guideline for the diagnosis and management of attention deficit hyperactivity disorder (ADHD)51 and its revision in 2019,52 which included algorithms developed by the PPI. A research group, including PPI members, developed and evaluated a CDS system based on the guideline.53 With the system, clinicians were more likely to use DSM criteria to diagnose ADHD. The pseudocode used to create the system is shown in Fig 5.
Pseudocode example. The code checks for the completion of survey tools and then prints survey forms and recommends an evaluation if the child has not been evaluated and has symptoms of ADHD. Specifically, if the parent form or the teacher form have not been completed and the parent reports symptoms of ADHD, the system would recommend screening.
Pseudocode example. The code checks for the completion of survey tools and then prints survey forms and recommends an evaluation if the child has not been evaluated and has symptoms of ADHD. Specifically, if the parent form or the teacher form have not been completed and the parent reports symptoms of ADHD, the system would recommend screening.
Terminologies and Standards
Clinical guideline preparation frequently underemphasizes the importance of word choice. Important aspects of wording include using accessible language, using established terminology, and minimizing ambiguity and vagueness.
Defining key terms early focuses the subsequent development of KAS. For example, the AAP’s guideline for infantile hemangiomas54 clearly defines the role of hemangioma specialists and the characteristics of infantile hemangiomas. By delineating the terms and scope, readers can discern if the guideline is applicable to their clinical situation.
The translation of guidelines into CDS depends on using clear and precise language. “Hedging” language, such as “the physician may consider,” can lead to inconsistent interpretation and patient harm. Ambiguity can be classified as syntactic, semantic, or pragmatic.55,
Syntactic (or structural) ambiguity occurs when punctuation or Boolean connectors (and, or, not) leave the meaning unclear (eg, “start empiric treatment with ampicillin and gentamicin or cefotaxime” could be “ampicillin AND (gentamicin OR cefotaxime)” or “(ampicillin AND gentamicin) OR cefotaxime”).
Semantic ambiguity occurs when the same term can have multiple interpretations (eg, the word cervical can refer to the anatomy of the neck or of the cervix). Abbreviations can have multiple meanings (eg, ROM: rupture of membranes vs right otitis media vs range of motion).
Pragmatic ambiguity is nonspecific verbiage in context that does not clarify (“Can you pass the salt?” can be a question of ability or a request). For guidelines, this often occurs when recommendations do not provide guidance for all clinical scenarios and clinicians choose >1 option. Reducing ambiguity is performed by ensuring that sufficient context is provided.
Limitations to the PPI Effort
Time and Effort
PPI members volunteer their services to meet regularly in person and online to review their work and plan for future guidelines, with assistance from AAP staff. Given this, the PPI has a limited capacity. Although the PPI does not directly support the development of guidelines produced by societies other than the AAP, the PPI has trained other societies in PPI tools and methods and helped other organizations improve guidance documents and processes.
Legal Considerations
The AAP’s Committee on Medical Liability and Risk Management (COMLRM) reviews every AAP guideline authored. Sometimes, the COMLRM is concerned about strongly formulated KAS. For example, “The pediatrician shall refer the patient to a cleft lip and palate multidisciplinary team.” Pediatricians in areas without a multidisciplinary cleft palate team would be unable to comply with the recommendation, possibly resulting in medicolegal risk to the pediatrician. In this example, the guideline committee was asked to consider this practice setting and revise their recommendations and algorithm. The revised recommendation included a conditional statement addressing resources.
Understanding the concerns of the COMLRM, PPI experts now evaluate recommendations for obstacles, such as the unavailability of diagnostic testing or lack of specialists in certain settings that would prevent a pediatrician from following the recommendation.
Conclusions
The AAP PPI has collaborated with authors for 18 years on enhancing clinical guidelines, clinical reports, and policies to improve quality and patient outcomes and reduce undesired variation and costs. The partnership has trained informaticians to apply a variety of tools and techniques to improve decidability and executability so that recommendations can be implemented by clinicians and EHR developers. This has proven to be a sustainable model for organizing volunteer informaticians in support of unambiguous, computable clinical recommendations in AAP statements. With the increasing use of EHRs in pediatrics, this effort has become more critical than ever.
Acknowledgment
This article is dedicated to our colleague and dear friend, Stuart T. Weinberg, whom we lost too soon. A nationally recognized pioneer in pediatric informatics, Stuart was a founding member and prolific contributor to the work of the AAP PPI. His insights helped shape and guide the PPI, and his humor always kept things light. We will miss him.
All authors contributed to the concept and designed the work, drafted the initial manuscript, critically reviewed and revised the manuscript, approved the final manuscript as submitted, and agreed to be accountable for all aspects of the work.
FUNDING: No external funding.
CONFLICT OF INTEREST DISCLOSURES: Dr Downs is a co-creator of the Child Health Improvement Through Computer Automation system clinical decision support system and co-founder of Digital Health Solutions, LLC, a company formed to make the software commercially available. Dr Michel is the creator of Guideline Elements Model 2 Quality Data Model, a web application terminology for translating guideline content into quality measure format. He receives no financial compensation for this application. This application was not used in the development of this paper; however, the Guideline Elements Model and other related content were used. Dr Michel also consults for RoshReview as a content author and AstarteMedical on neonatal nutrition. Both these companies use guidelines in their content development, but neither is directly affected by this paper or supported the development of this paper. Dr Fiks is a paid consultant of the American Academy of Pediatrics for his work with Pediatric Research in Office Settings. Dr Sharifi is a paid consultant of the American Academy of Pediatrics for her work with the Institute for Health Childhood Weight. Dr Grout received institutional grant funding from Pfizer Inc, unrelated to this work. Dr Leu is the owner of Zimi Medical Technologies, LLC, which holds intellectual property, and received travel expenses for being on a guideline development committee for the American Academy of Pediatrics, which are not financially related to this work. Drs Adams, Lehmann, Chaparro, Weinberg, and Mendonca, Mr Salmon, and Ms Okechukwu have indicated that they have no potential conflicts of interest relevant to this article to disclose.
- AAO-HNS
American Academy of Otolaryngology—Head and Neck Surgery
- AAP
American Academy of Pediatrics
- ADHD
attention deficit hyperactivity disorder
- CDS
clinical decision support
- COMLRM
Committee on Medical Liability and Risk Management
- CPG
clinical practice guidelines
- eGLIA
electronic GuideLine Implementability Appraisal
- EHR
electronic health record
- KAS
key action statements
- ROP
retinopathy of prematurity
- PPI
Partnership for Policy Implementation
Comments