Skip to Main Content
Skip Nav Destination

Commentary From the Council on Quality Improvement and Patient Safety

October 13, 2023

Commentary From the Council on Quality Improvement and Patient Safety

The American Academy of Pediatrics (AAP) Council on Quality Improvement and Patient Safety (COQIPS), established in 2012, strives to define, implement, and disseminate evidence-based practices for effective, equitable, and safe clinical care using quality improvement (QI) science, policy, and education to advance lifelong health. We believe QI and patient safety (PS) are fundamental to daily pediatric practice, so that every child receives the right care at the right time, in the right place.

Distilling the QI and PS literature into landmark articles for the 75th Anniversary of Pediatrics was both daunting and exhilarating. Four members of the COQIPS Executive Committee searched the Pediatrics archives and nominated high quality articles from each era before using a modified Delphi process to select the best representatives. The following 4 major themes arose: the long history of QI and PS in Pediatrics; the development of rigorous methodology; the importance of collaboration for success; and the constantly evolving learning systems. We explore our landmark articles through these themes in the four commentaries below.

Pediatrics Chronicles the Birth and Growth of Pediatric Quality Improvement

Mary Beth Miotto, MD, MPH, FAAP1, John Chuo, MD, MS, IA, FAAP2, Raina Paul, MD3, Sandra P. Spencer, MD, FAAP4

Affiliations: 1Assistant Professor of Pediatrics, UMass Chan School of Medicine, Worcester, MA; 2Professor of Clinical Pediatrics, Children’s Hospital of Philadelphia, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA; 3Department of Emergency Medicine, Children's Hospital of Orange County, University of California, Irvine, Orange CA; 4Associate Professor of Pediatrics, Section of Emergency Medicine, Children’s Hospital Colorado, Department of Pediatrics, University of Colorado Anschutz Medical Campus, Aurora, CO

Pediatrics has a long history of leadership in exploring issues of quality in pediatric care. In fact, although oft-cited “quality pathways” inherent in Toyota’s Statistical Process Control1 were introduced in 1949 and Avis Donabedian’s seminal health QI work “Evaluating the Quality of Medical Care”2 was welcomed in 1966, the AAP prioritized the development of pediatric quality standards in the January 1948 inaugural issue of its flagship journal Pediatrics. A new AAP committee and a dedication to quality were announced:

 “The Committee (on Governmental and Medical Agencies) held a meeting at Pittsburgh in February 1947. The Chairman read three personal recommendations, which had been presented to the Executive Board the day before the Committee meeting, on (1) an American Academy of Pediatrics National Child Health Conference, (2) standards of quality in pediatric medical care and child health services, and (3) the problem of pediatric medical care in low income groups. These recommendations had been referred to the new Committee for the Improvement of Child Health for study and recommendation.”3

Figure 1: The first issue of Pediatrics from January 1948

Although COQIPS does not map a direct lineage from this earlier Committee for the Improvement of Child Health, COQIPS has partnered with Pediatrics over their mutual timelines, helping to integrate QI processes into the mission and work of the AAP.

In the 1984 article “A Method for Evaluating Performance of Ambulatory Pediatric Tasks,”4 we first begin to recognize QI elements that continue to propel the work of COQIPS and the AAP. The report presented the work of the Ambulatory Care Medical Audit Demonstration (ACMAD) Project, which endeavored to evaluate the cost and effectiveness of using quality assurance measures to improve patient care and detailed a controlled trial to develop a collaborative method for 8 ambulatory pediatric practices. The trial design included a computerized system to process evaluation data, offered digestible reports to practices, and included branching logic and patient care objectives reminiscent of outcome measures. The authors highlighted the need for all pediatric quality “assurance” to engage the individual pediatrician in behavior changes by identifying credible practice deficiencies and offering feasible steps for improvement of patient care. While clinical QI has evolved substantially from the earlier quality assurance work, certain elements were present at that time; size and scope of the program was limited to ensure manageable goals for the clinicians in this project.

“For care to improve, providers must first be convinced that genuine deficiencies have been detected, and they must then choose to make changes that are successful in correcting those deficiencies. Corrective actions could be expensive and difficult to implement, and providers may therefore lack the resources or the motivation to achieve improvements. Evaluations that detect deficiencies in care and are credible to providers are an essential first step in quality assurance, and such evaluations may also be useful for such purposes as research into determinants of the quality of performance, appraisal of physicians in training, and evaluation of health care programs.”

The 1990s ushered in an era of clinical practice guidelines, and COQIPS facilitated the development of many of these documents. By 1999, Pediatrics had published over 250 practice parameters, clinical guidelines, and policy statements accessed by thousands of pediatric primary care and specialty clinicians. As the domain of health services research and QI grew, Ferris et al offered a critical assessment on the effectiveness of quality programs specifically designed to improve child health outcomes, including clinical guidelines, provider reminders, physician leader interventions, benchmarking, financial incentives, continuing medical education, and patient education.5 The article applied a QI lens to the field of QI itself and challenged the AAP, health services researchers, and pediatricians to look at barriers and limitations in pediatric QI methodologies. The authors identified differences in the use of child health QI between organizations and offered a call to action for the decades ahead:

 “The extent of an institution’s commitment to child QI seemed approximately proportional to that institution’s commitment to providing child health care services. Large children’s hospitals may have extensive activities, whereas smaller organizations, general hospitals, and health maintenance organizations were more likely to limit QI to an asthma pathway and/or monitoring immunization rates…QI in small group practice settings and community hospitals was rarely more than the occasional chart audit necessary for licensing.”

Through the first 2 decades of the 21st century, Pediatrics has continued to offer QI resources to its readers while simultaneously challenging the discipline itself. Pediatricians now welcome more evidence-based clinical practice guidelines but also insist on transparent guideline development processes and greater attention to the AAP strategic priorities of equity and inclusion. COQIPS continues to partner with Pediatrics to highlight QI methodologies in policy statements, such as “A New Era in Quality Measurement: The Development and Application of Quality Measures,” which in 2021 offered recommendations for practitioners and policymakers on prioritizing and evaluating QI measures designed especially for the special needs of children.6

“Quality measures are used for a variety of purposes, including improvements in clinical care, regulation, accreditation, public reporting, surveillance, and maintenance of certification (MOC) programs. In addition, the federal government and other payers are shifting to payment programs linked to quality, such as accountable care organizations (ACOs) and patient-centered medical homes, making it increasingly important that pediatric health care providers understand and implement quality measurement into clinical care processes.”

QI research published in Pediatrics allows us to further trace the evolution of clinical QI from the first practice parameter on “Management of Hyperbilirubinemia in the Healthy Term Newborn” published in October 19947 to the August 2022 “Clinical Practice Guideline Revision: Management of Hyperbilirubinemia in the Newborn Infant 35 or More Weeks of Gestation,”8 moving from expert consensus and race-based medicine to clinical evidence with an equity lens. The accompanying commentary by Wright and Trent highlighted the imperative for the AAP and Pediatrics to face the potential for harm perpetuated by implicit bias present in historical clinical practice guidelines.9

“In theory, clinical practice guidelines (CPGs) are designed to reduce cognitive load, standardize care delivery, improve clinical outcomes, and mitigate disparities through equitable practice. However, pediatric CPGs have recently come under scrutiny for being outdated and methodologically challenged in keeping up with emerging evidence. In addition, a systematic review found that, although there were some positive effects of the use of race or ethnicity in pediatric CPGs, there was a potential for negative effect almost half of the time.”

Over the course of this timeline, the pediatric community has increasingly recognized the imperative for QI publications to themselves be subject to QI methodologies, such as offered by the IHI Model for Improvement.10 This includes providing frontline clinicians valuable QI tools to operationalize improvement in their practice settings. The authors of the 2021 AAP policy statement “Preventing Home Medication Administration Errors” recruited implementation scientists to ensure practice relevance. The 2022 quality report “Reducing Pediatric Emergency Department Prescription Errors” includes key driver diagrams, descriptions of PDSA cycles, and explicit outcome measures.11,12

Inherent in the science of QI is a responsibility to change, to respond to the environment, and to hold healthcare stakeholders accountable to patients and society. As we trace the history of child health QI, it is evident that Pediatrics has enabled the AAP to make this science increasingly accessible and feasible for the pediatric team and to reduce care gaps that threaten the health of the children we serve.

References

  1. Dahlgaard JJ, Dahlgaard‐Park SM. Lean production, six sigma quality, TQM and company culture. The TQM magazine. 2006
  2. Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q. 1966;44(3):166-206
  3. Report of the meeting of the executive board, Washington, DC, July 7 and 8, 1947. Pediatrics. 1948;1(1):90–116
  4. Palmer RH, Strain R, Maurer JV, Thompson MS. A method for evaluating performance of ambulatory pediatric tasks. Pediatrics. 1984;73(3):269-277
  5. Ferris TG, Dougherty D, Blumenthal D, Perrin JM. A report card on quality improvement for children’s health care. Pediatrics. 2001;107(1):143-155
  6. Adirim T, Meade K, Mistry K, Council on Quality Improvement and Patient Safety, Committee on Practice and Ambulatory Management. A new era in quality measurement: the development and application of quality measures. Pediatrics.2017;139(1):e20163442
  7. Provisional Committee on Quality Improvement, Subcommittee on Hyperbilirubinemia. Practice parameter: management of hyperbilirubinemia in the healthy term newborn. Pediatrics. 1994;94(4):558-565
  8. Kemper AR, Newman TB, Slaughter JL, et al. Clinical practice guideline revision: management of hyperbilirubinemia in the newborn infant 35 or more weeks of gestation. Pediatrics. 2022;150(3):e2022058859
  9. .Wright JL, Trent ME. Applying an equity lens to clinical practice guidelines: getting out of the gatePediatrics. 2022;150(3):e2022058918
  10. Langley GL, Moen R, Nolan KM, Nolan TW, Norman CL, Provost LP. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. 2nd ed. Jossey-Bass Publishers; 2009
  11. Yin HS, Neuspiel DR, Paul IM, Franklin W, Tieder JS, Council on Quality Improvement and Patient Safety, Committee on Drugs. Preventing home medication administration errors. Pediatrics. 2021;148(6):e2021054666
  12. Devarajan V, Nadeau NL, Creedon JK, et al. Reducing pediatric emergency department prescription errors. Pediatrics. 2022;149(6):e2020014696

Clinical Practice Guidelines Exemplify Quality Improvement and Patient Safety

Sandra P. Spencer, MD, FAAP1, John Chuo, MD, MS, IA, FAAP2, Mary Beth Miotto, MD, MPH, FAAP3, Raina Paul, MD4

Affiliations: 1Section of Emergency Medicine, Children’s Hospital Colorado, Department of Pediatrics, University of Colorado Anschutz Medical Campus, Aurora, CO; 2Professor of Clinical Pediatrics, Children’s Hospital of Philadelphia, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA; 3Assistant Professor of Pediatrics, UMass Chan School of Medicine, Worcester, MA; 4Department of Emergency Medicine, Children's Hospital of Orange County, University of California, Irvine, Orange CA

Highlighted Article From Pediatrics

CPGs epitomize QI and PS and have become a gold standard for the AAP patient care policies. CPGs aim to create evidence-based clinical recommendations that can be implemented at any institution through process improvement methodologies which ultimately provide safe and high-quality patient care. When the AAP produces CPGs, it provides metrics, publishes implementation tool kits, seminars, and webinars, creates networks that embark on multicenter studies on the CPGs, and highlights PS pearls and pitfalls. While the importance of a CPG to our care practice seems ubiquitous, the connection between QI, PS, and was not always as intuitive.

CPG production was initially scattered and not always evidence based. Initially, CPGs were created as hard paper copies with the best intentions to spread high-quality and safe care but ended up in a drawer and never looked at again. In his 1999 article, “Evidence-Based Guidelines and Critical Pathways for Quality Improvement,” David A. Bergman described the aspirations and challenges of creating safe and high-quality CPGs.1 The article specifically highlights how CPGs require consideration of all aspects of QI science and methodology and the need for a revolution in how we approach CPGs using QI methodology.

The opening paragraphs highlight that CPGs, like many QI initiatives, exist to disseminate evidenced practice. Most publications in QI describe single or multicenter studies implementing best practices. Even the discussions allowing most QI studies to be exempt from institutional review board evaluation hinge on applying known best care to broad populations. However, Dr. Bergman noted that outside demands continue to influence the motivations to make CPGs. This motivation exists for QI as well: improving cost, decreasing length of stay, increasing wellness screening, and providing additional education are now standard topics of QI projects. While Dr. Bergman might have argued these types of influences detract from the goal of providing evidence-based practice, many now feel goals give CPGs additional purpose to improve health care delivery.2,3

Dr. Bergman stresses the importance of assessing the evidence before applying it. The use of classification schemas for evidence evaluation remains an integral part of both CPG and QI work. Today, we can add the AAP schema and GRADE, among others, to the US Preventive Services Task Force Classification Schema mentioned in the article.4 Dr. Bergman also recommends vigilance in interpreting data and evidence. He rightly notes the dangers of race-based medicine, in that applying findings from an all-white male study group to a larger population is completely inappropriate. In recognition of the influence of social determinants of health, he also highlights the need to consider resources available to the patient and family in front of the provider in comparison to the evidence available to those in the evidence. Perhaps if we had heeded these recommendations, some of our current stumbles when considering diversity, equity, and inclusion may have been avoided.5

Dr. Bergman describes a method for improving use of CPGs in a newborn nursery. Although not explicitly stated, this method concisely describes the Institute for Healthcare Improvement QI methodology.6 Multidisciplinary teams, including parents, who work from the bottom up remain critical to the QI process. A solid set of goals with a clear aim statement remains just as important today. We bring this to another level by making our aims SMART (specific, measurable, achievable, relevant, and time bound) and may need to divide our aims into sub-aims to make our goals achievable. We continue to associate our aims with measures to let us know whether we have achieved our goals. The balancing measure has been added to the outcome and process (performance) measures since this article was published. The article also highlights the importance of “rapid, iterative data collection and revision cycles” to monitor and optimize interventions. This version of the PDSA or PDCA cycle remains the bedrock of successful QI projects. The recognition of how CPGs are “an important and necessary tool for improvement,” but which are not “sufficient to achieve improvement” evokes reliability theory. This QI and PS safety tool focuses on making it easy to do the right thing and is heavily relied on in improvement work today.

The article makes strong note of the “dearth of well-designed studies” evaluating CPGs. It challenges studies that use a “predesign-postdesign without a control group.” It confronts the fact that “very few studies have measured the impact of the intervention.” For more than 20 years, pediatric QI scientists have responded in kind to this call to arms: so much so that Pediatrics added “Quality Reports” to its journal in 2011.7 Studies use AIM statements, key driver diagrams, PDSA cycles, and drilling down techniques, among other tools, to create and duplicate well-designed studies. We have used statistical process control charting to measure changes over time after a carefully defined baseline period to correlate change with interventions. By monitoring our center line shifts in our outcome, process, and balancing measures well into sustain periods, which sometimes last for years, we can demonstrate changes in outcomes.

Ultimately, Dr. Bergman challenges us to avoid complacency with our CPGs, QI projects, and PS efforts. New innovations, especially those coming from the “collective knowledge and creativity of the staff,” should be “encouraged and tested.” The QI/PS community has taken this advice to heart. We have leveraged new technologies, like the electronic medical record, order entry systems, electronic tablets, and artificial intelligence, to improve our data monitoring, decision support tools, and interactions with our patients. We have taught our trainees QI tools and expected our community to contribute to QI/PS work throughout their careers.8 It has thus been woven into the fabric of our daily work. When the COVID pandemic occurred, it was the frontline staff that developed all the necessary and effective changes to continue to provide safe, high-quality care. Our past mistakes surrounding diversity, equity, and inclusion, especially race-based medicine, represent opportunities for improvement that our community recognized, owned, and now actively seeks out ways to combat.9

The connection between QI, PS, and CPGs has strengthened since 1999. The pediatric community recognized, and acted on, all the suggestions within Dr. Bergman’s article. We will continue to face many of the challenges highlighted, the QI and PS tools we have developed will allow us to continually assess ourselves, our institutions, and our practices.

References

  1. Bergman DA. Evidence-based guidelines and critical pathways for quality improvement. Pediatrics. 1999:103(suppl E1):225-232
  2. Hall RT, Domenico HJ, Self WH, Hain PD. Reducing the blood culture contamination rate in a pediatric emergency department and subsequent cost savings. Pediatrics. 2013;131(1):e292-e297
  3. Horbar JD. The Vermont Oxford Network: evidence-based quality improvement for neonatology. Pediatrics. 1999;103(suppl E1):350-359
  4. Steering Committee on Quality, I. and Management. Classifying recommendations for clinical practice guidelines. Pediatrics. 2004;114(3): 874-877
  5. Ostfeld-Johns S, Aragona E, Hart L. Removing race from hyperbilirubinemia guidelines is not enough. JAMA Pediatr. 2022;176(12):1163-1164
  6. Nolan T, Resar R, Haraden C, Griffin FA. Improving the Reliability of Health Care. IHI Innovation series white paper. Institute for Healthcare Improvement; 2004
  7. Kemper AM, Moyer VA, First LR. Introducing quality reports. Pediatrics. 2011;127(1):187-188
  8. Headrick LA, Richardson A, Priebe GP. Continuous improvement learning for residents. Pediatrics. 1998;101(suppl 3):768-774
  9. AAP Subcommittee on Urinary Tract Infection of the Council on Quality Improvement and Patient Safety. 2011 AAP UTI guideline reaffirmed after review of new data. AAP News. 2014; 35(7):1


    Better Together: A Quality Improvement and Patient Safety Collaborative Journey

    Raina Paul, MD1, John Chuo, MD, MS, IA, FAAP2, Mary Beth Miotto, MD, MPH, FAAP3, Sandra P. Spencer, MD, FAAP4

    Affiliations: 1Department of Emergency Medicine, Children's Hospital of Orange County, University of California, Irvine, Orange CA; 2Professor of Clinical Pediatrics, Children’s Hospital of Philadelphia, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA; 3Assistant Professor of Pediatrics, UMass Chan School of Medicine, Worcester, MA; 4Associate Professor of Pediatrics, Section of Emergency Medicine, Children’s Hospital Colorado, Department of Pediatrics, University of Colorado Anschutz Medical Campus, Aurora, CO

    We are better together. The evolution of collaborative work in the pediatric QI and PS space provides much credence to this dogma, as highlighted by the works in the AAP flagship journal Pediatrics. As robust QI methodology took hold in pediatric healthcare in the 1970s-1990s, individual hospitals, clinics, and systems focused on improving and surmounting barriers in a serial and singular fashion. As semantics, framework, and QI research have become solidified, however, individual groups have recognized the tremendous benefits that can be garnered from working together in an “all teach, all learn” environment.

    In the landmark paper “A New Framework for Quality Partnerships in Children’s Hospitals,” Levy et al bring to focus individual hospital system challenges when surmounting barriers to QI work, including financial pressures, competing priorities, lack of data, or QI methodologic expertise.1 Disparate priorities among systems have led to siloed efforts with little improvement in actual outcomes. Singular QI initiatives are often reactive to external mandates, rather than reflective of the collective will and passion to improve care processes. For such efforts, gains are smaller, providers are less engaged, and the results are rarely sustainable. Collaborative work can overcome these challenges by allowing members to share expertise regarding measurement such as utilization of statistical process control, by increasing patient numbers for rarer pediatric conditions to provide robust statistical analyses, and by sharing change strategies across multiple sites.

    A clear association exists between sites with integration of QI work, such as focus on collaboration and improved mortality outcomes.2 Lee et al directly tested this question in their work “Comparison of Collaborative versus Single Site Quality Improvement to Reduce NICU Length of Stay.”2 Noting the disparate hospital lengths of stay between NICUs, authors compared variation between centers that conducted single-site QI initiatives, 25 centers that collaborated, and centers that did not participate in any specific QI work. The QI collaborative group demonstrated a significant 2.9 day reduction in length of stay (LOS) that was sustained over time, whereas centers in the other 2 groups saw no change in LOS.

    Historically, hospitals have been reticent to participate in QI collaboratives, due to the initial costs, the dedication of personnel resources (eg, project managers, data analysts, and statisticians), the time burden on providers to meet, and concerns about data coordination. However, hospitals that sustain a reduction in LOS or mortality realize improved outcomes for patients and reduced institutional costs.

    Sepsis kills 8 million children worldwide annually. Improving care for these children is difficult because therapeutic interventions must be delivered in a timely manner in busy and complex hospital environments. Studying this work is fraught with challenges because defining sepsis is problematic, and analysis requires risk stratification of patients to truly understand outcomes. The AAP-sponsored Pediatric Sepsis Shock Collaborative (PSSC) published their work from 2013-2016 and noted significant difficulties in standardizing time zero and the denominator as well as struggles with data submission.3 After this collaboration, The Children’s Hospital Association sponsored its “Improving Pediatric Sepsis Outcomes (IPSO)” collaborative4-6 that included 64 hospitals at its peak. This study leveraged identification of barriers published from the PSSC collaborative to inform their expectations regarding the timeline for building a “data pipeline,” metric development and key interventional strategies. Collaborative structure, including use of QI consultants, access to sepsis and QI subject matter experts, virtual and in-person webinars, and participant-friendly discussion forums resulted in significantly improved care for a condition with one of the highest mortality rates in pediatrics. IPSO has demonstrated that adherence to a bundle of care elements, including timely recognition of sepsis, prompt delivery of a fluid bolus, and rapid administration of antibiotics, was associated with a 48.9% relative reduction in mortality. This collaborative enrolled sufficient patients for this rarer disease condition to delineate conclusions regarding optimal components of a sepsis bundle, while improving adherence to this bundle over time.

    The efforts of the highly regarded Solutions for Patient Safety (SPS) collaborative have been impressively chronicled by Pediatrics. This complexly structured collaborative demonstrated impressive reductions in hospital acquired conditions (HACs) and serious safety events (SSEs) in 2017.7 Nearly 100% of hospitals used robust QI methodologies, such as the Institute for Healthcare Improvement’s Model for Improvement, Lean Six Sigma methods, and high reliability concepts to reduce HACs by 9%-71% and SSEs by 32%.

    Similarly, the AAP’s Quality Improvement and Innovation Network (QuINN) demonstrated success applying QI collaborative rubric to over 300 practicing pediatricians and clinics across the country. As highlighted in “Improving Newborn Screening Follow-up in Pediatric Practices,” 15 clinics across 11 states strived to optimize timely and comprehensive newborn screen result follow-up, documentation, and communication with families.8 The collaborative observed best practices in this shared knowledge space, including metric definition, use of expert consensus across specialties, sharing of change packages that included paper and electronic tools for implementation, robust collection of baseline and ongoing data, and inter-site sharing with interval coaching. This collective model succeeded in increasing documentation and communication of newborn screen results from 2 to 10 sites and utilization of specialized genetic decision support tool sheets from 1 to 7 practice sites.

    Finally, one of the oldest and data rich QI networks, the Vermont Oxford network (VON) epitomizes collaborative best practices in their work to improve care for neonates nationally.9 Beginning in 1990, this collaborative has amassed an incredible data trove for all neonates weighing 401 to 1,500 grams that can provide baseline data for multiple NICU-related QI projects and allow for benchmarking of outcomes across sites. VON has demonstrated how to distill thousands of data points into discrete, manageable variables to inform clinically relevant metrics that sites can use to implement change. VON has reported and compared outcomes using risk stratification addressing a common criticism of many QI collaboratives. Understanding how to improve care for similarly sick cohorts of children dramatically increases provider acceptance of the data provided to them because accurate benchmarking is key to understanding true process gaps.

    The PSSC, IPSO, SPS, QuINN and VON collaboratives harness key common cultural elements in collaborative work to bolster their successes. First, 100% of the sites lend their differential expertise in a formal and informal manner to all participating hospitals and clinics, through formal teaching and real-time sharing through listservs and in-person communication. Second, didactics focus not only on education about disease pathophysiology but also on quality and PS education. This includes utilization of key driver, fishbone, and pareto diagrams and charts to understand barriers, a focus on high-reliability strategies to inform interventions, and the use of statistical process control charts to understand and interpret the data. Third, these collaboratives provide personnel infrastructure to each site, including QI consultants, rich data support, and structured forums to discuss failures in a protected manner. Finally, each collaborative provides transparent data of all measures, allowing hospitals, executives, and leadership to understand their gaps and gains and to benchmark their outcomes. Healthy competition, in industry and healthcare, drives sustainable change.

    The AAP has a long history of supporting innovative and emerging science, including that of QI, PS, and implementation work evolving over the past 75 years. Publication of the trials and successes of individual sites, now transforming into vast learning networks, has allowed an ongoing blossoming of robust QI collaboratives certain to continue changing the lives of our pediatric patients.

    References

    1. Levy FH, Brilli RJ, First LR, et al. A new framework for quality partnerships in children’s hospitals. Pediatrics. 2011;127(6):1147-1156
    2. Lee HC, Bennett MV, Crockett M, et al. Comparison of collaborative versus single-site quality improvement to reduce NICU length of stay. Pediatrics. 2018;142(1):e20171395
    3. Depinet H, Macias CG, Balamuth F, et al.Pediatric septic shock collaborative improves emergency department sepsis care in children. Pediatrics. 2022;149(3):e2020007369
    4. Larsen GY, Brilli R, Macias CG, et al. Development of a quality improvement learning collaborative to improve pediatric sepsis outcomes. Pediatrics. 2021;147(1):e20201434
    5. Paul R, Neidner M, Brilli R, et al. Metric development for the multicenter improving pediatric sepsis outcomes (IPSO) collaborative. Pediatrics. 2021;147(5):e2020017889
    6. Paul R, Niedner M, Riggs R, et al. Bundled care to reduce sepsis mortality: the improving pediatric sepsis outcomes (IPSO) collaborative. Pediatrics. 2023;152(2):e2022059938
    7. Lyren A, Brilli RJ, Zieker K, Marino M, Muething S, Sharek PJ. Children’s hospitals’ solutions for patient safety collaborative impact on hospital-acquired harm. Pediatrics. 2017;140(3):e20163494
    8. Hinton CF, Neuspiel DR, Gubernick RS, et al. Improving newborn screening follow-up in pediatric practices: Quality Improvement Innovation Network. Pediatrics. 2012;130(3):e669-e6675
    9. Horbar JD. The Vermont Oxford Network: evidence-based quality improvement for neonatology. Pediatrics. 1999;103(suppl E1):350-359

    Learning Systems as Engines for Practice Change

    John Chuo, MD, MS, IA, FAAP1, Mary Beth Miotto, MD, MPH, FAAP2, Raina Paul, MD3, Sandra P. Spencer, MD, FAAP4

    Affiliations: 1Professor of Clinical Pediatrics, Children’s Hospital of Philadelphia, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA; 2Assistant Professor of Pediatrics, UMass Chan School of Medicine, Worcester, MA; 3Department of Emergency Medicine, Children’s Hospital of Orange County, University of California, Irvine, Orange CA; 4Associate Professor of Pediatrics, Section of Emergency Medicine, Children’s Hospital Colorado, Department of Pediatrics, University of Colorado Anschutz Medical Campus, Aurora, CO

    If CPGs are the blueprint for better healthcare delivery, learning systems are the engine for change. When implementing CPGs in a local setting, how do we know whether they are consistently used, adhered to, and achieving their intended results over time? How do we know when it is time to revise a practice guideline because of new knowledge and discoveries? Health learning systems (HLS) are the part of an organization’s infrastructure designed to answer such questions by integrating “internal data and experience systematically with external evidence.”1 By monitoring health outcomes, health quality and cost, and experience associated with CPG implementations, a HLS helps CPG developers determine the need to revise existing CPGs and/or adjust health system processes and tools in order to adopt or adapt CPGs at the frontlines of patient care.

    Learning systems at the national level

    How does the AAP’s learning system facilitate important changes? To start, its leaders are willing to establish a culture where continuous data-driven learning and improvement become the de facto mindset of daily routine. By establishing COQIPS as well as other QI subgroups within other sections and councils, the AAP empowers its members, with their networks of subject matter experts in clinical care and research, to write, disseminate, and scrutinize evidence-based clinical guidelines. All the while, they continue rigorous review of new research findings that not only report on the effectiveness of implementing these guidelines but also any new evidence that may impact the original recommendations. The AAP provides an indispensable team of highly skilled managers and facilitators who can assist teams to work effectively and efficiently at a reasonable cadence. Teams review published research using digital libraries, communicate broadly using secure email and listservs, leverage consensus-achieving methods such as Delphi and Nominal Group Technique, and perform iterative Plan/Predict-Do-Study-Act approaches to implement change. Such resources, processes, and tools are the core components of the AAP’s learning system.

    By improving and using its own learning system, the AAP has modified its own CPGs as new knowledge becomes available over time. One example is “Management and Prevention of Hyperbilirubinemia in the Newborn Infant ≥35 weeks’ Gestation,” first published in 1994 by the Provisional Committee on Quality Improvement, revised in 2004, clarified with a statement in 2008, and fully revised again in 2022.2-5 After examining new research findings, the 2022 AAP CPG committee recommended raising the phototherapy thresholds, revised the hourly risk-assessment approach based on bilirubin concentrations, and proposed “escalation of care” strategies to address elevated bilirubin concentrations. Learning systems can help regional and local implementation of such new and/or revised guidelines.

    Learning systems at the collaborative level

    The AAP’s flagship journal, Pediatrics, is one of its greatest learning assets because it provides an important venue to disseminate findings from other pediatric learning systems otherwise known as collaboratives and/or networks. These collaboratives target specific clinical areas such as improving pediatric sepsis outcomes6-8 with the goal of helping participating centers to implement intervention bundles designed to reduce mortality/morbidity, to collect and analyze results over the course of the project (as single centers and in aggregation), and to adjust their interventions based on shared learning. Members from each center learn ways to engage stakeholders across organizations, manage change based on actionable data, and use collaboration frameworks like the Institute for Healthcare Improvement’s Breakthrough Series. A landmark publication by the VON in 19999 was among the first to focus specifically on the inner mechanisms of a collaborative learning system. In it, they describe how 34 participating NICUs worked together over 2 years and formed multidisciplinary teams to develop 4 key habits for improvement—change, practice as a process, collaborative learning, and evidence-based practice. VON created a central knowledge repository to share and socialize change ideas for improving neonatal care. The authors described 7 components of their learning system: “1) multidisciplinary collaboration within and among hospitals; 2) feedback of information from the Network database regarding clinical practices and patient outcomes; 3) training in quality-improvement methods; 4) site visits to project NICUs; 5) benchmarking visits to superior performers within the Network; 6) identification and implementation of ‘potentially better practices’; and 7) evaluation of the results.”9,10

    Learning systems at the practice level

    Learning systems become even more important at the frontline of care, where it really matters. To improve and optimize care, one needs to be able to compare and benchmark results from one’s own practice. The numerous quality reports published in Pediatrics, such as “Improving Adherence to PALS Septic Shock Guidelines,”11 are testimonials to effective use of learning systems at the microsystems level. In this example, the study team achieved 100% adherence to the monitored interventions in a cohort of 116 patients with septic shock and had lower mortality compared to a historical cohort. We learn that success depends on clear and aligned goals, available resources and people with the right skillsets, strong intrinsic motivation, clearly articulated action plans, and judicious use of actionable data. Flattened hierarchy and psychological safety allow assumptions to be challenged, mitigating risk for cognitive bias. Leaders and frontline staff should agree on a common mental model so their goals, possibly different, can still align with each other. Without these qualities, the care system is at risk of becoming stagnant and unable to adapt when unforeseen circumstances such as a pandemic happens.

    Improvement is impossible without the ability to adapt, and adaptation is impossible without the means to learn. Pediatric learning systems help us examine local data and experience as guidelines are implemented into clinical practice. This “on the job learning” helps organizations know how best to adapt CPGs into their culture and adapt as their environment changes.

    References

    1. About Learning Health Systems. Content last reviewed May 2019. Agency for Healthcare Research and Quality, Rockville, MD. https://www.ahrq.gov/learning-health-systems/about.html
    2. Provisional Committee on Quality Improvement, Subcommittee on Hyperbilirubinemia. Practice parameter: management of hyperbilirubinemia in the healthy term newborn. Pediatrics. 1994;94(4):558-565
    3. Subcommittee on Hyperbilirubinemia. Management of hyperbilirubinemia in the newborn infant 35 or more weeks of gestation. Pediatrics. 2004;114(1): 297–316
    4. Maisels MJ, Bhutani VK, Bogen D, Newman TB, Stark AR, Watchko JF. Hyperbilirubinemia in the newborn infant ≥35 weeks’ gestation: an update with clarifications. Pediatrics. 2009;124(4):1193-1198
    5. Kemper AR, Newman TB, Slaughter JL, et al. Clinical practice guideline revision: management of hyperbilirubinemia in the newborn infant 35 or more weeks of gestation. Pediatrics. 2022;150(3):e2022058859
    6. Depinet H, Macias CG, Balamuth F, et al. Pediatric septic shock collaborative improves emergency department sepsis care in children. Pediatrics. 2022;149(3):e2020007369
    7. Larsen GY, Brilli R, Macias CG, et al. Development of a quality improvement learning collaborative to improve pediatric sepsis outcomes. Pediatrics. 2021;147(1):e20201434
    8. Paul R, Neidner M, Brilli R, et al. Metric development for the multicenter improving pediatric sepsis outcomes (IPSO) collaborative. Pediatrics. 2021;147(5):e2020017889
    9. Horbar JD. The Vermont Oxford Network: evidence-based quality improvement for neonatology. Pediatrics. 1999;103(suppl E1):350-359
    10. Horbar JD, Plsek PE, Leahy K. NIC/Q 2000. NIC/Q 2000: establishing habits for improvement in neonatal intensive care units. Pediatrics. 2003;111(suppl E1):e397-e410
    11. Paul R, Melendez E, Stack A, Capraro A, Monuteaux M, Neuman MI. Improving adherence to PALS septic shock guidelines. Pediatrics
    Close Modal

    or Create an Account

    Close Modal
    Close Modal