Panel management processes have been used to help improve population-level care and outreach to patients outside the health care system. Opportunities to resolve gaps in preventive care are often missed when patients present outside of primary care settings but still within the larger health care system. We hypothesized that we could design a process of “inreach” capable of resolving care gaps traditionally addressed solely in primary care settings. Our aim was to identify and resolve gaps in vaccinations and screening for lead exposure for children within our primary care registry aged 2 to 66 months who were admitted to the hospital. We sought to increase care gaps closed from 12% to 50%.
We formed a multidisciplinary team composed of primary care and hospital medicine physicians, nursing leadership, and quality improvement experts within the Division of General and Community Pediatrics. The team identified a smart aim, mapped the process, predicted failure modes, and developed a key driver diagram. We identified, tested, and implemented multiple interventions related to role assignment, identification of admitted patients with care gaps, and communication with the inpatient teams.
After increasing the reliability of our process to identify and contact the hospital medicine team caring for patients who needed action to 88%, we observed an increase in the preventive care gaps closed from 12% to 41%.
A process to help improve preventive care for children can be successfully implemented by using quality improvement methodologies outside of the traditional domains of primary care.
As part of the evolution of the population health concept, various frameworks for understanding health care and health have emerged.1 These models address health more holistically than just absence of disease or chronic disease management; and to achieve this potential, health systems must focus on populations and not just the individuals. These frameworks also recognize the importance of health promotion in early childhood, which has been the goal of Bright Futures for decades.2
Despite this focus on population health and promotion, gaps continue to exist in delivery of key services. Approximately 70% of children are up to date with vaccinations by 2 years old,3 falling short of the Healthy People 2020 goal of 80%.4 Screening for lead exposure is another important health promotion activity. The Centers for Disease Control and Prevention recommends lead screening for all children in older homes or in geographic areas with high rates of lead exposure.5 A recent American Academy of Pediatrics policy statement on prevention of childhood lead toxicity recognizes the risks to normal development, even with low lead levels, and endorses universal screening in high risk areas.6 However, in review of 2018 national Healthcare Effectiveness Data and Information Set data from Medicaid health maintenance organizations, only 69.6% of children had lead screening by age 2 years.7 This gap is greater in children who are behind on well-child care and in populations with low-income housing.8–11
Various outreach strategies have attempted to close care gaps, including the use of reminder and recall systems.12 Despite these outreach efforts, vaccine-preventable diseases continue to cause illnesses with a large financial burden throughout the country, including the measles outbreak in 2019.13
A less common strategy to address gaps is to resolve them during hospitalization. This practice is currently recommended by the Centers for Disease Control and Prevention Advisory Committee on Immunization Practices14 and has been a recommendation cited in the literature since the 1990s. However, barriers to resolving gaps outside the primary care setting persist. In previous studies evaluating vaccination status, researchers found as many as 50% of hospitalized children have vaccination delay.15–17 This incidence is higher in patients who are ethnic minorities, in lower socioeconomic groups, with transportation barriers, without insurance, or with previously missed vaccination opportunities15–21 and are similar to risk factors for children not up to date for lead screening.8–11 Lack of access to up-to-date records in the inpatient setting has been a major challenge in determining the correct status of patients.15–17 ,19–21 With continuous improvements to technology and the availability of vaccination history in electronic health record (EHR) systems, knowledge of gaps is becoming more accessible outside the primary care setting. Despite these advancements, our team has observed a continued discrepancy between the documentation around vaccination and lead-screening status by the hospital medicine (HM) team and patients’ true status.
Our objective was to identify and resolve vaccination and lead-screening gaps for children aged 2 to 66 months within our primary care registry who were admitted to HM. We anticipated an inreach process in which the primary care team identified and alerted HM to such patients would help resolve these gaps. Our specific aim was to improve closure of vaccination and lead care gaps from 12% to 50%.
Methods
Context
This project was conducted at Cincinnati Children’s Hospital Medical Center (CCHMC). The primary care clinics are the medical home for ∼37 000 active patients with ∼63 000 visits per year. Roughly 36% of patients are <66 months old, and ∼85% are enrolled in Medicaid. Since 2015, these clinics have been part of the All Children Thrive Learning Network, whose vision is to “help the 66 000 children in Cincinnati be the healthiest in the nation through strong partnerships.”22,23 Alongside a focus on visit-level metrics,24,25 we identified opportunities for additional gains in population health, both by trying to increase the number of patients interacting with primary care (outreach) and by targeting clinical areas outside the primary care clinics but within the larger health system (inreach). HM serves as the general inpatient service for CCHMC primary care, other safety-net providers, and most other independently employed pediatricians.
Study of the Interventions
We used the Model for Improvement as the basic framework.26 We formed a multidisciplinary team composed of HM and primary care physicians, nursing leadership, and quality improvement (QI) experts within the Division of General and Community Pediatrics.
Phase 1: Theory Development
The idea for the targeted improvement project originated in November 2016 in conjunction with projects aimed at improving the health of the population served by the primary care clinics, as a part of CCHMC’s 2020 strategic plan. In addition to improvements to care during primary care visits, the team wanted to develop a strategy to close preventive care gaps in which patients in the primary care registry touched the larger health care system. On the basis of chart review of admissions in the 12 months before starting this project, we found that an average of 46 children from our primary care registry were admitted to HM per month, representing 44% of all our patients’ admissions to any service. The team felt this rate was high enough to be able to develop and test a new process. We socialized the improvement effort with leaders from HM to gain support and establish stakeholder engagement, focusing the effort on 3 inpatient units covered by HM. Several months later, we added an additional unit at a satellite location. During this planning period, the team mapped the process from identification of an admitted patient to closure of the gap (Fig 1), predicted failure modes, and developed a key driver diagram (Fig 2).
Key driver diagram. Interventions are associated with relevant key drivers. Boldface type denotes key interventions. LOR 2 responds to 10−2 performance per the Institute of Healthcare Improvement.26 The remaining interventions have 10−1 performance. LOR 1, Level 1 Reliability; LOR 2, Level 2 Reliability; SMART, specific, measurable, applicable, realistic, and timely.
Key driver diagram. Interventions are associated with relevant key drivers. Boldface type denotes key interventions. LOR 2 responds to 10−2 performance per the Institute of Healthcare Improvement.26 The remaining interventions have 10−1 performance. LOR 1, Level 1 Reliability; LOR 2, Level 2 Reliability; SMART, specific, measurable, applicable, realistic, and timely.
Phase 2: Testing of Prototype Model
Our team designed an approach to identify patients in our patient registry who had been admitted with care gaps, with a preliminary focus on vaccinations. Initially the process was labor intensive, requiring manual chart review to identify patients with gaps. We conducted a series of plan-do-study-act (PDSA) cycles to develop and optimize an automated daily report to identify patients with vaccination care gaps and admitted to HM.
We also conducted PDSAs to streamline communication. Initially, we depended on a time-consuming paging system. Subsequently, we obtained a secure messenger communication platform already used on inpatient services to enable a primary care team member to directly alert the HM team to patient-specific gaps. This PDSA resulted in more rapid response to the message and often resulted in closed-loop communication regarding whether HM ordered vaccines. We identified another failure mode with patients often discharged before HM being contacted about the detected gaps. To address this, the time for the daily report was changed to 8:00 am from 10:00 am to allow for earlier gap review and communication with HM.
As testing for the prototype model evolved, the team explored adding a lead level order to blood in the laboratory from tests previously ordered during the admission. We conducted several PDSA ramps to identify patients due for lead screening and to develop a process for ordering the test. This initiative proved feasible, and we then added lead into the care gap column on the daily report and adopted this new screening method into our process.
Phase 3: Implementation of Standard Process
After testing of the prototype model proved feasible, we used high reliability strategies to improve and sustain the process. We documented the process in a standard operating procedure, including the following: (1) primary care registered nurse (RN) runs and reviews gap report, (2) RN sends message to HM resident with orders to write, (3) HM enters orders. We also reviewed process failures on a weekly basis.
Through initial testing that involved an RN clinical manager and a physician, we determined that a designated clinical RN would be the most consistent team member to assess daily gaps and contact the inpatient team. By assessing and verifying vaccine status of admitted patients, the RN works at the highest scope of practice, seeking advice from a designated physician when they had questions. The redesign also allowed expansion of the process to weekends when the clinic was open, further improving the consistency of contact with care teams. With input and testing from several core nurses that typically serve in a lead RN clinical role, the team developed standard work instructions. After 90 days, the team had trained >80% of the RNs on use of these standards, which provided coverage 6 days of the week.
Measures
The primary outcome measure was the percentage of gaps closed during the hospitalization plotted monthly on a statistical process control chart (P-chart). The unit of measurement was the individual care gap. The following examples are illustrative: Patient A was due for hepatitis B and polio vaccines and lead screening; therefore, the patient had 3 gaps total. If vaccines were given but lead was not obtained, 2 of the 3 gaps would be closed. Patient B was due for hepatitis B, polio, varicella, pneumococcal, and hepatitis A vaccines, and, therefore had 5 gaps total. If 4 of the vaccines were completed, 4 gaps would be closed. The total between the 2 patients would be 8 gaps, 6 of which were closed (75%).
We also developed a process measure to track the proportion of patients with gaps that the nurse appropriately identified and contacted HM about. The unit of measurement was the patient, with the denominator as primary care patients, aged 2 to 66 months, with a vaccination or lead gap admitted to HM. We set the goal of 80% for this process measure because this reliability level indicates that a new stable system has been achieved.27
Analysis
We used standard probability-based rules to identify common versus special cause variation for the process and outcome measures.26 Eight or more consecutive points above or below the centerline were used to prompt a midline shift on the control charts. We used χ2 tests to examine differences between patients included in this study and those the same age concurrently admitted to the hospital but cared for at other primary care practices.
Ethical Considerations
The institutional review board reviewed the study protocol and determined it to be exempt with waiver of informed consent.
Results
Cumulatively, from July 2017 to April 2019, 648 of 1867 (34.7%) care gaps were closed in 1061 patients (Table 1). When compared with the patients from other primary care practices, the included patients were slightly older and were more likely to be male, African American, and on public insurance (Table 2).
Care Gap Closed
| Care Gap | No. Care Gaps Closed | No. Care Gaps | % Care Gaps Closed |
|---|---|---|---|
| Vaccinations | 627 | 1760 | 35.6 |
| Lead | 21 | 107 | 19.6 |
| Total | 648 | 1867 | 34.7 |
| Care Gap | No. Care Gaps Closed | No. Care Gaps | % Care Gaps Closed |
|---|---|---|---|
| Vaccinations | 627 | 1760 | 35.6 |
| Lead | 21 | 107 | 19.6 |
| Total | 648 | 1867 | 34.7 |
Demographics of Admitted Patients: General Pediatrics and Non–General Pediatrics
| Variable | CCHMC Primary Care Patients (n = 1061), n (%) | Non-CCHMC Primary Care Patients (n = 6706), n (%) | P |
|---|---|---|---|
| Child age | <.001 | ||
| 0–11 mo | 495 (46.7) | 3459 (51.6) | .02 |
| 1 y | 259 (24.4) | 1376 (20.5) | .02 |
| 2 y | 95 (9.0) | 766 (11.4) | .10 |
| 3 y | 100 (9.4) | 504 (7.5) | .19 |
| 4 y | 73 (6.9) | 400 (6.0) | 1 |
| 5 y | 39 (3.7) | 201 (3.0) | 1 |
| Sex | .05 | ||
| Male | 621 (58.5) | 3707 (55.3) | .1 |
| Female | 440 (41.5) | 2999 (44.7) | .1 |
| Race | <.001 | ||
| African American/Black | 782 (73.7) | 989 (14.7) | <.001 |
| White | 216 (20.4) | 5085 (75.8) | <.001 |
| Hispanic/Latinx | 21 (2.0) | 280 (4.2) | .003 |
| Asian | 19 (1.8) | 138 (2.1) | 1 |
| Other | 14 (1.3) | 72 (1.1) | 1 |
| Unknown | 9 (0.8) | 142 (2.1) | .06 |
| Insurance type | <.001 | ||
| Public | 951 (89.6) | 3357 (50.1) | <.001 |
| Private | 46 (4.3) | 2874 (42.9) | <.001 |
| Self-pay or none | 64 (6.0) | 475 (7.1) | .63 |
| Variable | CCHMC Primary Care Patients (n = 1061), n (%) | Non-CCHMC Primary Care Patients (n = 6706), n (%) | P |
|---|---|---|---|
| Child age | <.001 | ||
| 0–11 mo | 495 (46.7) | 3459 (51.6) | .02 |
| 1 y | 259 (24.4) | 1376 (20.5) | .02 |
| 2 y | 95 (9.0) | 766 (11.4) | .10 |
| 3 y | 100 (9.4) | 504 (7.5) | .19 |
| 4 y | 73 (6.9) | 400 (6.0) | 1 |
| 5 y | 39 (3.7) | 201 (3.0) | 1 |
| Sex | .05 | ||
| Male | 621 (58.5) | 3707 (55.3) | .1 |
| Female | 440 (41.5) | 2999 (44.7) | .1 |
| Race | <.001 | ||
| African American/Black | 782 (73.7) | 989 (14.7) | <.001 |
| White | 216 (20.4) | 5085 (75.8) | <.001 |
| Hispanic/Latinx | 21 (2.0) | 280 (4.2) | .003 |
| Asian | 19 (1.8) | 138 (2.1) | 1 |
| Other | 14 (1.3) | 72 (1.1) | 1 |
| Unknown | 9 (0.8) | 142 (2.1) | .06 |
| Insurance type | <.001 | ||
| Public | 951 (89.6) | 3357 (50.1) | <.001 |
| Private | 46 (4.3) | 2874 (42.9) | <.001 |
| Self-pay or none | 64 (6.0) | 475 (7.1) | .63 |
Early PDSAs were focused on the feasibility of identifying primary care patients admitted to HM, evaluating gaps in those patients, and successfully contacting the inpatient team to prompt vaccine administration before discharge. These early efforts to improve identification, evaluation, and contact resulted in an increase in the outcome measure from 12% of gaps closed to 31% in July 2017 (Fig 3).
Monthly annotated statistical process control chart (P-chart) depicting the percentage of total eligible care gaps closed during hospitalization for general pediatrics patients admitted to HM. COVID-19, coronavirus disease 2019.
Monthly annotated statistical process control chart (P-chart) depicting the percentage of total eligible care gaps closed during hospitalization for general pediatrics patients admitted to HM. COVID-19, coronavirus disease 2019.
We attempted to initiate the process measure to match the time line of the outcome measure and the pilot testing. Limitations to capturing real-time data prevented us from accurately measuring the percentage of patients admitted with gaps that were assessed during the pilot phase. We quickly realized we needed to implement a new, defined process, consistently performed daily as part of standard work within the clinic to make the effort sustainable. During the clinic workflow implementation phase, we measured the process and assessed every failure, allowing us to examine Pareto diagrams and target interventions in our most common failure categories. Through this process of failure review, the process measure increased from its baseline of 62% of patients assessed for gaps to 88% assessed in 2.5 months (Fig 4). Improvement in the process measure led to further improvement in the outcome measure, from 31% of gaps closed to 41% in September 2018.
Control chart depicting the percentage of all general pediatric patients admitted to HM with gaps in care who were assessed by the clinic team during their inpatient admission, plotted in groups of 10 patients due to variability of admissions over time.
Control chart depicting the percentage of all general pediatric patients admitted to HM with gaps in care who were assessed by the clinic team during their inpatient admission, plotted in groups of 10 patients due to variability of admissions over time.
Analysis of retrospective data from May 2016 to September 2019 revealed an average of 30% of the remaining vaccinations and lead gaps not closed during the hospitalization were closed during a subsequent primary care visit up to 1 month after discharge.
Qualitatively, the types of failures noted changed over time (Fig 5). During the development of the process measure and initial testing, the most common failure was related to not having staff assigned to review the report both during the week and on the weekend. After additional RNs were trained and the role became part of the daily assigned work, the most common cause of failure changed to patients not appearing on the report because of brief length of stay (same-day admission and discharge). Interestingly, after an EHR upgrade, a new failure category emerged in which the gaps were not correctly populating on the report. The failures in this category took several months to resolve, but, in most cases, the RNs were double-checking to confirm the patient’s correct gaps in the EHR. Failures still occurred if the report listed no care gaps.
Changes in monthly process failures over time. Early failures occurred from staffing levels, lack of trained staff to perform the role, and weekend coverage. These failures decreased as more RNs were trained and coverage for the role was expanded. From July to September, a new problem appeared after an EHR upgrade with incorrect care gaps listed on the report. The problem was mitigated in early October 2018.
Changes in monthly process failures over time. Early failures occurred from staffing levels, lack of trained staff to perform the role, and weekend coverage. These failures decreased as more RNs were trained and coverage for the role was expanded. From July to September, a new problem appeared after an EHR upgrade with incorrect care gaps listed on the report. The problem was mitigated in early October 2018.
Special cause has been noted in 2 separate months since full implementation, evidenced by the measurement above and below the upper and lower control limits, respectively. We investigated potential causes and found a relationship with the number of children admitted with multiple care gaps (>4). The month below the lower control limit had children admitted with multiple gaps over winter holidays when the clinic was not fully staffed, leading to the abnormally low percentage.
Discussion
Using QI methodology, we designed, tested, and implemented an inreach model that increased the percentage of care gaps closed in a pediatric medical system. The primary care team identified gaps and notified an inpatient care team of opportunities to close gaps. Weekly learnings from robust measurement and failure data allowed us to target specific areas for interventions and improve the percentage of patients with gaps who were assessed by the primary care team. The creation of a culture for early identification and mitigation of problems has allowed us to sustain reliable process improvements.
At project outset, we had a strong belief that improvement in gap closure through the administration of vaccinations and lead screening while patients were admitted was possible. In observation at CCHMC, gap closure was already happening without any standard process in place, as evidenced by a baseline of 12% gap closure, not 0%.
A recognized hurdle for this work was the complexity of the vaccination regimen. This barrier was described in a previous pilot program aimed at vaccinating inpatient children. That program evaluated vaccination status, and whether needed vaccines were either given before discharge or recommended at a follow-up encounter in primary care within one month. In that study, most children who received vaccinations received them while admitted.17 Bell et al15 also described a program in which inpatient children were successfully vaccinated. That program required a person up to 5 hours per day to contact patients’ primary providers’ offices for vaccination records, which was a recognized limitation.15 By starting the process in the primary care physician office using the EHR and streamlining communication, we were able to close preventive care gaps without adding new personnel.
A key to success was the nursing team taking an active role in assessment and decision-making, allowing them to function at the top of their scope of practice. As the process became more stable and sustainable, the RNs demonstrated ownership in facilitating gap closure, often contacting teams on other units not included in the measure. The RNs were instrumental in recognizing and escalating system problems for both identifying gaps and for reliable measurement.
The project did have several limitations preventing further gap closure. One of the primary causes of failure is the limitation of reporting currently available. If a patient is admitted after midnight (when the data for the report are updated), that patient will not appear on that morning’s report for the RNs to review. With brief hospitalizations, some of those children are discharged before review the next day. EHR reporting capabilities continue to evolve, which may allow the opportunity to optimize the process through earlier identification of admitted patients’ gaps, potentially automating decision support for the inpatient team.
There are also limitations to the measure created to evaluate the success of the QI project. It was modeled to match coding for rules in the EHR to determine when patients are due for lead screens or vaccinations. We were not able to identify which patients would have come to clinic, so we could not judge how many gaps might have been closed without our new process. Pahud et al17 listed similar limitations in their intervention pilot program in limiting vaccination administration to during the hospitalization or 1 month post discharge. As in the previous study, it is possible that patients in our study would have been caught up without our intervention during admission.17 This intervention on inpatients is also likely better suited to specific procedures, like needed vaccinations, laboratories, or imaging. The team felt other gaps, like developmental screening, were better addressed in the primary care office where longitudinal relationships with the patients and families exist.
Whereas we were not able to quantify specific balancing measures like cost or staff time, we did not add any new staff, and the process was not considered time intensive once streamlined and in the regular, ongoing operations of clinic. Aspects of our intervention (eg, specific messaging platform and EHR) and dedicated QI personnel may not be available in every context. However, the drivers of primary care–inpatient collaboration, real-time identification of gaps, closed-loop communication, and activated, knowledgeable nurses are likely generalizable to other locations.
Conclusions
A new process to close preventive care gaps during hospital admissions can be successfully implemented. During the coronavirus disease 2019 pandemic, such gaps have increased from delayed well-child care, and many of the areas hardest hit by the pandemic in this country were already at risk for vaccination delay.28 As next steps, we are scaling our inreach strategy to include other safety-net providers in the All Children Thrive Learning Network and offering to draw blood for a lead level to those inpatients due for lead screening.
Acknowledgments
We thank the nurses in the pediatric primary care center for their diligence in identifying patients behind in care, communicating gaps to the inpatient team, and quickly identifying problems in the process for early escalation and mitigation. We also thank the residents and attending physicians on the HM services for working with our team to improve care for our shared patients, especially Drs Angela Statile and Ndidi Unaka for their leadership.
Dr Morehous and Ms White conceptualized and designed the study, drafted the initial manuscript, and reviewed and revised the manuscript; Ms Reyner designed the data retrieval program, analyzed the data, and reviewed and revised the manuscript; Dr Brinkman contributed to the interpretation of data and revised the manuscript for important intellectual content; Drs DeBlasio and Iyer and Ms Kleiman conceptualized and designed the study and critically reviewed the manuscript for important intellectual content; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
FUNDING: No external funding.
Comments