Pediatricians’ Participation in Quality Improvement at the Time of Enrollment in MOC

Pediatric diplomates enrolling (new or continuing enrollment) in the ABP’s MOC program during calendar year 2016 were provided with the opportunity to complete a brief survey. Completion of the survey was not tied to reports on completed MOC activities, receipt of MOC points, or certification status. Those who were continuing their MOC enrollment for the next 5-year cycle were required to have participated in QI activities as part of their previous 5-year MOC cycle.

Demographic characteristics were obtained from the ABP’s certification management database and included sex, graduation year, academic affiliation, and generalist or subspecialist status. The survey also included items regarding respondents’ self-reported QI participation in the past year, motivations and beliefs about QI, perceptions regarding infrastructure support for QI in their workplace setting, and QI skills. Respondents rated the importance of 7 factors, identified from the literature^10,11 and by QI experts who reviewed the survey drafts, on their decision to participate in QI projects in general as follows: “not at all important,” “somewhat important,” or “very important.” These factors were conceptualized as intrinsic (personal) or extrinsic (external or environmental) motivators for QI participation. Respondents were asked to indicate the extent to which they agreed or disagreed with 5 statements about QI in general and 5 statements about infrastructure support for QI activities in their work or practice setting. Lastly, respondents were asked to rate their self-confidence in 3 QI-related skills on a 5-point scale ranging from no confidence to high confidence.

Survey responses were linked to demographic data and deidentified by the ABP before transmission to the Child Health Evaluation and Research Center in Microsoft Excel format (Redmond, WA). The Child Health Evaluation and Research team reviewed data for accuracy before importing into SAS 9.4 (SAS Institute, Inc, Cary, NC) for statistical analyses.

Descriptive statistics, including frequencies and percentages, were calculated for demographic variables and practice characteristics (academic affiliation, generalist or subspecialist status). To test our hypotheses about academic affiliation and generalist or specialist status, we categorized participants in 1 of 4 practice affiliation categories: (1) academic generalists, (2) academic specialists, (3) nonacademic generalists, or (4) nonacademic specialists. Academic physicians were identified as any respondent who reported they were either full-time faculty (29%) or part-time faculty (7%). Graduation year was dichotomized as <2002 or ≥2002 on the basis of the increased focus regarding health care quality and the integration of QI programs into medical education after the publication of the Institute of Medicine report Crossing the Quality Chasm in 2001. The proportion of respondents indicating that specific intrinsic and extrinsic factors were “very important” motivators, that selected “agree/strongly agree” with the statements about QI, and who had moderate or high confidence in QI skills were calculated. χ² statistics were calculated to test for associations between demographic and practice characteristics and responses to questions about QI. P values <.05 were considered to be statistically significant. The project was approved by the University of Michigan Institutional Review Board.

Survey responses were received from 8714 of the 11 890 diplomates who enrolled in MOC in 2016, for a response rate of 73.3%. Of those, analyses were conducted on the 8314 surveys completed by respondents who were clinically active (95.4%). Respondents were mostly women (65.0%), graduated before 2002 (63.5%), and did not have an academic appointment (64.10%) (Table 1). Generalists were more prevalent than subspecialists (56.4% vs 43.6%). Academic appointments were less prevalent among generalists than specialists (16.6% vs 60.8%). For over half of the respondents (54%), this was the first time they had enrolled in MOC.

Overall, 86.6% of respondents reported participation in at least 1 QI project in the previous year. Half (52.3%) of respondents were involved as a participant only, 22.5% were project leaders only, and 11.8% had served in both a participant and a lead role. There was variation in previous-year participation in a QI project by practice affiliation categories, ranging from 79.9% for nonacademic generalists to 92.4% for academic specialists.

There were only small demographic differences among those who did and did not participate in a QI activity in the previous year. Overall, 88% of male respondents and 86% of female respondents participated. Specialists were more likely to participate compared with generalists (90.2% vs 83.7%). The proportion of those who had participated in QI was slightly higher for those who had graduated in 2002 or later compared with those graduating before 2002 (88% vs 86%). There was no difference in rates of QI participation among those who had (87%) or had not (87%) previously enrolled in MOC.

Table 2 provides information on the participation of respondents in different types of QI projects and demonstrates differences across practice affiliation categories among the subset of respondents who had participated in a QI project in the past year (n = 7196). Most who participated in QI projects did so through their practice or employer (58.6%), online via the ABP’s Performance Improvement Modules (35.8%), or through self-initiated projects (28.4%). Fewer did so through collaboratives (21.2% via professional organizations such as the American Academy of Pediatrics (AAP) and 9.2% via a collaborative improvement network) or the AAP’s Education in Quality Improvement for Pediatric Practice courses (16.0%). The greatest contrasts in participation by practice affiliation categories were observed for academic specialists and nonacademic generalists in practice- or employer-based programs (68.5% vs 51.1%; P < .0001) and in self-initiated projects (41.2% vs 19.0%; P < .0001). Much less variation was observed for the other demographic variables including graduation year (data not shown).

The extrinsic requirement for QI to maintain board certification was the dominant motivator among all respondents (50.7%), followed by 2 intrinsic factors: identify gaps in practice and implement change (40.3%) and opportunity to collaborate with others (36.9%). Nearly all respondents agreed that practice improvement is a professional responsibility, and three-fourths agreed that QI improves care. Although statistically significant, differences in motivations and beliefs did not vary in meaningful ways by practice affiliation category (Table 3) nor by any of the other demographic variables including graduation year (data not shown).

The majority of respondents perceived their workplace to encourage QI participation (78.5%) and to provide adequate support with information technology (IT) (67.7%), education (61.3%), and human or fiscal resources (57.8%). However, less than half of respondents perceived their workplace as providing adequate time for QI activities (48.5%; Table 3). Agreement that the work or practice setting encourages QI participation was greatest among academic specialists and lowest among nonacademic generalists (87.8% vs 72.0%; P < .0001), whereas agreement that the workplace provides adequate IT support to access and interpret clinical data was lowest among academic specialists and greatest among nonacademic generalists (64.3% vs 70.6%; P < .0001). Perceptions of the workplace providing adequate time for QI activities differed little among generalists or specialists or among those who graduated before 2002 and those graduating in 2002 or later. However, the number of respondents who perceived their workplace as supportive of QI participation was much lower among those who did not participate in a QI activity in the previous year. Only 54.7% agreed that their employer encouraged their participation in QI compared with 82.1% of those who were involved in a QI project.

A greater proportion of respondents felt at least moderately confident in their use of data to track changes (46.9%) compared with their use of QI methods to test strategies (34.6%) and use of QI tools to inform improvement efforts (28.8%). This pattern was consistent across practice affiliation categories (Fig 1), with academic specialists having the highest confidence in each skill and nonacademic generalists having the lowest (P < .0001 for each comparison). Confidence in use of data to track changes was higher among those who graduated before 2002 but did not differ by year of graduation for use of QI tools or QI methods (data not shown).

With our data, we show that efforts by regulators of hospital and practice settings, training programs, and pediatric certification to encourage participation in QI have resulted in most clinically active pediatricians who enrolled in the ABP’s MOC program in 2016 reporting engagement in QI work in the previous year. However, despite almost 2 decades of national attention to the quality of health care and efforts to embed continual QI in health care delivery activities, ∼1 in 5 nonacademic pediatricians and 1 in 10 academic pediatricians did not report participating in any QI activities in the previous year. No differences were noted by whether the respondent was initially enrolling in MOC or re-enrolling. In addition, because the sample for this study was composed of those enrolling in the MOC program, which requires QI activity, it is likely that the rate of QI participation among non-MOC participants is lower.

The level and type of involvement varied across practice affiliation categories. The greatest contrasts were seen between academic specialists and nonacademic generalist pediatricians, with online activities sponsored by the ABP and AAP endorsed more often by nonacademic generalists.

Many individuals engage in specific behaviors because of external requirements or perceptions they have regarding outside influences on their actions. With regard to our respondents, external motivations for QI work were rated as very important by more respondents compared with the intrinsic motivations. More generalists rated the requirement of QI to maintain certification and the ability to obtain continuing medical education credit as very important compared with specialists, regardless of academic appointment. Conversely, fewer generalists rated the intrinsic motivations of identifying gaps in practice and collaboration as very important compared with specialists.

Although pediatricians almost universally identified improvement as a professional responsibility, one-fourth still do not endorse the notion that QI improves care. For this proportion of pediatricians who doubt the potential impact of QI, external motivations may be necessary to initiate the process of incorporating QI into practice. This must occur concurrently with efforts to demonstrate the impact of QI on clinical care because <50% of our sample reported that their experience participating in QI has positively changed their perspectives on the value of QI. The onus is on those who develop QI programs to ensure that the impact and value of improvement in patient care is transparent to participants. Once value of the effort becomes apparent, intrinsic motivation may begin to drive more behaviors in this arena. Methods to strengthen internal motivations for QI work may result in more persistent behavior change to continuously assess quality of care and identify opportunities for improvement than the external motivations that are currently driving forces behind QI participation.^12,–14 

QI requirements are happening in the face of intense pressures on work hours and at the same time as increases of other requirements, including documentation, administration, and learning electronic medical record (EMR) systems. EMRs have the potential to increase access to data to measure clinical care, but currently there are few rigorous measures of pediatric quality that can be calculated from EMR and administrative data.^15,–26 Interestingly, the perceived levels of employer or workplace encouragement for QI are higher than the actual time, resources, or educational support for QI methods indicated by our respondents. This finding suggests that at least some pediatricians are not provided with the tools needed to implement QI activities in their workplace. It will be essential for employers and institutions to ensure that pediatricians are provided with the resources needed to succeed in incorporating QI into their practices. For those pediatricians in small or independent practices, shared QI infrastructure through practice associations, AAP chapter activities, or other vehicles may provide important mechanisms for providing QI infrastructure. However, additional efforts are required to continuously develop new mechanisms to build capacity, incentivize QI participation, and increase the feasibility for physicians to participate in meaningful activities to improve care.

Our findings highlight variation in the level of QI involvement across practice affiliation categories. In particular, we identified lower levels of engagement in QI by nonacademic pediatricians that may be the result of less access to practice- or employer-based QI programs, limited infrastructure support for QI, or a lack of self-confidence in their ability to initiate projects. Our findings signal greater opportunity to (1) increase QI skills through additional educational venues for trainees and for physicians in practice and (2) increase support for QI work in nonacademic settings.

Residency and fellowship training now include required QI project education and experiences.²⁷ Training programs have the potential to increase confidence in QI.^28,29 Although we did not observe meaningful differences in QI participation by year since graduation, it is possible that there has been a more gradual implementation of QI into the curriculum of residency and fellowship programs that will only show results in the coming years. The competence of trainees in QI and their confidence in the skills to implement change in practice and ultimately improve the quality of care may increase as more individuals who were required to complete QI projects during training enter the workforce.²⁷ In particular, the launch of the ACGME’s Clinical Learning Environment Review program in 2012³⁰ may increase greater QI exposure during residency and build faculty capacity to mentor trainees on QI projects. Clinical Learning Environment Review’s impact could not be measured with the current cohort. Future researchers should examine the effectiveness of such efforts during training.

Pediatricians who practice in academic settings also may gain exposure to QI methods through educational initiatives^31,32 offered at the national meetings of the Pediatric Academic Societies and the AAP or through quality-specific conferences such as the Conference on Advancing Implementation and Quality Improvement Science for Children's Health Care. However, without additional support, either in the workplace or through other mechanisms, pediatricians may continue to doubt their ability to design and implement programs relevant to their practice.

One effort to improve practical QI skills among practicing physicians, pioneered by the Institute for Healthcare Improvement, has been the collaborative model.³³ Collaboratives are structured to facilitate shared learning across many teams working on the same problem.³⁴ QI collaboratives have been established in both primary and specialty care, but in our results, a greater involvement is shown in collaborative QI among pediatricians with academic appointments, in which collaboratives typically focus on specific disease processes. In contrast, networks that are practice based or affiliated with state or AAP chapter efforts offer opportunities for generalists to engage in QI projects in outpatient and inpatient settings.^35,–41 

There are several unanswered questions about QI in practice and as a part of MOC in pediatrics. The impact of financial pressures on QI initiatives in private practice and in other settings is unknown. There are costs to participate in QI initiatives and collaboratives, staff time is needed to collect data or use data collected through the electronic health record, and providers must devote time away from direct patient care to plan practice changes, review outcomes, and determine next steps for improving the quality of care.⁴² Therefore, understanding the net impact of any QI program on the actual care delivered to children will be essential. In addition, only a little over one-half of pediatricians endorsed that they could design QI projects relevant to their practice through available MOC part 4 QI options, and less than one-half endorsed that external credentialing was important to maintain the public trust in the quality of care. The medical community and regulatory credentialing bodies must continue to collaborate to develop QI programs that are relevant, practical, and feasible for clinicians.

This study is subject to the following limitations. First, we asked about involvement in QI projects in the previous year. This decreases the potential for recall bias but does not capture earlier QI projects that could have had impacts on patient care delivered by the individual. Second, our results only represent perspectives of individuals who have decided to pursue MOC. Individuals with lower levels of comfort with QI may be deterred from enrolling in MOC because of the requirement for projects related to practice improvement.

Nearly 2 decades after Crossing the Quality Chasm, our findings point to positive attitudes of pediatricians toward QI, but there has been limited diffusion of QI skills to practitioners or limited support from their institutions or employers to facilitate practice change through QI methods. There remains a translational gap between realization by practitioners and health systems that QI is important and the skills, experience, and time required to meaningfully participate in QI work. This gap remains an impediment to truly changing practice and improving the quality of care delivered to children in the United States.

AAP

American Academy of Pediatrics

ABP

American Board of Pediatrics

ACGME

Accreditation Council for Graduate Medical Education

EMR

electronic medical record

IT

information technology

MOC

Maintenance of Certification

QI

quality improvement