Board certification has been part of the social contract in which physicians commit to maintaining up-to-date scientific knowledge and improving the quality of patient care. However, the maintenance of certification program has been controversial. This review summarizes the philosophical underpinnings, published literature, recent improvements, and future directions of the American Board of Pediatrics maintenance of certification program.

After satisfactorily completing accredited training and passing an initial certifying examination, physicians in the United States become “board-certified” and designated as diplomates of a specialty board under the umbrella of the American Board of Medical Specialties (ABMS). The ABMS maintenance of certification (MOC) program extends the principles of learning and assessment beyond formal residency training and throughout the physician’s career. It requires the physician to demonstrate professionalism (part 1), engage in life-long learning (part 2), pass periodic medical knowledge assessments (part 3), and participate in quality improvement (QI) activities (part 4). The certification process is based on professionalism, defined as the willingness of members of the profession to declare (“profess”) to the public a practice based on the highest standards of competencies, integrity, and commitment to the patient’s interests ahead of the interests of the physician.1 

Although many physicians have improved care and earned MOC credit for these efforts, the MOC program itself has been controversial in the physician community. Some have sharply criticized a perceived lack of published literature in support of MOC, arguing that there is no documented benefit that would justify the time and expense required to maintain certification.2,3 Others question the entire philosophical underpinning of requiring physicians to demonstrate knowledge and a commitment to improve care as part of a certification process, arguing that a valid license and participation in continuing medical education (CME) is sufficient.2,4 

The purpose of this article is to address the rationale for MOC, review the existing literature on MOC, and explain the status and future direction of the American Board of Pediatrics (ABP) MOC program.

MOC reflects the long tradition of physician professionalism voluntarily placing the patient first. Paul Starr’s 1984 Pulitzer Prize–winning book, The Social Transformation of American Medicine, articulates the societal willingness to grant physicians respect, autonomy, self-regulation, and financial rewards in exchange for competence, altruism, moral behavior, and promotion of the public good.5,6 The Physician Charter, published in 2002 and now endorsed by over 140 medical organizations, reframes the social contract for the 21st century with the physician’s commitment to set and maintain standards of competence and integrity based on professional responsibilities, including updating scientific knowledge and improving the quality of care.1 The ABP is part of this self-regulatory framework, with the mission of providing “…assurance to the public that a general pediatrician or pediatric subspecialist has successfully completed accredited training and fulfills the continuous evaluation requirements that encompass the 6 core competencies” (from the ABP mission statement). The certifying boards have targeted QI as the primary vehicle for a catalytic effect on care, in part because of the urgent appeals from the National Academy of Medicine that the health care system make QI and patient safety a priority.7,10 

Every year, pediatric training programs declare ∼4700 pediatricians, including pediatric subspecialists, competent for independent practice. This decision is based on years of close observation and assessment by faculty in the training programs, which are overseen by the Accreditation Council of Graduate Medical Education (ACGME). In 1999, the ACGME introduced a structured framework for these assessments based on the 6 core competencies of professionalism, medical knowledge, practice-based learning and improvement (ie, QI), systems-based practice, patient care, and communication.11 The ABMS codesigned the core competencies with ACGME recognizing that the exponential expansion of medical knowledge and the growing societal expectation to improve practice and increase patient safety could not be satisfied by certification solely at graduation from training. Rapid changes in knowledge and practice would require the application of core competencies throughout a physician’s career.7,10,12 Therefore, the certifying boards moved from a single lifetime certificate to a requirement for MOC. The rationale for objective, external assessment as part of professional self-regulation arises because self-assessment of performance correlates so poorly with objective external assessment.13 

Physicians care about the quality of patient care and have therefore rightly asked for published data on the relationship of MOC to patient care processes and outcomes. Without attempting a systematic review, PubMed and the Cochrane Library were queried for English-language, peer-reviewed journal articles published between January 1, 2000 until December 22, 2016 that addressed the relationship between MOC and patient care. Articles were included if (1) they described at least 1 patient care process or patient outcome and (2) participating physicians received MOC credit by an ABMS member board. In some cases, the description of clinical data and the role of MOC were separated into companion papers. Opinion pieces were excluded, as were articles that (1) represented specialties dissimilar to pediatrics in clinical context, training, or outcome measures (eg, all surgical specialties, anesthesiology, radiology, and pathology) and (2) focused on MOC physician engagement, psychometric analysis, or performance predictors. The search for the phrases “maintenance of certification” or “physician certification” yielded 276 citations, of which 263 were eliminated because they lacked either a documented clinical process or outcome, could not be linked to MOC, reflected specialties dissimilar to pediatrics, focused on nonclinical aspects of MOC, or constituted commentaries. Twenty-one additional citations meeting inclusion criteria were obtained from the author’s personal library. The resulting 34 citations consisted of 33 cohort or QI studies (Tables 1 and 2) and 1 systematic review. Two-thirds of the studies were published within the past 24 months, suggesting a quickening pace of publication on the relationship of MOC to clinical processes and outcomes.

TABLE 1

Retrospective Cohort Studies on the Association Between MOC and Clinical Processes or Outcomes

SourceSpecialtyMD Sample Size, NResultsMOC Associated With Better Clinical Processes/Outcomes?
Holmboe et al (2008)14  Internal medicine 3602 Top quartile score on MOC exam associated with greater likelihood of performing recommended diabetes care (composite measure OR, 1.17; 95% CI, 1.07–1.27) and mammography screening (OR, 1.14; 95% CI, 1.08–1.21) than physicians in the lowest quartile, even after adjustment for multiple factors. No difference among the groups in lipid testing of patients with cardiovascular disease (OR, 1.00; 95% CI, 0.91–1.10). 
Turchin et al (2008)15  Internal medicine 301 Probability of appropriate treatment intensification decreased by 21.3% for every decade since last board certification. (P = .0097). 
Holmboe et al (2010)16  Internal medicine 236 Higher MOC exam scores associated with better performance on the overall (r = 0.19; P < .01), chronic care (r = 0.14, P = .04), and preventive services composites (r = 0.17, P = .01). 
Hess et al (2012)17  Internal medicine 676 After controlling for physician and patient characteristics, MOC examination scores (particularly in the endocrine content domain) were significantly associated with the diabetes outcomes and process measures based on composite scores (β = 0.22, P < .001). 
Gray et al (2014)18  Internal medicine 956 MOC internists vs 974 grandfathered internists The MOC requirement was not statistically associated with cohort differences in the growth of the annual ACSH rate (per 1000 beneficiaries, 0.1 [95% CI, −1.7 to 1.9]; P = .92), but was associated with a cohort difference in the annual, per-beneficiary cost growth of −$167 (95% CI, −$270.5 to −$63.5; P = .002; 2.5% of overall mean cost). N (no difference in clinical outcomes) 
Y (modest cost savings with MOC) 
Hayes et al (2014)19  Internal medicine 71 MOC internists vs 34 grandfathered internists After adjustment for practice site, panel size, years since certification, and clustering by physician, there were no differences in outcomes for patients cared for by internists with time-limited or time-unlimited certification for any HEDIS measure. 
Phillips et al (2016)20  Family medicine Improvement compared among 3 cohorts using different QI approaches: Measurable improvement in all 3 cohorts (PQRS, PPM, and PQRS/PPM). 
PQRS (N = 297)  Example of PPM pre- and postimprovement, % (95% CI): foot exam, 18.1% (17.4–18.8) and retina exam 15.9% (15.3–16.5). 
ABFM PPM (N = 7264)  PPM-only projects had greater improvement than PQRS-only projects on 4 of 6 process measures and greater hemoglobin A1c improvement (P < .05). 
Combined PQRS and PPM (N = 765)  PQRS and PQRS/PPM projects were associated with greater blood pressure and cholesterol control improvement than PPM only (P < .05). 
Wilson et al (2014)21  Emergency medicine 42 000 emergency physicians treating acute chest pain in the ED Hospitals with board-certified emergency physicians (all of whom must participate in MOC) have lower odds of missed acute MI (adjusted OR, 0.60; 99% CI, 0.50–0.73). 
SourceSpecialtyMD Sample Size, NResultsMOC Associated With Better Clinical Processes/Outcomes?
Holmboe et al (2008)14  Internal medicine 3602 Top quartile score on MOC exam associated with greater likelihood of performing recommended diabetes care (composite measure OR, 1.17; 95% CI, 1.07–1.27) and mammography screening (OR, 1.14; 95% CI, 1.08–1.21) than physicians in the lowest quartile, even after adjustment for multiple factors. No difference among the groups in lipid testing of patients with cardiovascular disease (OR, 1.00; 95% CI, 0.91–1.10). 
Turchin et al (2008)15  Internal medicine 301 Probability of appropriate treatment intensification decreased by 21.3% for every decade since last board certification. (P = .0097). 
Holmboe et al (2010)16  Internal medicine 236 Higher MOC exam scores associated with better performance on the overall (r = 0.19; P < .01), chronic care (r = 0.14, P = .04), and preventive services composites (r = 0.17, P = .01). 
Hess et al (2012)17  Internal medicine 676 After controlling for physician and patient characteristics, MOC examination scores (particularly in the endocrine content domain) were significantly associated with the diabetes outcomes and process measures based on composite scores (β = 0.22, P < .001). 
Gray et al (2014)18  Internal medicine 956 MOC internists vs 974 grandfathered internists The MOC requirement was not statistically associated with cohort differences in the growth of the annual ACSH rate (per 1000 beneficiaries, 0.1 [95% CI, −1.7 to 1.9]; P = .92), but was associated with a cohort difference in the annual, per-beneficiary cost growth of −$167 (95% CI, −$270.5 to −$63.5; P = .002; 2.5% of overall mean cost). N (no difference in clinical outcomes) 
Y (modest cost savings with MOC) 
Hayes et al (2014)19  Internal medicine 71 MOC internists vs 34 grandfathered internists After adjustment for practice site, panel size, years since certification, and clustering by physician, there were no differences in outcomes for patients cared for by internists with time-limited or time-unlimited certification for any HEDIS measure. 
Phillips et al (2016)20  Family medicine Improvement compared among 3 cohorts using different QI approaches: Measurable improvement in all 3 cohorts (PQRS, PPM, and PQRS/PPM). 
PQRS (N = 297)  Example of PPM pre- and postimprovement, % (95% CI): foot exam, 18.1% (17.4–18.8) and retina exam 15.9% (15.3–16.5). 
ABFM PPM (N = 7264)  PPM-only projects had greater improvement than PQRS-only projects on 4 of 6 process measures and greater hemoglobin A1c improvement (P < .05). 
Combined PQRS and PPM (N = 765)  PQRS and PQRS/PPM projects were associated with greater blood pressure and cholesterol control improvement than PPM only (P < .05). 
Wilson et al (2014)21  Emergency medicine 42 000 emergency physicians treating acute chest pain in the ED Hospitals with board-certified emergency physicians (all of whom must participate in MOC) have lower odds of missed acute MI (adjusted OR, 0.60; 99% CI, 0.50–0.73). 

ACSH, ambulatory care sensitive hospitalizations; HEDIS, Healthcare Effectiveness Data Information Set; MI, myocardial infarction; N, no; PPM, performance in practice module; PQRS, Physician Quality Reporting System; Y, yes;

TABLE 2

QI Studies for Which MOC Credit Was Awarded

SourceSpecialtyControl Group and SettingPhysician Sample SizeRepresentative ResultsMOC Acknowledged
Fiks et al (2016)22  General pediatrics Control group included. Single center. MOC: (N = 27); non-MOC: (N = 200) MOC participants significantly increased captured HPV vaccination opportunities relative to non-MOC clinicians by 5.7% for HPV dose 1 at preventive visits and by 0.7% and 5.6% for doses 1 and 2, respectively, at acute visits. 
Vernacchio et al (2014)23  General pediatrics Control group included. Single center. MOC group: (N = 56) non-MOC group: (N = 129) Significant improvement in: asthma action plan use (62.4% → 76.8% cohort 1, 50.6% → 88.4% cohort 2, 53.0% → 79.6% cohort 3) and asthma control tests (4.6% → 55.2% cohort 1, 9.0% → 67.8% cohort 2, 15.2% → 61.4% cohort 3). 
The % reduction in asthma exacerbations was significant for cohort 1 (37.8% → 19.9%, P = .0002), but did not reach statistical significance in cohort 2 (27.8% → 20.7%, P = .1) or cohort 3 (36.6% → 26.9%, P = .1). 
Galliher et al (2014)24  Family medicine Control group included (physicians from multiple practices enrolled in a registry). ABFM diabetes MOC module group: (N = 17) Although all groups improved over time, the MOC module groups showed significantly greater improvement in 11 out of 24 diabetes care processes and intermediate outcomes than physicians not completing MOC modules. 
Other ABFM MOC module group: (N = 20) 
No MOC module group: (N = 39) 
Simpkins et al (2007)25  Internal medicine Control group included (cluster randomized trial). Multiple practices. 8 practices (19 internists) in the intervention group Primary outcome measure: N (no effect of MOC on primary outcome measure of corticosteroid dispensation; mixed effects on secondary outcomes). 
8 practices (21 internists) in the control group No difference in dispensation of inhaled corticosteroids between intervention and control group patients (adjusted OR, 1.0; 95% CI, 0.64–1.56). 
Secondary outcome measures: 
Decreased likelihood of asthma action plan discussion among intervention group patients (36% vs 46%, adjusted OR, 0.67; 95% CI, 0.48–0.93). 
Increased likelihood of discussing asthma triggers among intervention group patients (77% vs 70%; adjusted OR, 1.62; 95% CI, 1.08–2.42). 
LaBresh et al (2014)26  General pediatrics and family medicine Control group included (cluster randomized trial) Intervention group (using MOC QI strategies): 16 practices A composite score of cardiovascular risk reduction improved by 13.4% in the intervention group (48.2% → 69.8%) compared with the control group (47.4% → 55.2%) (P = .01). 
National QI Collaborative. Non-MOC control group: 16 practices 
Anderson et al (2015)27  Pediatric cardiology National QI Collaborative (no control group) 52 cardiac centers Cumulative aggregate interstage mortality decreased by 44% from 9.5% at baseline (2008 to May 2013) to 5.3% (June 2013 to August 2014) after QI interventions for which MOC credit was offered. 
Duncan et al (2015)28  General pediatrics National QI Collaborative (no control group) 21 practices The preventive services score (summary score of 17 “Bright Futures” recommendations) increased significantly from 3.99 to 6.22 (P < .001) for both the 9-mo (7 measures) and 24-mo visits (8 measures). 
Miller et al (2011)29  Pediatric critical care medicine National QI Collaborative (no control group) 29 centers The average aggregate baseline PICU CLABSI rate decreased 56% over 36 mo from 5.2 CLABSIs per 1000 line days (95% CI: 4.4–6.2) to 2.3 CLABSIs per 1000 line days (95% CI: 1.9–2.9 CLABSIs per 1000 line days); P < .0001. N (but participants received MOC credit as part Children’s Hospital Association Quality Transformation Network)30  
Starmer et al (2014)31  Pediatric residency National QI Collaborative (no control group) 875 pediatric residents 23% relative reduction in the overall medical error rate (24.5 → 18.8 errors per 100 admissions, P < .001). N (but role of MOC as incentive acknowledged in a separate methods paper32
30% relative reduction in the rate of preventable adverse events (4.7 → 3.3 events per 100 admissions, P < .001) 
Crandall et al (2012)33  Pediatric gastroenterology National QI Collaborative (no control group) 6 centers Significantly increased remission rates for Crohn’s disease (55% → 68%) and ulcerative colitis (61% → 72%) N (but MOC acknowledged in methods paper34
Shaw et al (2013)35  General pediatrics National Improvement Network (no control group) 31 projects (average 10 participants per project) Sample results: 
 Developmental screening increased from 20.9% (baseline) to 50.8% (final measure) 
 Documented obesity self-management increased from 14.2% (baseline) to 60.7% (final measure) 
 Documented asthma action plan increased from 20.5% (baseline) to 60.7% (final measure) 
Sanabria et al (2015)36  General pediatrics State-wide collaborative (no control group) 5 practices; 28 physicians Chart review score (number of charts with documented completion of key clinical activities required for transitioning from adolescent to adult care) increased from baseline of 0.5–1.1 to 6.3–8.4 (cycle 3). 
Riley et al (2016)37  Adolescent primary care State-wide collaborative (no control group) 44 physicians Time alone with adolescent patient increased from 77.3% to 90.3% of patient encounters (P < .001). 
Explanation of minor consents and parental notifications increased from 44.9% to 88.6% (P < .001). 
Confidential risk behavior screen completion increased from 59.1% to 84.9% (P < .001). 
Brandt et al (2013)38  General pediatrics State-wide collaborative (no control group) 5 practices. 16 physicians BMI documentation increased from 49% to >90%. Nutrition counseling increased from 52% to 87%. Physical activity counseling increased from 39% to 77%. 
Documentation of weight category increased from 67% to 94%. 
Gittelman et al (2015)39  General pediatrics State-wide collaborative (no control group) 6 practices. 16 pediatricians Documented use of injury prevention screening tool increased from 0% (baseline) to 97.2% after interventions. 
John et al (2014)40  General pediatrics Single center (no control group) 29 practices. 120 providers Increased BMI documentation and nutrition counseling from 50% to 90%. 
Hayward et al (2016)41  Pediatric rheumatology and pediatric nephrology Single center (no control group) 30 providers (pediatricians and nurse practitioners). Attending pediatrician cosigned all orders. Pregnancy testing before cyclophosphamide infusion increased from 25% (baseline) to 100% after provider training, order set revision, and protocol/MOC launch 
Mittal et al (2014)42  Multidisciplinary pediatric team Single center (no control group) Unknown After bronchiolitis clinical practice guideline implementation, significant decreases in: chest radiograph usage (59.7% → 39%, P < .0001); bronchodilator use (27% → 14%, P < .002); steroid use (19% → 11%, P < .01); length of stay (2.3 d → 1.9 d, P < .05). 
Huang et al (2013)43  General pediatrics Individual physicians participating in PIM meeting ABP MOC requirements (no control group) 51 pediatricians Documented compliance with all 12 obesity management recommendations improved from 68% (baseline) to 93% (third cycle); P < .02. Y (PIM meeting ABP MOC requirements also treated as the QI intervention resulting in improvement) 
Sheu et al (2016)44  Pediatric gastroenterology Individual physicians participating in PIM meeting ABP MOC requirements (no control group) 134 pediatric gastroenterologists Improvement in: 
 performance of time-out before procedure (87.3% → 99.1%, P < .0001), 
 documentation of duodenal biopsies (63.3% → 96.1%, P < .0001), 
 communication of endoscopy report to PCP (69.9% → 96.1%, P < .0001). 
Peterson et al (2014)45  Family medicine Individual physicians participating in ABFM PIM (no control group) 7924 diabetes modules completed Improvement in % patients with: Y (ABFM MOC also treated as the QI intervention resulting in improvement) 
 hemoglobin A1c <7.0 (57.4% → 61.3%, P < .05) 
 foot exam (68% → 85.8%, P < .05) 
 retina exam (55.5% → 71.1%, P < .05) 
Peterson et al (2016)46  Family medicine Individual physicians participating in ABFM PIM (no control group) 7319 hypertension modules completed Improvement in % patients with: Y (ABFM MOC also treated as the QI intervention resulting in improvement) 
 controlled blood pressure (87.4% → 92.6%, P < .05), 
 low-sodium diet counseling (74.1% → 92.7%, P < .05), 
 exercise counseling (82.4% → 94.4%, P < .05). 
Kolasinski and Price (2015)47  Family medicine and internal medicine Individual physicians participating in ABMS multispecialty portfolio PIMs 52 family physicians and 19 internists Improvement in % patients with controlled blood pressure (79.5% → 84.6%, P = .05). Y (ABMS MOC also treated as the QI intervention resulting in improvement) 
Ford II et al (2016)48  Psychiatry Individual physicians participating in ABPN-approved PIM (no control group) 92 psychiatrists Improved documentation of patient: tobacco use (74% → 93%, P = .001), motivation (47% → 86%, P = .001), smoking cessation counseling (40% → 80%, P = .001), prescription of smoking cessation medication (53% → 70%, P = .001), and provision of information about prescribed medications (46% → 72%, P = .001). Y (ABPN MOC also treated as the QI intervention resulting in improvement) 
Lambing et al (2015)49  Family medicine Individual physicians participating in ABFM PIM on osteoporosis (no control group) 62 family physicians Improvement in: DXA measurement (66.1% → 82.8%, P = .002). Y (ABFM MOC also treated as the QI intervention resulting in improvement) 
Prescribed pharmacologic therapy (53.3% → 58.8%, P = .02). 
SourceSpecialtyControl Group and SettingPhysician Sample SizeRepresentative ResultsMOC Acknowledged
Fiks et al (2016)22  General pediatrics Control group included. Single center. MOC: (N = 27); non-MOC: (N = 200) MOC participants significantly increased captured HPV vaccination opportunities relative to non-MOC clinicians by 5.7% for HPV dose 1 at preventive visits and by 0.7% and 5.6% for doses 1 and 2, respectively, at acute visits. 
Vernacchio et al (2014)23  General pediatrics Control group included. Single center. MOC group: (N = 56) non-MOC group: (N = 129) Significant improvement in: asthma action plan use (62.4% → 76.8% cohort 1, 50.6% → 88.4% cohort 2, 53.0% → 79.6% cohort 3) and asthma control tests (4.6% → 55.2% cohort 1, 9.0% → 67.8% cohort 2, 15.2% → 61.4% cohort 3). 
The % reduction in asthma exacerbations was significant for cohort 1 (37.8% → 19.9%, P = .0002), but did not reach statistical significance in cohort 2 (27.8% → 20.7%, P = .1) or cohort 3 (36.6% → 26.9%, P = .1). 
Galliher et al (2014)24  Family medicine Control group included (physicians from multiple practices enrolled in a registry). ABFM diabetes MOC module group: (N = 17) Although all groups improved over time, the MOC module groups showed significantly greater improvement in 11 out of 24 diabetes care processes and intermediate outcomes than physicians not completing MOC modules. 
Other ABFM MOC module group: (N = 20) 
No MOC module group: (N = 39) 
Simpkins et al (2007)25  Internal medicine Control group included (cluster randomized trial). Multiple practices. 8 practices (19 internists) in the intervention group Primary outcome measure: N (no effect of MOC on primary outcome measure of corticosteroid dispensation; mixed effects on secondary outcomes). 
8 practices (21 internists) in the control group No difference in dispensation of inhaled corticosteroids between intervention and control group patients (adjusted OR, 1.0; 95% CI, 0.64–1.56). 
Secondary outcome measures: 
Decreased likelihood of asthma action plan discussion among intervention group patients (36% vs 46%, adjusted OR, 0.67; 95% CI, 0.48–0.93). 
Increased likelihood of discussing asthma triggers among intervention group patients (77% vs 70%; adjusted OR, 1.62; 95% CI, 1.08–2.42). 
LaBresh et al (2014)26  General pediatrics and family medicine Control group included (cluster randomized trial) Intervention group (using MOC QI strategies): 16 practices A composite score of cardiovascular risk reduction improved by 13.4% in the intervention group (48.2% → 69.8%) compared with the control group (47.4% → 55.2%) (P = .01). 
National QI Collaborative. Non-MOC control group: 16 practices 
Anderson et al (2015)27  Pediatric cardiology National QI Collaborative (no control group) 52 cardiac centers Cumulative aggregate interstage mortality decreased by 44% from 9.5% at baseline (2008 to May 2013) to 5.3% (June 2013 to August 2014) after QI interventions for which MOC credit was offered. 
Duncan et al (2015)28  General pediatrics National QI Collaborative (no control group) 21 practices The preventive services score (summary score of 17 “Bright Futures” recommendations) increased significantly from 3.99 to 6.22 (P < .001) for both the 9-mo (7 measures) and 24-mo visits (8 measures). 
Miller et al (2011)29  Pediatric critical care medicine National QI Collaborative (no control group) 29 centers The average aggregate baseline PICU CLABSI rate decreased 56% over 36 mo from 5.2 CLABSIs per 1000 line days (95% CI: 4.4–6.2) to 2.3 CLABSIs per 1000 line days (95% CI: 1.9–2.9 CLABSIs per 1000 line days); P < .0001. N (but participants received MOC credit as part Children’s Hospital Association Quality Transformation Network)30  
Starmer et al (2014)31  Pediatric residency National QI Collaborative (no control group) 875 pediatric residents 23% relative reduction in the overall medical error rate (24.5 → 18.8 errors per 100 admissions, P < .001). N (but role of MOC as incentive acknowledged in a separate methods paper32
30% relative reduction in the rate of preventable adverse events (4.7 → 3.3 events per 100 admissions, P < .001) 
Crandall et al (2012)33  Pediatric gastroenterology National QI Collaborative (no control group) 6 centers Significantly increased remission rates for Crohn’s disease (55% → 68%) and ulcerative colitis (61% → 72%) N (but MOC acknowledged in methods paper34
Shaw et al (2013)35  General pediatrics National Improvement Network (no control group) 31 projects (average 10 participants per project) Sample results: 
 Developmental screening increased from 20.9% (baseline) to 50.8% (final measure) 
 Documented obesity self-management increased from 14.2% (baseline) to 60.7% (final measure) 
 Documented asthma action plan increased from 20.5% (baseline) to 60.7% (final measure) 
Sanabria et al (2015)36  General pediatrics State-wide collaborative (no control group) 5 practices; 28 physicians Chart review score (number of charts with documented completion of key clinical activities required for transitioning from adolescent to adult care) increased from baseline of 0.5–1.1 to 6.3–8.4 (cycle 3). 
Riley et al (2016)37  Adolescent primary care State-wide collaborative (no control group) 44 physicians Time alone with adolescent patient increased from 77.3% to 90.3% of patient encounters (P < .001). 
Explanation of minor consents and parental notifications increased from 44.9% to 88.6% (P < .001). 
Confidential risk behavior screen completion increased from 59.1% to 84.9% (P < .001). 
Brandt et al (2013)38  General pediatrics State-wide collaborative (no control group) 5 practices. 16 physicians BMI documentation increased from 49% to >90%. Nutrition counseling increased from 52% to 87%. Physical activity counseling increased from 39% to 77%. 
Documentation of weight category increased from 67% to 94%. 
Gittelman et al (2015)39  General pediatrics State-wide collaborative (no control group) 6 practices. 16 pediatricians Documented use of injury prevention screening tool increased from 0% (baseline) to 97.2% after interventions. 
John et al (2014)40  General pediatrics Single center (no control group) 29 practices. 120 providers Increased BMI documentation and nutrition counseling from 50% to 90%. 
Hayward et al (2016)41  Pediatric rheumatology and pediatric nephrology Single center (no control group) 30 providers (pediatricians and nurse practitioners). Attending pediatrician cosigned all orders. Pregnancy testing before cyclophosphamide infusion increased from 25% (baseline) to 100% after provider training, order set revision, and protocol/MOC launch 
Mittal et al (2014)42  Multidisciplinary pediatric team Single center (no control group) Unknown After bronchiolitis clinical practice guideline implementation, significant decreases in: chest radiograph usage (59.7% → 39%, P < .0001); bronchodilator use (27% → 14%, P < .002); steroid use (19% → 11%, P < .01); length of stay (2.3 d → 1.9 d, P < .05). 
Huang et al (2013)43  General pediatrics Individual physicians participating in PIM meeting ABP MOC requirements (no control group) 51 pediatricians Documented compliance with all 12 obesity management recommendations improved from 68% (baseline) to 93% (third cycle); P < .02. Y (PIM meeting ABP MOC requirements also treated as the QI intervention resulting in improvement) 
Sheu et al (2016)44  Pediatric gastroenterology Individual physicians participating in PIM meeting ABP MOC requirements (no control group) 134 pediatric gastroenterologists Improvement in: 
 performance of time-out before procedure (87.3% → 99.1%, P < .0001), 
 documentation of duodenal biopsies (63.3% → 96.1%, P < .0001), 
 communication of endoscopy report to PCP (69.9% → 96.1%, P < .0001). 
Peterson et al (2014)45  Family medicine Individual physicians participating in ABFM PIM (no control group) 7924 diabetes modules completed Improvement in % patients with: Y (ABFM MOC also treated as the QI intervention resulting in improvement) 
 hemoglobin A1c <7.0 (57.4% → 61.3%, P < .05) 
 foot exam (68% → 85.8%, P < .05) 
 retina exam (55.5% → 71.1%, P < .05) 
Peterson et al (2016)46  Family medicine Individual physicians participating in ABFM PIM (no control group) 7319 hypertension modules completed Improvement in % patients with: Y (ABFM MOC also treated as the QI intervention resulting in improvement) 
 controlled blood pressure (87.4% → 92.6%, P < .05), 
 low-sodium diet counseling (74.1% → 92.7%, P < .05), 
 exercise counseling (82.4% → 94.4%, P < .05). 
Kolasinski and Price (2015)47  Family medicine and internal medicine Individual physicians participating in ABMS multispecialty portfolio PIMs 52 family physicians and 19 internists Improvement in % patients with controlled blood pressure (79.5% → 84.6%, P = .05). Y (ABMS MOC also treated as the QI intervention resulting in improvement) 
Ford II et al (2016)48  Psychiatry Individual physicians participating in ABPN-approved PIM (no control group) 92 psychiatrists Improved documentation of patient: tobacco use (74% → 93%, P = .001), motivation (47% → 86%, P = .001), smoking cessation counseling (40% → 80%, P = .001), prescription of smoking cessation medication (53% → 70%, P = .001), and provision of information about prescribed medications (46% → 72%, P = .001). Y (ABPN MOC also treated as the QI intervention resulting in improvement) 
Lambing et al (2015)49  Family medicine Individual physicians participating in ABFM PIM on osteoporosis (no control group) 62 family physicians Improvement in: DXA measurement (66.1% → 82.8%, P = .002). Y (ABFM MOC also treated as the QI intervention resulting in improvement) 
Prescribed pharmacologic therapy (53.3% → 58.8%, P = .02). 

Table entries sorted by control group and setting. ABIM, American Board of Internal Medicine; ABPN, American Board of Psychiatry and Neurology; DXA, dual radiograph absorptiometry; N, no; PCP, primary care physician; PIM, Performance Improvement Module; Y, yes.

The research on MOC has adopted 2 complementary perspectives. One perspective is to treat MOC or the MOC exam as an intervention and ask the question, “What is the relationship between MOC and outcomes for the patient or the learner?” The second perspective treats MOC as an incentive or marker of participation in professional activities ultimately designed to improve health care and asks, “What happens to health care processes and outcomes during QI activities for which MOC credit is awarded?”

Within the first perspective of MOC as an intervention, the tasks needed to complete the part 3 exam (learning) and the part 4 QI activity (gap analysis, learning, data collection, QI intervention, and data analysis) are treated as the MOC intervention that might affect a care process or outcome. The research addressing this question has largely come from internal medicine, family medicine, and emergency medicine by using cohort studies based on large insurance claims databases, such as the Medicare database.

In the studies adopting the second perspective, MOC credit informs the public (eg, parents, payers, hospital credentials, etc) of those physicians who have met a professional standard, and hence, MOC also acts as an incentive for participation. Most of these studies come from pediatrics, because the ABP has adopted the QI network or learning collaborative as the gold standard for its MOC part 4 activities.

Eight retrospective cohort studies involving a total of 57 176 diplomates have examined 9 separate processes or outcomes, of which 6 showed a positive association between MOC and the relevant process or outcome (Table 1). The positive associations involve better clinical processes, improved clinical outcomes, or cost savings among physicians participating in MOC or scoring well on the MOC exam.14,17,20,21 The largest physician cohort (N = 42 000) was evaluated by Wilson et al,21 who determined that board-certified emergency medicine physicians (all of whom are required to participate in MOC) were less likely to miss a diagnosis of acute myocardial infarction in the emergency department (ED) than noncertified physicians staffing the ED (odds ratio [OR], 0.6; 99% confidence interval [CI], 0.5–0.73). Consistent with these studies showing a positive association between MOC and clinical outcomes, a systematic review covering both the permanent certificate era and the time-limited certificate (MOC) era up until 2013 found that board certification was associated with better clinical care, but also noted methodologic heterogeneity and modest effect sizes.50 

Two studies showed either no association or a modest association between MOC and a beneficial outcome. Hayes et al19 found no significant difference in Healthcare Effectiveness Data Information Set measures for preventive care among internists participating in MOC compared with nonparticipants in MOC who worked within the Veterans Administration QI framework. Gray et al18 found no difference in ambulatory care sensitive hospitalization rates among internists participating in MOC versus nonparticipating internists, but did identify a modest cost saving achieved by the MOC group.

Table 2 summarizes the results from 25 QI studies, where MOC part 4 credit was awarded to diplomates who have engaged in a formal process to improve care. The data are reported as interrupted time series investigations with baseline data followed by several cycles of QI interventions and repeated outcome measurements. A total of 2090 diplomates were included in the 16 studies that specify the number of physician participants. Of the remaining 9 studies, 2 studies reported 7319 and 7924 performance improvement module (PIM) completions, respectively, but a 1:1 relationship between PIM completion and participating physician cannot be verified.45,46 Six studies list the number of participating centers or practices (N = 157). One report was from the National Improvement Partnership Network, which consisted of 15 statewide QI partnerships at the time of the report.35 

Five out of the 25 QI studies included control groups, and 2 out of these 5 were cluster randomized trials (Table 2).22,26 In the cluster randomized trial by LaBresh et al,26 the intervention group using the ABP MOC QI strategies significantly reduced a composite measure of cardiovascular risk in children compared with the control group. Conversely, the cluster randomized trial by Simpkins et al25 found no difference in the primary outcome measure of dispensed inhaled corticosteroids to adult asthmatic patients when the American Board of Internal Medicine MOC group was compared with the control group. Secondary outcome measures in the Simpkins et al25 study showed mixed results.

Among the 3 studies with nonrandomized control groups, Fiks et al22 found that MOC-participating pediatricians increased the vaccination capture rate for human papillomavirus (HPV) by up to 5.7% compared with nonparticipating pediatricians. Vernacchio et al23 determined that pediatricians participating in the MOC project increased the use of asthma action plans and asthma control tests significantly among their patients. The reduction in asthma exacerbations was significantly greater in 1 out of 3 cohorts of patients whose pediatricians participated in the MOC project compared with the patients under the care of nonparticipating pediatricians. The authors speculated that the absence of a difference in asthma exacerbations in the 2 remaining cohorts may have been due to diffusion of the MOC QI program knowledge or procedures to nonparticipating physicians in the same practice group.23 Galliher et al24 found that family physicians completing the American Board of Family Medicine (ABFM) MOC modules improved diabetes care and outcomes more than physicians who did not complete these modules. In summary, the majority of controlled studies showed improved processes or outcomes in the MOC intervention groups compared with the control groups.

The 20 QI studies without control groups employed standard QI methodologies to examine outcomes at baseline and then after ≥1 interventions (Table 2). The ABP took an active role in promoting the early development of national pediatric collaboratives, and several publications cite the importance of MOC credit in achieving the QI goals (Table 2). For example, the National Pediatric Cardiology Quality Improvement Collaborative, involving 52 pediatric cardiac centers, achieved a 44% relative reduction in interstage mortality after hypoplastic left heart syndrome repair and acknowledged the importance of MOC as an incentive for participation in the collaborative.27 Crandall et al33 reported that the ImproveCareNow collaborative significantly increased remission rates for inflammatory bowel disease. ImproveCareNow was the prototype learning collaborative for MOC created by the ABP.34 Similarly, Miller et al29 described a 56% reduction in central line–associated blood stream infections (CLABSI) among pediatric intensive care patients. A summary paper on exemplar learning collaboratives highlights that participating pediatric intensivists received MOC credit for reducing CLABSI rates through the Children’s Hospital Association Quality Transformation Network.30 Starmer et al31 developed the I-PASS handoff system, which resulted in the 23% reduction of medical errors among pediatric residents. A separate methods paper from the same authors described the use of MOC as an incentive for participation in I-PASS.32 Duncan et al28 showed that a QI collaborative increased the use of “Bright Futures” preventive services during child well-visits. This collaborative functioned under the auspices of the American Academy of Pediatrics Quality Improvement Innovation Networks, for which MOC credit is awarded. The ABP gives MOC credit to faculty for supervising resident QI projects and an emerging body of literature has identified the value of MOC credit as an incentive to faculty who teach residents how to systematically improve the quality of their patient care.51,52 In summary, the preponderance of the existing literature supports the association of MOC with improved care and the importance of MOC credit as an incentive (Table 2).

The authors of MOC studies have faced challenges that are often present in education, QI, and implementation research.53 Although random assignment of subjects to an intervention and control group is considered a standard feature of high-quality research, the nature of board certification in the United States makes this criterion difficult to attain. Nearly all graduating residents attempt to become board certified, and confounding variables, such as age or graduating institution, separate those attaining and maintaining board certification from those who do not.54,55 

Pediatrics faces some special challenges in MOC research. These include the scarcity of validated pediatric quality measures, small sample sizes in pediatrics, and the absence of a national pediatric database akin to the Medicare database for adults.56 Even when these limitations are overcome, the measurable effect size of an individual physician’s contribution to an outcome is likely to be modest among the many other factors that affect patient outcomes, including access to care, patient compliance, the contributions of other health care team members, and genetic polymorphisms. These factors notwithstanding, physicians have a professional obligation not only to deliver the best care, but to attempt to continuously improve care in the practice, community, and society.57 In addition, the absence of validated quality measures need not always impede policymaking. For instance, a National Academy of Medicine committee on improving diagnosis in health care acknowledged the inability to measure the accuracy of diagnoses, but has nevertheless concluded that “…certification and accreditation organizations should ensure that health care professionals have and maintain the competencies needed for effective performance in the diagnostic process.”9 

The overall utility of any assessment program depends not only on the validity, but also on the acceptability of the program, particularly among those being assessed.58 The examples in Tables 1 and 2 show that many physicians have successfully embraced the challenges of 21st century professionalism and demonstrably improved care in the context of MOC, yet a number of physician surveys have registered dissatisfaction with MOC across all specialties.2,3,59,60 Specific concerns have included the perceived burden, relevance, cost, and complexity of the MOC program. At the same time, hospitals have increased their reliance on MOC as a criterion for privileging.61 A national survey of 3621 randomly chosen adult respondents indicated that 82% felt board certification was important or very important in the selection of a pediatrician.62 Regular assessments of the pediatrician’s quality of care and medical knowledge were important to 95% and 88% of members of the public, respectively.62 

The ABP and indeed all certifying boards have been trying to bridge this acceptability gap between physicians and the public by designing assessments that are relevant to practice and assigning credit to activities embedded into the daily physician workflow that increase learning or improve care. Improvements in the ABP MOC program illustrate this point.

The ABP has adopted an explicit value set that includes continuous improvement of its own programs. Through formal pediatrician focus groups and informal outreach, the ABP has solicited suggestions for the improvement of MOC. Hence, numerous improvements to the ABP MOC program have been added in the past 2 years and will be added in the coming years based on continuing dialogue with the pediatric community.

In response to diplomate requests to expand the number of options eligible for MOC part 2 credit, the ABP is collaborating with the Accreditation Council for Continuing Medical Education so that any CME activity that includes an assessment of learning and feedback to the learner may also offer MOC part 2 credit. Beginning in January 2017, participating CME providers will automatically notify the ABP once the pediatrician has completed the learning activity and received CME credit so that MOC part 2 credit can also be awarded by the ABP. Table 3 lists some of the types of learning activities and assessments that will qualify.

TABLE 3

Some Types of CME Activities and Assessments Qualifying for Automatic MOC Part 2 Credit

Activity TypesSample Assessment OptionsFeedback Method
Regularly scheduled series (eg, grand rounds, case conference, tumor board, etc) Quiz Feedback on best answers to quiz 
Audience response system Answer to each question is shared 
Peer or leader assessment and feedback Best practice is discussed and shared 
Journal-based CME Quiz Feedback on best answers to quiz 
Simulation Demonstrated skill by learner and observed by instructor Individual performance compared with best practice is discussed and shared 
Journal peer (manuscript) review Assessment and feedback from editor Editor provides feedback on the adequacy of the review 
Activity TypesSample Assessment OptionsFeedback Method
Regularly scheduled series (eg, grand rounds, case conference, tumor board, etc) Quiz Feedback on best answers to quiz 
Audience response system Answer to each question is shared 
Peer or leader assessment and feedback Best practice is discussed and shared 
Journal-based CME Quiz Feedback on best answers to quiz 
Simulation Demonstrated skill by learner and observed by instructor Individual performance compared with best practice is discussed and shared 
Journal peer (manuscript) review Assessment and feedback from editor Editor provides feedback on the adequacy of the review 

The MOC Assessment in Pediatrics (MOCA-Peds) grew out of an ABP conference on the “Future of Testing,” which surveyed recent developments in the field of assessment and noted the call for regulators to drive learning as part of the assessment.63 Therefore, the ABP has decided to build on pioneering work by the American Board of Anesthesiology and pilot a replacement for the secure test center exam that has offered 200 questions every 10 years.

With the pilot beginning in 2017, the MOC Assessment in Pediatrics will be a more continuous assessment and learning tool in which questions may be answered on a laptop or mobile device when it is convenient for the diplomate. Current plans call for 20 questions per quarter, which should not only give a more reliable assessment of knowledge (as more data points are collected), but also an enhanced learning opportunity with immediate feedback explaining the rationale for the correct answer and references for additional study. Although questions will need to be answered within a specific time and by the individual diplomate, books or online resources may be used when answering questions if so desired.

Over the last 3 years, the ABP has streamlined the QI component of MOC (part 4) with 3 goals in mind. First, diplomates need the flexibility to decide what areas of their professional practice to target for improvement efforts. Therefore, new MOC part 4 pathways emphasize the ability to “create your own” QI project with simplified documentation requirements. The ABP may also delegate approval authority for MOC part 4 to institutions (including American Academy of Pediatrics chapters) that have an MOC QI portfolio so that the relevant QI projects are decided by the institution, not the ABP. These efforts have resulted in a dramatic increase in the number of applications for QI credit–based activities designed by individuals, small groups, single institutions, or QI networks (Fig 1).

FIGURE 1

Number of ABP diplomates receiving MOC part 4 credit each year for QI projects initiated by the individual, team, network, or institution. New pathways to “create your own QI project” and to participate in a PCHM were introduced in 2015.

FIGURE 1

Number of ABP diplomates receiving MOC part 4 credit each year for QI projects initiated by the individual, team, network, or institution. New pathways to “create your own QI project” and to participate in a PCHM were introduced in 2015.

Second, many pediatricians focus their professional activities in specific areas, such as education, research, administration, or advocacy. All areas are important and may contribute to improved health for children. Therefore, the ABP awards MOC part 4 credit for any professional activity that uses QI methodologies to improve the health of children.

Third, the ABP is acutely aware of the multitude of regulatory requirements that all physicians face. Hence, MOC part 4 credit is specifically designed to award credit for activities the pediatrician is already doing in response to other requests. For instance, educators who improve the quality of a training program to meet ACGME program evaluation and improvement requirements will get simultaneous MOC part 4 credit for that activity. Pediatricians whose practices achieve recognition as a patient-centered medical home by the National Committee for Quality Assurance (NCQA) automatically satisfy MOC part 4 requirements for an entire 5-year cycle. The patient-centered medical home has been associated with improved outcomes at lower cost for children with chronic diseases.64 Participating pediatricians often qualify for incentive payments from payers. The NCQA has recently extended the medical home accreditation to specialty practices. For instance, if a hematology clinic at an academic medical center becomes accredited by NCQA, the participating hematologists may receive a full 5-year cycle of MOC part 4 credit. These examples illustrate QI activities highly relevant to pediatric practice that qualify for MOC part 4 credit.

The ABP has listened carefully to concerns about the cost of MOC. Fees have been frozen for the past several years, and there has been a longstanding policy to waive the MOC fee for the first 5 years after graduation from residency. Nevertheless, the principal complaint has been around charging a lump-sum fee ($1304) every 5 years for MOC, which was viewed as a burden by many and exceeded the limits of professional expense accounts offered by some pediatric practices. To address this concern, the ABP will also offer an annual payment option ($275) beginning in 2018. By comparison, the annual fee is lower than specialty society dues and compares favorably with the fees charged by other certifying boards. An economic analysis of MOC should also balance expense against income and acknowledge that the income potential of board-certified physicians exceeds that of noncertified ones.65 

Pediatrics has a distinguished history of placing the patients’ needs first, not only through the actions of individual pediatricians but as an entire profession willing to hold itself accountable to a set of professional standards. The rapid changes in scientific knowledge, technology, and society as well as high public expectations have called on our profession to demonstrate this commitment anew for the 21st century. Although not every study shows a positive effect of MOC and methodologic challenges remain, a rapidly growing body of literature has demonstrated that MOC is associated with better care or has been an incentive for physicians to collaborate in systematically improving patient care and outcomes. The ABP’s commitment to listen carefully to pediatricians and continuously improve the MOC program is leading to enhanced flexibility and relevance of the program without increasing costs to the diplomate. The ABP will continue to nurture relationships with pediatricians, patients, parents, and families with the shared purpose of healthier children for the future.

     
  • ABFM

    American Board of Family Medicine

  •  
  • ABMS

    American Board of Medical Specialties

  •  
  • ABP

    American Board of Pediatrics

  •  
  • ACGME

    Accreditation Council of Graduate Medical Education

  •  
  • CI

    confidence interval

  •  
  • CLABSI

    central line–associated blood stream infection

  •  
  • CME

    continuing medical education

  •  
  • ED

    emergency department

  •  
  • HPV

    human papillomavirus

  •  
  • MOC

    maintenance of certification

  •  
  • NCQA

    National Committee for Quality Assurance

  •  
  • OR

    odds ratio

  •  
  • PIM

    performance improvement module

  •  
  • QI

    quality improvement

FUNDING: No external funding.

1
Medical Professionalism Project
.
Medical professionalism in the new millennium: a physicians’ charter.
Lancet
.
2002
;
359
(
9305
):
520
522
[PubMed]
2
Teirstein
PS
.
Boarded to death–why maintenance of certification is bad for doctors and patients.
N Engl J Med
.
2015
;
372
(
2
):
106
108
[PubMed]
3
Ackerman
KG
,
Lee
B
,
Kushner
JA
.
Dissatisfaction with maintenance of certification in academic pediatrics.
Pediatr Res
.
2016
;
79
(
2
):
240
242
[PubMed]
4
Strasburger
VC
.
Ain’t misbehavin’: is it possible to criticize maintenance of certification (MOC)?
Clin Pediatr (Phila)
.
2011
;
50
(
7
):
587
590
[PubMed]
5
Starr
P
.
The Social Transformation of American Medicine
.
New York, NY
:
Basic Books
;
1982
6
Cruess
SR
,
Cruess
RL
.
Professionalism: a contract between medicine and society.
CMAJ
.
2000
;
162
(
5
):
668
669
[PubMed]
7
Institute of Medicine (US)
.
Committee on Quality of Health Care in America
.
Crossing the Quality Chasm: A New Health System for the 21st Century
.
Washington, DC
:
National Academy Press
;
2001
8
Smith
M
,
Saunders
R
,
Stuckhardt
L
,
McGinnis
JM
, eds.
Best Care at Lower Cost: The Path to Continuously Learning Health Care in America
.
Washington, DC
:
National Academies Press
;
2013
9
Balogh
EP
,
Miller
BT
,
Ball
JR
, eds.
Improving Diagnosis in Health Care
.
Washington, DC
:
National Academies Press
;
2015
10
Kohn
LT
,
Corrigan
JM
,
Donaldson
MS
, eds.
To Err is Human: Building a Safer Health System
.
Washington, DC
:
National Academies Press
;
2000
11
Batalden
P
,
Leach
D
,
Swing
S
,
Dreyfus
H
,
Dreyfus
S
.
General competencies and accreditation in graduate medical education.
Health Aff (Millwood)
.
2002
;
21
(
5
):
103
111
[PubMed]
12
Attwood
TK
,
Kell
DB
,
McDermott
P
,
Marsh
J
,
Pettifer
SR
,
Thorne
D
.
Calling International Rescue: knowledge lost in literature and data landslide!
Biochem J
.
2009
;
424
(
3
):
317
333
[PubMed]
13
Zell
E
,
Krizan
Z
.
Do people have insight into their abilities? A metasynthesis.
Perspect Psychol Sci
.
2014
;
9
(
2
):
111
125
[PubMed]
14
Holmboe
ES
,
Wang
Y
,
Meehan
TP
, et al
.
Association between maintenance of certification examination scores and quality of care for Medicare beneficiaries.
Arch Intern Med
.
2008
;
168
(
13
):
1396
1403
[PubMed]
15
Turchin
A
,
Shubina
M
,
Chodos
AH
,
Einbinder
JS
,
Pendergrass
ML
.
Effect of board certification on antihypertensive treatment intensification in patients with diabetes mellitus.
Circulation
.
2008
;
117
(
5
):
623
628
[PubMed]
16
Holmboe
ES
,
Weng
W
,
Arnold
GK
, et al
.
The comprehensive care project: measuring physician performance in ambulatory practice.
Health Serv Res
.
2010
;
45
(
6 pt 2
):
1912
1933
[PubMed]
17
Hess
BJ
,
Weng
W
,
Holmboe
ES
,
Lipner
RS
.
The association between physicians’ cognitive skills and quality of diabetes care.
Acad Med
.
2012
;
87
(
2
):
157
163
[PubMed]
18
Gray
BM
,
Vandergrift
JL
,
Johnston
MM
, et al
.
Association between imposition of a maintenance of certification requirement and ambulatory care-sensitive hospitalizations and health care costs.
JAMA
.
2014
;
312
(
22
):
2348
2357
[PubMed]
19
Hayes
J
,
Jackson
JL
,
McNutt
GM
,
Hertz
BJ
,
Ryan
JJ
,
Pawlikowski
SA
.
Association between physician time-unlimited vs time-limited internal medicine board certification and ambulatory patient care quality.
JAMA
.
2014
;
312
(
22
):
2358
2363
[PubMed]
20
Phillips
RL
,
Blackburn
B
,
Peterson
LE
,
Puffer
JC
.
Maintenance of certification, Medicare quality reporting, and quality of diabetes care.
Am J Med Qual
.
2016
;
31
(
3
):
217
223
[PubMed]
21
Wilson
M
,
Welch
J
,
Schuur
J
,
O’Laughlin
K
,
Cutler
D
.
Hospital and emergency department factors associated with variations in missed diagnosis and costs for patients age 65 years and older with acute myocardial infarction who present to emergency departments.
Acad Emerg Med
.
2014
;
21
(
10
):
1101
1108
[PubMed]
22
Fiks
AG
,
Luan
X
,
Mayne
SL
.
Improving HPV vaccination rates using maintenance-of-certification requirements.
Pediatrics
.
2016
;
137
(
3
):
e20150675
[PubMed]
23
Vernacchio
L
,
Francis
ME
,
Epstein
DM
, et al
.
Effectiveness of an asthma quality improvement program designed for maintenance of certification.
Pediatrics
.
2014
;
134
(
1
). Available at: www.pediatrics.org/cgi/content/full/134/1/e242
[PubMed]
24
Galliher
JM
,
Manning
BK
,
Petterson
SM
, et al
.
Do professional development programs for maintenance of certification (MOC) affect quality of patient care?
J Am Board Fam Med
.
2014
;
27
(
1
):
19
25
[PubMed]
25
Simpkins
J
,
Divine
G
,
Wang
M
,
Holmboe
E
,
Pladevall
M
,
Williams
LK
.
Improving asthma care through recertification: a cluster randomized trial.
Arch Intern Med
.
2007
;
167
(
20
):
2240
2248
[PubMed]
26
LaBresh
KA
,
Ariza
AJ
,
Lazorick
S
, et al
.
Adoption of cardiovascular risk reduction guidelines: a cluster-randomized trial.
Pediatrics
.
2014
;
134
(
3
). Available at: www.pediatrics.org/cgi/content/full/134/3/e732
[PubMed]
27
Anderson
JB
,
Beekman
RH
 III
,
Kugler
JD
, et al;
National Pediatric Cardiology Quality Improvement Collaborative
.
Improvement in interstage survival in a national pediatric cardiology learning network.
Circ Cardiovasc Qual Outcomes
.
2015
;
8
(
4
):
428
436
[PubMed]
28
Duncan
PM
,
Pirretti
A
,
Earls
MF
, et al
.
Improving delivery of Bright Futures preventive services at the 9- and 24-month well child visit.
Pediatrics
.
2015
;
135
(
1
). Available at: www.pediatrics.org/cgi/content/full/135/1/e178
[PubMed]
29
Miller
MR
,
Niedner
MF
,
Huskins
WC
, et al;
National Association of Children’s Hospitals and Related Institutions Pediatric Intensive Care Unit Central Line-Associated Bloodstream Infection Quality Transformation Teams
.
Reducing PICU central line-associated bloodstream infections: 3-year results.
Pediatrics
.
2011
;
128
(
5
). Available at: www.pediatrics.org/cgi/content/full/128/5/e1077
[PubMed]
30
Billett
AL
,
Colletti
RB
,
Mandel
KE
, et al
.
Exemplar pediatric collaborative improvement networks: achieving results.
Pediatrics
.
2013
;
131
(
suppl 4
):
S196
S203
[PubMed]
31
Starmer
AJ
,
Spector
ND
,
Srivastava
R
, et al;
I-PASS Study Group
.
Changes in medical errors after implementation of a handoff program.
N Engl J Med
.
2014
;
371
(
19
):
1803
1812
[PubMed]
32
Starmer
AJ
,
O’Toole
JK
,
Rosenbluth
G
, et al;
I-PASS Study Education Executive Committee
.
Development, implementation, and dissemination of the I-PASS handoff curriculum: a multisite educational intervention to improve patient handoffs.
Acad Med
.
2014
;
89
(
6
):
876
884
[PubMed]
33
Crandall
WV
,
Margolis
PA
,
Kappelman
MD
, et al;
ImproveCareNow Collaborative
.
Improved outcomes in a quality improvement collaborative for pediatric inflammatory bowel disease.
Pediatrics
.
2012
;
129
(
4
). Available at: www.pediatrics.org/cgi/content/full/129/4/e1030
[PubMed]
34
Crandall
WV
,
Boyle
BM
,
Colletti
RB
,
Margolis
PA
,
Kappelman
MD
.
Development of process and outcome measures for improvement: lessons learned in a quality improvement collaborative for pediatric inflammatory bowel disease.
Inflamm Bowel Dis
.
2011
;
17
(
10
):
2184
2191
[PubMed]
35
Shaw
JS
,
Norlin
C
,
Gillespie
RJ
,
Weissman
M
,
McGrath
J
.
The national improvement partnership network: state-based partnerships that improve primary care quality.
Acad Pediatr
.
2013
;
13
(
6
suppl
):
S84
S94
[PubMed]
36
Sanabria
KE
,
Ruch-Ross
HS
,
Bargeron
JL
,
Contri
DA
,
Kalichman
MA
.
Transitioning youth to adult healthcare: new tools from the Illinois Transition Care Project.
J Pediatr Rehabil Med
.
2015
;
8
(
1
):
39
51
[PubMed]
37
Riley
M
,
Ahmed
S
,
Lane
J
,
Reed
BD
,
Locke
A
.
Using maintenance of certification (MOC) as a tool to improve the delivery of confidential care for adolescent patients.
J Pediatr Adolesc Gynecol
.
2017
;
30
(
1
):
76
81
[PubMed]
38
Brandt
KL
,
Booker
JM
,
McGrath
J
.
Clinical quality improvement for identification and management of overweight in pediatric primary care practices.
Clin Pediatr (Phila)
.
2013
;
52
(
7
):
620
627
[PubMed]
39
Gittelman
MA
,
Denny
S
,
Anzeljc
S
,
FitzGerald
M
,
Arnold
MW
.
A pilot quality improvement program to increase pediatrician injury anticipatory guidance.
J Trauma Acute Care Surg
.
2015
;
79
(
3
suppl
1
):
S9
S14
[PubMed]
40
John
T
,
Morton
M
,
Weissman
M
, et al
.
Feasibility of a virtual learning collaborative to implement an obesity QI project in 29 pediatric practices.
Int J Qual Health Care
.
2014
;
26
(
2
):
205
213
[PubMed]
41
Hayward
K
,
Haaland
WL
,
Hrachovec
J
, et al
.
Reliable pregnancy testing before intravenous cyclophosphamide: a quality improvement study.
Pediatrics
.
2016
;
138
(
6
):
e20160378
[PubMed]
42
Mittal
V
,
Darnell
C
,
Walsh
B
, et al
.
Inpatient bronchiolitis guideline implementation and resource utilization.
Pediatrics
.
2014
;
133
(
3
). Available at: www.pediatrics.org/cgi/content/full/133/3/e730
[PubMed]
43
Huang
JS
,
Chun
S
,
Sandhu
A
,
Terrones
L
.
Quality improvement in childhood obesity management through the maintenance of certification process.
J Pediatr
.
2013
;
163
(
5
):
1313
1316.e1
[PubMed]
44
Sheu
J
,
Chun
S
,
O’Day
E
, et al
.
Outcomes from pediatric gastroenterology maintenance of certification using web-based modules
[published online ahead of print December 13, 2016].
J Pediatr Gastroenterol Nutr
. doi:10.1097/MPG.0000000000001488
[PubMed]
45
Peterson
LE
,
Blackburn
B
,
Phillips
RL
,
Puffer
JC
.
Improving quality of care for diabetes through a maintenance of certification activity: family physicians’ use of the chronic care model.
J Contin Educ Health Prof
.
2014
;
34
(
1
):
47
55
[PubMed]
46
Peterson
LE
,
Blackburn
B
,
Puffer
JC
,
Phillips
RL
 Jr
.
Family physicians’ quality interventions and performance improvement for hypertension through maintenance of certification.
J Healthc Qual
.
2016
;
38
(
3
):
175
186
[PubMed]
47
Kolasinski
VA
,
Price
DW
.
Maintenance of certification part IV quality-improvement project for hypertension control: a preliminary retrospective analysis.
Perm J
.
2015
;
19
(
2
):
36
40
[PubMed]
48
Ford
JH
 II
,
Oliver
KA
,
Giles
M
,
Cates-Wessel
K
,
Krahn
D
,
Levin
FR
;
MOC Addiction Psychiatry
.
Maintenance of certification: how performance in practice changes improve tobacco cessation in addiction psychiatrists’ practice.
Am J Addict
.
2017
;
26
(
1
):
34
41
[PubMed]
49
Lambing
C
,
Moll
A
,
Hite
M
.
Optimizing fracture prevention in patients with osteoporosis.
J Am Board Fam Med
.
2015
;
28
(
6
):
819
821
[PubMed]
50
Lipner
RS
,
Hess
BJ
,
Phillips
RL
 Jr
.
Specialty board certification in the United States: issues and evidence.
J Contin Educ Health Prof
.
2013
;
33
(
suppl 1
):
S20
S35
[PubMed]
51
Rosenbluth
G
,
Tabas
JA
,
Baron
RB
.
What’s in it for me? Maintenance of certification as an incentive for faculty supervision of resident quality improvement projects.
Acad Med
.
2016
;
91
(
1
):
56
59
[PubMed]
52
Shaikh
U
,
Slee
C
.
Triple duty: integrating graduate medical education with maintenance of board certification to improve clinician communication at hospital discharge.
J Grad Med Educ
.
2015
;
7
(
3
):
462
465
[PubMed]
53
Cook
DA
,
Levinson
AJ
,
Garside
S
.
Method and reporting quality in health professions education research: a systematic review.
Med Educ
.
2011
;
45
(
3
):
227
238
[PubMed]
54
Jeffe
DB
,
Andriole
DA
.
Factors associated with American Board of Medical Specialties member board certification among US medical school graduates.
JAMA
.
2011
;
306
(
9
):
961
970
[PubMed]
55
Norcini
JJ
,
Boulet
JR
,
Whelan
GP
,
McKinley
DW
.
Specialty board certification among U.S. citizen and non-U.S. citizen graduates of international medical schools.
Acad Med
.
2005
;
80
(
10
suppl
):
S42
S45
[PubMed]
56
Mistry
KB
,
Chesley
F
,
LLanos
K
,
Dougherty
D
.
Advancing children’s health care and outcomes through the pediatric quality measures program.
Acad Pediatr
.
2014
;
14
(
5
suppl
):
S19
S26
[PubMed]
57
Lesser
CS
,
Lucey
CR
,
Egener
B
,
Braddock
CH
 III
,
Linas
SL
,
Levinson
W
.
A behavioral and systems view of professionalism.
JAMA
.
2010
;
304
(
24
):
2732
2737
[PubMed]
58
Van Der Vleuten
CP
.
The assessment of professional competence: developments, research and practical implications.
Adv Health Sci Educ Theory Pract
.
1996
;
1
(
1
):
41
67
[PubMed]
59
Opipari
VP
,
Daniels
SR
,
Wilmott
RW
,
Jacobs
RF
.
Association of Medical School Pediatric Department Chairs principles of lifelong learning in pediatric medicine.
JAMA Pediatr
.
2016
;
170
(
11
):
1087
1092
[PubMed]
60
Cook
DA
,
Blachman
MJ
,
West
CP
,
Wittich
CM
.
Physician attitudes about maintenance of certification: a cross-specialty national survey.
Mayo Clin Proc
.
2016
;
91
(
10
):
1336
1345
[PubMed]
61
Freed
GL
,
Dunham
KM
,
Gebremariam
A
.
Changes in hospitals’ credentialing requirements for board certification from 2005 to 2010.
J Hosp Med
.
2013
;
8
(
6
):
298
303
[PubMed]
62
Freed
GL
,
Dunham
KM
,
Clark
SJ
,
Davis
MM
;
Research Advisory Committee of the American Board of Pediatrics
.
Perspectives and preferences among the general public regarding physician selection and board certification.
J Pediatr
.
2010
;
156
(
5
):
841
845.e1
[PubMed]
63
Norcini
J
,
Anderson
B
,
Bollela
V
, et al
.
Criteria for good assessment: consensus statement and recommendations from the Ottawa 2010 Conference.
Med Teach
.
2011
;
33
(
3
):
206
214
[PubMed]
64
Mosquera
RA
,
Avritscher
EBC
,
Samuels
CL
, et al
.
Effect of an enhanced medical home on serious illness and cost of care among high-risk children with chronic illness: a randomized clinical trial.
JAMA
.
2014
;
312
(
24
):
2640
2648
[PubMed]
65
Medscape
. Physician compensation report: 2013. Available at: www.medscape.com/features/slideshow/compensation/2013/public. Accessed December 13, 2016

Competing Interests

POTENTIAL CONFLICT OF INTEREST: Dr Nichols is a full-time employee of the American Board of Pediatrics.

FINANCIAL DISCLOSURE: The author has indicated he has no financial relationships relevant to this article to disclose.