CONTEXT:

Unnecessary imaging and pathology procedures represent low-value care and can harm children and the health care system.

OBJECTIVE:

To perform a systematic review of interventions designed to reduce unnecessary pediatric imaging and pathology testing.

DATA SOURCES:

We searched Medline, Embase, Cinahl, PubMed, Cochrane Library, and gray literature.

STUDY SELECTION:

Studies we included were: reports of interventions to reduce unnecessary imaging and pathology testing in pediatric populations; from developed countries; written in the English language; and published between January 1, 1996, and April 29, 2017.

DATA EXTRACTION:

Two researchers independently extracted data and assessed study quality using a Cochrane group risk of bias tool. Level of evidence was graded using the Oxford Centre for Evidence-Based Medicine grading system.

RESULTS:

We found 64 articles including 44 before-after, 14 interrupted time series, and 1 randomized controlled trial. More effective interventions were (1) multifaceted, with 3 components (mean relative reduction = 45.0%; SD = 28.3%) as opposed to 2 components (32.0% [30.3%]); or 1 component (28.6%, [34.9%]); (2) targeted toward families and clinicians compared with clinicians only (61.9% [34.3%] vs 30.0% [32.0%], respectively); and (3) targeted toward imaging (41.8% [38.4%]) or pathology testing only (48.8% [20.9%]), compared with both simultaneously (21.6% [29.2%]).

LIMITATIONS:

The studies we included were limited to the English language.

CONCLUSIONS:

Promising interventions include audit and feedback, system-based changes, and education. Future researchers should move beyond before-after designs to rigorously evaluate interventions. A relatively novel approach will be to include both clinicians and the families they manage in such interventions.

Low-value care (LVC) is care that provides little or no benefit, may cause patients harm, or yields marginal benefits at a disproportionately high cost.1 This problem of LVC, or unnecessary care, is gaining wider recognition through professionally led initiatives such as Choosing Wisely.2 These initiatives strive to achieve clinician consensus on what constitutes LVC, with the hope that identifying LVC procedures will catalyze efforts to reduce such care. Doing so is crucial to a sustainable health care system, given that LVC is thought to constitute up to one-third of health care costs,3 at least in adult care in the United States. LVC practices commonly include unnecessary imaging and pathology procedures.

To reduce LVC, we need to know how to change clinician practices. Attempts to reduce LVC practices identified by Choosing Wisely have so far yielded only small magnitude changes.4 However, when delivered across populations, even small changes can represent clinically meaningful change and reduce costs. Although systematic reviews of the evidence on interventions to reduce LVC have been published,5,6 they are focused almost exclusively on adults and do not differentiate pediatric populations. Effective interventions to reduce LVC in children may differ from those in adults because of additional factors driving LVC in children, such as increased parental demand and decreased clinician confidence in treating children compared with adults.

In this systematic review, we aim to summarize evidence from the literature regarding interventions to reduce unnecessary imaging and pathology tests in pediatric populations (mean age ≤18 years) as 2 common, high-volume exemplars of LVC. To identify potentially sustainable solutions, we limited our review to studies that included a follow-up period of >6 months.

This systematic review was conducted according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.7 A protocol was registered with the International Prospective Register of Systematic Reviews on September 19, 2016 (registration number CRD42016047960).

A systematic search of the Medline, Embase, Cinahl, PubMed, and Cochrane Library databases was conducted by using Medical Subject Headings and keywords adapted for each database. Four broad categories were included: (1) unnecessary procedures; (2) imaging and pathology tests; (3) intervention, trials, or reviews; and (4) children and adolescents (Supplemental Information). The search was limited to literature published in developed countries between January 1, 1996, and April 29, 2017. A gray literature search was employed via a keyword structure that paralleled that of the main search (Supplemental Information). This included searches of Google Scholar, Choosing Wisely publications (United States, Canada, Australia), EVOLVE (ie, the Royal Australasian College of Physicians “do-not-do” lists) publications, and gray literature databases.

Studies were included if they contained reports on 1 or more intervention(s) to reduce unnecessary imaging or pathology tests among pediatric patients. We selected studies that were aimed at reducing unnecessary or LVC as per established guidelines (for example, “do-not-do” procedures [ie, National Institute for Health and Care Excellence (NICE) Guidelines8]); tests repeated routinely, or without clinical indication to do so (Choosing Wisely2). Studies were required to use a control group; however, to allow for a comprehensive review, we were liberal in defining this (eg, usual practice, other active intervention, nonexposed control group, or pre-intervention comparison). We expected interventions to include 1 or more of the following attributes on the basis of the adult literature9: education (eg, lectures or guidelines distribution), audit and feedback (eg, clinician or organization performance is compared with peers), system and/or process based (eg, electronic clinical decision support, computer order entry changes, or institutional guideline), incentive or penalty schemes (eg, reward or punishment for certain ordering practices), or guideline publication (eg, an external body published a guideline, with no specific adaptation for the institution studied). We considered studies to be relevant to a pediatric population if the total patient sample had a mean age ≤18 years. Literature in any language other than English was excluded during the screening process because of limits on available resources to conduct translation. Studies with a follow-up period of ≤6 months were excluded so that the interventions reported here reflect medium- to longer-term results.

Studies were screened for eligibility via a 2-stage process, first by screening title and abstract, followed by full-text review. For studies without sufficient information to assess eligibility, first authors were contacted for missing information. Data including study design, sample characteristics, intervention components, and outcomes were extracted by 2 coauthors using a standardized data extraction form within the EPPI-Reviewer software.10 The same 2 authors independently assessed risk of bias (at the outcome level) for all included papers by using the “Cochrane Effective Practice and Organization of Care (EPOC) guidelines for assessing risk of bias in studies with a separate control group,"11 and discrepancies were resolved via joint article review and discussion. Additionally, the level of evidence for each study was graded from 1 to 5 according to a modified version of the Oxford Centre for Evidence-Based Medicine levels of evidence system, in which 1 = a properly powered and conducted randomized controlled trial (RCT) or systematic review with meta-analysis and 5 = opinion of respected authorities or case reports.12 

Where not directly reported, we calculated a relative reduction (RR) in testing for each study and an absolute reduction (AR) in testing for studies in which reported data made this possible. ARs were calculated as the percentage of patients who received the test in the pre-intervention (for before-after studies) or comparator group minus the percentage of patients who received the test in the post-intervention or intervention group. RRs were calculated as the number of tests conducted in the pre-intervention or comparator group minus the number of tests conducted in the post-intervention or intervention group, divided by the number of tests conducted in the pre-intervention or comparator group. Both ARs and RRs were conducted with the unit of analysis as the type of test, with some studies containing reports of effects of the intervention on multiple tests (eg, computed tomography [CT] and ultrasound, or multiple laboratory tests). We did not quantitatively pool study results because there was a large degree of heterogeneity in study design and reporting of outcomes. As such, we present a narrative summary of ARs and RRs using means and SDs to summarize effects. We present these results stratified by interventions targeting imaging only, pathology only, and imaging and pathology, followed by a summary of the overall effects. All analyses were performed by using Stata version 14.0 (StataCorp, College Station, TX).

Database searches generated 10 554 results, of which 2423 were duplicates (Fig 1). After screening on title and abstract, 155 full-text articles were reviewed for eligibility. The gray literature search produced 3104 studies, from which only 1 nonduplicated paper was eligible for inclusion. In addition, 1 article13 was identified via discussion with colleagues at The Royal Children’s Hospital (from where this review was conducted). A total of 64 studies were included in the final review.

FIGURE 1

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flowchart of included and excluded studies.

FIGURE 1

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flowchart of included and excluded studies.

Thirty-two studies contained reports of interventions to reduce unnecessary imaging, with sample sizes ranging from 61 to 160 000 (see Table 1). Of these, 23 were from the United States and 4 were from Australia. The majority (n = 16) were conducted in the emergency department (ED), with 5 conducted in inpatient populations, 4 conducted in outpatient populations, and 4 conducted across multiple settings. Most (n = 21) were conducted in pediatric hospitals, 8 in general hospitals, and 1 across a mix of sites.14 Twenty-four studies were conducted at single sites. Nearly all studies (n = 27) targeted clinicians, and only 2 studies targeted a combination of patients and/or families and clinicians.15,16 Three studies (in a series on reducing imaging in irritable bowel syndrome17,19) targeted patients and their families only. Follow-up periods ranged from 10 months to 7 years, and most studies were before-after (n = 23) or interrupted time series designs (n = 5), with only 1 RCT.20 Regarding intervention complexity, the majority of interventions (n = 18) comprised only a single component, of which most (n = 11) were a system- and/or process-based change, though 4 studies13,21,23 contained reports of the effects of an external guideline publication on imaging rates. Of the 14 multifaceted interventions, nearly all (n = 13) were a system- and/or process-based change alongside an education component, with or without an audit and feedback component. Absolute changes in imaging ranged from a 92% reduction to a 21% increase, and relative changes ranged from a 100% reduction to an 81% increase.

TABLE 1

Included Studies Targeting Imaging (Only): Study Detail and Intervention Characteristics

Author (y) and SettingTest and ConditionDesign (No. Total Sample)Intervention DescriptionLOEIntervention CategoryFollow-up PeriodPrimary Outcomes
IASEG
Imaging only: target = clinicians (only) (n = 27) 
Akenroye and Stack24 (2016) US ED pediatric hospital single site CT, radiograph in minor head trauma and croup Interrupted time series (n = 5542) Algorithm developed for 11 EBGs and order sets. Binders; laminated copies; discussions; Web-based videos; pocket cards; individual performance management; anonymous online survey.   2 y CT rate in minor head injury: pre 1 = ∼24%; pre 2 = ∼21%; post 1 = ∼12%; post 2 = ∼9%; AR = 15%; RR = 62.5% 
Mean radiographs in croup per month: pre = 13; post = 7; RR = 46% 
Kwon and Jung25 (2017) South Korea ED general hospital single site AXR in abdominal pain Interrupted time series (n = 14 244) List of catchphrases and information comprising indications, diagnostic tests, radiation doses (eg, really need abdominal plain film? Erect and supine, or erect alone?). Monthly educational sessions and feedback on radiograph use and/or adverse events.   1 y Total imaging: pre = 90% (9647); post = 77% (2710); P < .001; AR = 13%; RR = 14.4% 
Erect and supine: pre = 84% (8959); post = 14% (501); P < .001; AR = 70%; RR = 83% 
Mittal et al26 (2014) US IP pediatric hospital single site CXR in bronchiolitis Before-after (n = 3686) Developed CPG on the basis of internal data, literature, and existing CPGs. Provider education, electronic order sets, online access, screen savers, monthly team meetings; regular feedback.   1.5 y CXR use: pre = 59.7% (742); post 1 = 45.1% (523); P < .0001; post 2 = 39% (485); P < .0001; AR = 20.7%; RR = 34.7% 
Nigrovic et al27 (2015) US ED pediatric hospital single site CT and/or MRI in head injury Interrupted time series (n = 6856) Guidelines developed from PECARN and Choosing Wisely in 2 phases: peer feedback; CPOE; discharge information, posters in ED; quarterly e-mail and in-person education; individual and/or department feedback.   2.5 y CT use: pre = 21%; post 1 = 15%; post 2 = 9%; AR = 12%; RR = 57.1% 
MRI use: pre = 0.6%; post = 0.3%; P = .09; AR = 0.3%; RR = 50% 
Ralston et al28 (2016) US IP general hospitals multisite CXR in bronchiolitis Before-after (n = 1869) Each site implemented separately. Pathways; respiratory tool; family tools; slides; monthly webinars; group feedback; online workspace (by AAP; materials and real-time data); site coaches.   1 y CXR use: pre = 12%; post = 6.7%; P = .05; AR = 5.3%; RR = 44.2% 
Range of change for all hospitals was between −100% and +174% 
Shah et al29 (2016) US ED pediatric hospital single site CT in appendicitis Before-after (n = 824) Diagnostic algorithm developed from literature review: earlier surgical consult; prioritized ultrasound over CT. Ongoing education; individual feedback; reminders at meetings.   1 y CT use: pre = 75.4%; post = 24.2%; P < .0001; AR = 51.2%; RR = 67.9% 
Ultrasound before CT: pre = 24.4%; post = 95.3%; P < .0001; AI = 70.9%; RI = 290.6% 
Goldberg et al30 (2011) US IP pediatric hospital single site CT in head injury Before-after (n = 712) Developed guideline for mild TBI. Provided to personnel; intranet site; shared with the radiology; printed in resident handbook. Tracking of guideline compliance; real-time feedback.    2 y Rate of repeated head CTs: pre = 40%; post = 26%; P < .0001. 
AR = 14%; RR = 35% 
Boutis et al14 (2013) Canada ED mixed hospitals multisite Radiograph ankle injury Interrupted time series (n = 2151) Multimodal intervention to use LRAR. Teaching session; pocket cards; posters; site champions; CPOE decision support. Later: kept decision support system and other strategies removed.    1 y Ankle radiographs: pre = 96.5% (412); post 1 = 73.5% (211); post 2 = 71.3% (143); AR = 25.2%; RR = 26.1% 
Buckmaster and Boon31 (2005) Australia ED general hospital multisite CXR in asthma Before-after (n = 663) “Unnecessary” defined and education developed for all staff involved in care. Formal presentations; display of posters in departments; ongoing education; orientation to new pediatric staff.    10 mo Total CXR conducted: pre = 51.5% (260); post = 36.5% (72); AR = 15%; RR = 29.1% 
Depinet et al32 (2016) US ED pediatric hospital single site CT in appendicitis Interrupted time series (n = 1886) Two phases: (1) passive: pathway developed on the basis of PAS and diffused through meetings, conferences, posters, Web site, and e-mail; (2) active: CPOE decision support within EMR.    2.5 y CT use (moderate risk): pre = 22.3%; post 1 = 10.2%; post 2 = 12.2%; AR = 10.1%; RR = 45.3% 
CT use (high risk): pre = 25.2%; post 1 = 15.7%; post 2 = 14.4%; AR = 10.8%; RR = 42.9% 
Ultrasound use (all groups): pre = 59.5%; post 1 = 83.7%; post 2 = 88.8%; AI = 29.3%; RI = 49.2% 
Gildenhuys et al33 (2009) Australia ED pediatric hospital single site CXR in asthma Before-after (n = 526) Asthma CPG developed by Australian government/RACP and adapted for ED. Role of project officer was: implement, mentoring, communication with ED staff; feedback; changes; educational sessions; implemented final CPG.    1 y Total CXR: pre = 48%; post = 31%; P = .0001 
AR = 17%; RR = 35.4% 
Russell et al34 (2013) US ED pediatric hospital single site CT in appendicitis Before-after (n = 166) CPG focused on early surgical consult rather than advanced imaging, and ultrasound as first-line imaging. Formal education process conducted and continuously reinforced to promote compliance.    1 y CT: pre= 90% (63); post = 48% (46); AR = 42%; RR = 46.7% 
Ultrasound: pre= 6.9% (5); post = 39.6% (38); AI = 32.7%; RI = 473.9% 
Browning et al35 (2005) United Kingdom (Scotland) ED pediatric hospital single site Skull radiograph in head injury Before-after (n = 371) ED policy modified on the basis of NICE guidelines. Skull radiographs restricted to children <1 y old with visible evidence of head injury or suspicious history for nonaccidental injury.     4 y Skull radiograph: pre = 77.3% (140); post = 29.5% (56); P < .001; AR = 47.8%; RR = 61.8% 
CT: pre = 1.7% (3); post = 0.5% (1); P = .36; AR = 1.2%; RR = 70.6% 
Bowen et al36 (2011) Canada ED/IP/OP/ICU pediatric hospital single site Diagnostic imaging Before-after (n = 9925) CPOE system with diagnostic imaging referral guidelines. Clinician order: (1) automatically sent out; or (2) if not appropriate, guideline pop-up; clinician accepts advice or continues with order.     10 mo Inappropriate orders: pre = (not reported); post = 957 (10.9%); reduction: “2% of inappropriate orders were cancelled based on decision support.” 
Chien et al20 (2017) US general medical services multisite Diagnostic imaging RCT (n = ∼160 000) Prices added to order set for imaging tests, with clinicians randomly assigned across 3 study arms: (1) median paid price per test; (2) internal (to org) and external pricing; and (3) no pricing (control). Pediatric focused clinicians analyzed separately from adult-focused clinicians, although whole patient sample was pediatric.     11 mo (Mean [SD] per 100 encounters) pediatric-focused clinicians: control = 2.5 (2.2); single price = 3.0 (2.2); P = .12; paired prices = 3.2 (5.0); P = .23 
RI = 20%; 28%; adult-focused clinicians: control = 3.2 (4.8); single price = 6.2 (6.8); P < .001; paired prices = 5.2 (7.9); P = .04. 
RI = 93.8%; 62.5% 
Luther et al37 (2016) US ED pediatric hospital single site Radiograph in Torus fractures Before-after (n = 273) SCAMP developed from literature, analysis of practice variation and consensus. All providers required to use the algorithm, although deviations were recorded and informed later process changes.     3 y (Mean [SD]) total radiographs per patient: pre = 3.3 (1.22); post = 1.3 (0.51); P < .001; RR = 60.6% 
Price et al38 (1999) US ICU pediatric hospital single site CXR in PICU Before-after (n = 3510) No longer allowed standing orders for routine daily chest radiographs. Each required a written order and clinical indication.     2 y Mean CXR per month: pre = 363; post = 228; P < .01; RR = 37.1% 
Reed et al39 (2005) UK (Scotland) ED pediatric hospital single site Skull radiograph in head injury Before-after (n = 3402) Departmental head injury policy revised to restrict ordering of skull radiographs to patients aged <1 y. Indications for admission for observation or CT were unchanged.     4 y Skull radiograph use: pre = 22.1% (340); post = 0% (0); AR = 22.1%; RR = 100% 
Rosati et al40 (2015) US ED pediatric hospital single site CT in spine injury Before-after (n = 233) Guidelines developed to incorporate NEXUS criteria. Presence of NEXUS criteria necessitates imaging; first line is plain film, then CT or MRI. If positive results, then neurosurgery consult.     1 y Total CT use: pre = 43.0% (55); post = 28.6% (30); AR = 14.4%; RR = 33.5% 
Ruffing et al41 (2012) US ED pediatric hospital single site CT in abdominal pain and/or appendicitis Before-after (n = 1008) Surgical residents to evaluate patient and discuss with surgery before ordering CT. Alternative evaluation pathways including ultrasound and observation were considered. Encouraged use of ultrasound.     1 y Abdominal CT use: pre = 32% (44); post = 15% (86); P < .001; AR = 17%; RR = 53.1% 
Stylianos42 (2002) US IP mixed hospitals multisite Follow-up imaging in liver and/or splenic injury Before-after (n = 1144) APSA trauma committee defined guidelines for ICU stay, length of hospital stay, and use of pre- and postdischarge follow-up imaging. Current study applied the guidelines prospectively.     3 y Follow-up imaging: grade I: pre = 34.4%; post = 7.3%; P < .001; AR = 27.1%; RR = 78.8% 
Grade II: pre = 46.3%; post = 8.9%; P < .001; AR = 37.4%; RR = 80.8% 
Grade III: pre = 54.1%; post = 10.9%; P < .001; AR = 43.2%; RR = 79.9% 
Grade IV: pre= 51.8%; post = 23.3%; P < .001; AR = 28.5%; RR = 55.0% 
Thompson et al43 (2013) US ED pediatric hospital single site Repeat CT and/or radiograph Before-after (“before” n = 139) Trauma image repository launched to send images ahead of the patient to receiving hospital. Examines number and reason for repeat images before and after the repository.     1 y Repeat radiograph: pre = 70.3%; repeat CT: pre = 16.7%; repeat CT and radiograph: pre = 7.2%; ** preliminary data analysis indicates: “reduction” = ∼8% 
Zamora-Flores et al44 (2014) US IP general hospital single site CXR in bronchiolitis Before-after (n = 332) Implemented CPG on the basis of AAP guidelines, adjusted for additional risk factors of young children in specific region, to reduce unnecessary costs, and provide more consistent care.     10 mo CXR: pre = 88%; post = 75%; P = .14; AR = 13.0%; RR = 14.8% 
Arlen et al21 (2015) US ED pediatric hospital multisite RBUS and VCUG in UTI Before-after (n = 350) AAP revised practice parameters, which invigorated debate regarding management of vesicoureteral reflux. Electronic medical chart review of before and after guideline revision.     1 y VCUG: pre = 72.1% (124); post = 30.9% (55); P < .001; AR = 41.2%; RR = 57.1%; RBUS: pre = 75.6% (130); post = 58.4% (104); P < .001; AR = 17.2%; RR = 22.8% 
Graf et al22 (2008) US OP pediatric hospital single site CT and/or MRI in nonacute pediatric headache Before-after (n = 725) Study examines period before and after the publication of a CPG (American Academy of Neurology). Unclear whether physicians were aware/unaware or followed/ignored the guidelines.     2 y Neuroimaging use rates: pre 1 = 43% (59); pre 2 = 41% (81); pre 3 = 47% (93); post = 45% (92); P = n.s.; AR = 2%; RR = 4.3% 
Judkins et al23 (2013) Australia OP /GP pediatric hospital single site Renal ultrasound/MCUG/DMSA in UTI Before-after (n = 786) NICE guidelines for UTI published in 2007, promoting less aggressive approach: a reduction in the use of MCUG and antibiotic prophylaxis, and decrease in DMSA scans.     2 y MCUG: pre = 63% (210); post = 31% (102); P < .0001; AR = 32%; RR = 50.8% 
Renal ultrasound: pre = 97% (324); post = 99% (322); P = .07; AI = 2%; RI = 2.1% 
NM: pre = 26% (85); post = 47% (152); P < .001; AI = 21%; RI = 80.8% 
South13 (2009) Australia mixed hospitals multisite RBUS/MCUG/IVP/NM in UTI Before-after (n = not reported) Publication of studies and reviews beginning late 1990s challenged existing UTI treatments. Medicare database queried for use of renal ultrasound, MCU, IVP, NM, and costs before and after.     7 y (Rates per 100 000 children): RBUS (0–4 y.o.): pre = ∼1900; post = ∼2000 RI = 5%; RBUS (5–14 y.o): pre = ∼1000; post = ∼1250; RI = 25% MCUG (0–4 y): pre = ∼600; post = ∼200; RR = 67%; MCUG (5–14 y.o.): pre = ∼100; post = ∼10; RR = 90% 
IVP (0–4 y.o.) pre = ∼16; post = ∼3; RR = 81%; IVP (5–14 y.o.): pre = ∼27; 
Post = ∼3; RR = 89% 
NM (0–4 y.o.) pre = ∼110; post = ∼50; RR = 55%; NM (5–14 y.o.): pre = ∼35; 
Post = ∼12; RR = 66% 
Imaging only: target = combined clinicians and patients and/or families (n = 2)  
Jerardi et al16(2013) US ED/IP pediatric hospital single site RBUS/VCUG in UTI Before-after (n = 214) Changed guideline to recommend VCUG only for patients with abnormal RBUS. Lectures, education materials, order set and EMR documentation changes, and family education materials.   1 y VCUG postnormal RBUS (inappropriate): IP: pre = 92%; post = 0%; AR = 92%; RR = 100% 
ED: pre = 100%; post = 40%; AR = 60%; RR = 60% 
Higgins et al15 (1998) US ED/GP general hospital single site CXR in asthma Before-after (n = 61) Patients’ parents attended 2 group asthma education classes. Instructed parents in guidelines; assigned to a primary care provider. Providers attended up to 5 1-h asthma update lectures.    11 mo Total CXR: pre = 119; POST = 9; RR = 92.4% 
Imaging only: target = patients (only) (n = 3)  
Kalani et al18 (2012) US assumed OP general hospital single site Imaging in IBS Nonrandomized controlled trial (n = not reported) Multidisciplinary behavioral therapy: two 90 min sessions using education, reassurance, dietary intervention, exercise instruction, and guided imagery. Reviewed 3 y pre and 3 y post.     3 y “Use of imaging”: Control: pre = (not reported); post: (not reported); no difference 
Study: pre = (not reported); post: (not reported); P < .003; “reduction” = 50% 
Kalani et al19(2013) US OP general hospital multisite (kaiser) Imaging in IBS Nonrandomized controlled trial (n = 75) Multidisciplinary behavioral therapy: two 90 min sessions using education, reassurance, dietary intervention, exercise instruction, and guided imagery. Reviewed 1 y pre and 4 y post.     4 y GI-related imaging per person: control = 4.23 (1.21); study = 1.03 (0.33); P < .05; RR = 75.7% 
Overall imaging per person: control = 11.08 (4.03); study = 2.93 (0.57); P < .05; RR = 73.6% 
Kalani et al17 (2014) US OP general hospital single site Imaging in IBS Nonrandomized controlled trial (n = 183) Multidisciplinary behavioral therapy: two 90 min sessions using education, reassurance, dietary intervention, exercise instruction, and guided imagery. Reviewed 3 y pre and 4 y post     4 y Mean imaging per person: control = 4.4 (1.51); study = 2.2 (0.85); P < .05; RR = 50% 
Author (y) and SettingTest and ConditionDesign (No. Total Sample)Intervention DescriptionLOEIntervention CategoryFollow-up PeriodPrimary Outcomes
IASEG
Imaging only: target = clinicians (only) (n = 27) 
Akenroye and Stack24 (2016) US ED pediatric hospital single site CT, radiograph in minor head trauma and croup Interrupted time series (n = 5542) Algorithm developed for 11 EBGs and order sets. Binders; laminated copies; discussions; Web-based videos; pocket cards; individual performance management; anonymous online survey.   2 y CT rate in minor head injury: pre 1 = ∼24%; pre 2 = ∼21%; post 1 = ∼12%; post 2 = ∼9%; AR = 15%; RR = 62.5% 
Mean radiographs in croup per month: pre = 13; post = 7; RR = 46% 
Kwon and Jung25 (2017) South Korea ED general hospital single site AXR in abdominal pain Interrupted time series (n = 14 244) List of catchphrases and information comprising indications, diagnostic tests, radiation doses (eg, really need abdominal plain film? Erect and supine, or erect alone?). Monthly educational sessions and feedback on radiograph use and/or adverse events.   1 y Total imaging: pre = 90% (9647); post = 77% (2710); P < .001; AR = 13%; RR = 14.4% 
Erect and supine: pre = 84% (8959); post = 14% (501); P < .001; AR = 70%; RR = 83% 
Mittal et al26 (2014) US IP pediatric hospital single site CXR in bronchiolitis Before-after (n = 3686) Developed CPG on the basis of internal data, literature, and existing CPGs. Provider education, electronic order sets, online access, screen savers, monthly team meetings; regular feedback.   1.5 y CXR use: pre = 59.7% (742); post 1 = 45.1% (523); P < .0001; post 2 = 39% (485); P < .0001; AR = 20.7%; RR = 34.7% 
Nigrovic et al27 (2015) US ED pediatric hospital single site CT and/or MRI in head injury Interrupted time series (n = 6856) Guidelines developed from PECARN and Choosing Wisely in 2 phases: peer feedback; CPOE; discharge information, posters in ED; quarterly e-mail and in-person education; individual and/or department feedback.   2.5 y CT use: pre = 21%; post 1 = 15%; post 2 = 9%; AR = 12%; RR = 57.1% 
MRI use: pre = 0.6%; post = 0.3%; P = .09; AR = 0.3%; RR = 50% 
Ralston et al28 (2016) US IP general hospitals multisite CXR in bronchiolitis Before-after (n = 1869) Each site implemented separately. Pathways; respiratory tool; family tools; slides; monthly webinars; group feedback; online workspace (by AAP; materials and real-time data); site coaches.   1 y CXR use: pre = 12%; post = 6.7%; P = .05; AR = 5.3%; RR = 44.2% 
Range of change for all hospitals was between −100% and +174% 
Shah et al29 (2016) US ED pediatric hospital single site CT in appendicitis Before-after (n = 824) Diagnostic algorithm developed from literature review: earlier surgical consult; prioritized ultrasound over CT. Ongoing education; individual feedback; reminders at meetings.   1 y CT use: pre = 75.4%; post = 24.2%; P < .0001; AR = 51.2%; RR = 67.9% 
Ultrasound before CT: pre = 24.4%; post = 95.3%; P < .0001; AI = 70.9%; RI = 290.6% 
Goldberg et al30 (2011) US IP pediatric hospital single site CT in head injury Before-after (n = 712) Developed guideline for mild TBI. Provided to personnel; intranet site; shared with the radiology; printed in resident handbook. Tracking of guideline compliance; real-time feedback.    2 y Rate of repeated head CTs: pre = 40%; post = 26%; P < .0001. 
AR = 14%; RR = 35% 
Boutis et al14 (2013) Canada ED mixed hospitals multisite Radiograph ankle injury Interrupted time series (n = 2151) Multimodal intervention to use LRAR. Teaching session; pocket cards; posters; site champions; CPOE decision support. Later: kept decision support system and other strategies removed.    1 y Ankle radiographs: pre = 96.5% (412); post 1 = 73.5% (211); post 2 = 71.3% (143); AR = 25.2%; RR = 26.1% 
Buckmaster and Boon31 (2005) Australia ED general hospital multisite CXR in asthma Before-after (n = 663) “Unnecessary” defined and education developed for all staff involved in care. Formal presentations; display of posters in departments; ongoing education; orientation to new pediatric staff.    10 mo Total CXR conducted: pre = 51.5% (260); post = 36.5% (72); AR = 15%; RR = 29.1% 
Depinet et al32 (2016) US ED pediatric hospital single site CT in appendicitis Interrupted time series (n = 1886) Two phases: (1) passive: pathway developed on the basis of PAS and diffused through meetings, conferences, posters, Web site, and e-mail; (2) active: CPOE decision support within EMR.    2.5 y CT use (moderate risk): pre = 22.3%; post 1 = 10.2%; post 2 = 12.2%; AR = 10.1%; RR = 45.3% 
CT use (high risk): pre = 25.2%; post 1 = 15.7%; post 2 = 14.4%; AR = 10.8%; RR = 42.9% 
Ultrasound use (all groups): pre = 59.5%; post 1 = 83.7%; post 2 = 88.8%; AI = 29.3%; RI = 49.2% 
Gildenhuys et al33 (2009) Australia ED pediatric hospital single site CXR in asthma Before-after (n = 526) Asthma CPG developed by Australian government/RACP and adapted for ED. Role of project officer was: implement, mentoring, communication with ED staff; feedback; changes; educational sessions; implemented final CPG.    1 y Total CXR: pre = 48%; post = 31%; P = .0001 
AR = 17%; RR = 35.4% 
Russell et al34 (2013) US ED pediatric hospital single site CT in appendicitis Before-after (n = 166) CPG focused on early surgical consult rather than advanced imaging, and ultrasound as first-line imaging. Formal education process conducted and continuously reinforced to promote compliance.    1 y CT: pre= 90% (63); post = 48% (46); AR = 42%; RR = 46.7% 
Ultrasound: pre= 6.9% (5); post = 39.6% (38); AI = 32.7%; RI = 473.9% 
Browning et al35 (2005) United Kingdom (Scotland) ED pediatric hospital single site Skull radiograph in head injury Before-after (n = 371) ED policy modified on the basis of NICE guidelines. Skull radiographs restricted to children <1 y old with visible evidence of head injury or suspicious history for nonaccidental injury.     4 y Skull radiograph: pre = 77.3% (140); post = 29.5% (56); P < .001; AR = 47.8%; RR = 61.8% 
CT: pre = 1.7% (3); post = 0.5% (1); P = .36; AR = 1.2%; RR = 70.6% 
Bowen et al36 (2011) Canada ED/IP/OP/ICU pediatric hospital single site Diagnostic imaging Before-after (n = 9925) CPOE system with diagnostic imaging referral guidelines. Clinician order: (1) automatically sent out; or (2) if not appropriate, guideline pop-up; clinician accepts advice or continues with order.     10 mo Inappropriate orders: pre = (not reported); post = 957 (10.9%); reduction: “2% of inappropriate orders were cancelled based on decision support.” 
Chien et al20 (2017) US general medical services multisite Diagnostic imaging RCT (n = ∼160 000) Prices added to order set for imaging tests, with clinicians randomly assigned across 3 study arms: (1) median paid price per test; (2) internal (to org) and external pricing; and (3) no pricing (control). Pediatric focused clinicians analyzed separately from adult-focused clinicians, although whole patient sample was pediatric.     11 mo (Mean [SD] per 100 encounters) pediatric-focused clinicians: control = 2.5 (2.2); single price = 3.0 (2.2); P = .12; paired prices = 3.2 (5.0); P = .23 
RI = 20%; 28%; adult-focused clinicians: control = 3.2 (4.8); single price = 6.2 (6.8); P < .001; paired prices = 5.2 (7.9); P = .04. 
RI = 93.8%; 62.5% 
Luther et al37 (2016) US ED pediatric hospital single site Radiograph in Torus fractures Before-after (n = 273) SCAMP developed from literature, analysis of practice variation and consensus. All providers required to use the algorithm, although deviations were recorded and informed later process changes.     3 y (Mean [SD]) total radiographs per patient: pre = 3.3 (1.22); post = 1.3 (0.51); P < .001; RR = 60.6% 
Price et al38 (1999) US ICU pediatric hospital single site CXR in PICU Before-after (n = 3510) No longer allowed standing orders for routine daily chest radiographs. Each required a written order and clinical indication.     2 y Mean CXR per month: pre = 363; post = 228; P < .01; RR = 37.1% 
Reed et al39 (2005) UK (Scotland) ED pediatric hospital single site Skull radiograph in head injury Before-after (n = 3402) Departmental head injury policy revised to restrict ordering of skull radiographs to patients aged <1 y. Indications for admission for observation or CT were unchanged.     4 y Skull radiograph use: pre = 22.1% (340); post = 0% (0); AR = 22.1%; RR = 100% 
Rosati et al40 (2015) US ED pediatric hospital single site CT in spine injury Before-after (n = 233) Guidelines developed to incorporate NEXUS criteria. Presence of NEXUS criteria necessitates imaging; first line is plain film, then CT or MRI. If positive results, then neurosurgery consult.     1 y Total CT use: pre = 43.0% (55); post = 28.6% (30); AR = 14.4%; RR = 33.5% 
Ruffing et al41 (2012) US ED pediatric hospital single site CT in abdominal pain and/or appendicitis Before-after (n = 1008) Surgical residents to evaluate patient and discuss with surgery before ordering CT. Alternative evaluation pathways including ultrasound and observation were considered. Encouraged use of ultrasound.     1 y Abdominal CT use: pre = 32% (44); post = 15% (86); P < .001; AR = 17%; RR = 53.1% 
Stylianos42 (2002) US IP mixed hospitals multisite Follow-up imaging in liver and/or splenic injury Before-after (n = 1144) APSA trauma committee defined guidelines for ICU stay, length of hospital stay, and use of pre- and postdischarge follow-up imaging. Current study applied the guidelines prospectively.     3 y Follow-up imaging: grade I: pre = 34.4%; post = 7.3%; P < .001; AR = 27.1%; RR = 78.8% 
Grade II: pre = 46.3%; post = 8.9%; P < .001; AR = 37.4%; RR = 80.8% 
Grade III: pre = 54.1%; post = 10.9%; P < .001; AR = 43.2%; RR = 79.9% 
Grade IV: pre= 51.8%; post = 23.3%; P < .001; AR = 28.5%; RR = 55.0% 
Thompson et al43 (2013) US ED pediatric hospital single site Repeat CT and/or radiograph Before-after (“before” n = 139) Trauma image repository launched to send images ahead of the patient to receiving hospital. Examines number and reason for repeat images before and after the repository.     1 y Repeat radiograph: pre = 70.3%; repeat CT: pre = 16.7%; repeat CT and radiograph: pre = 7.2%; ** preliminary data analysis indicates: “reduction” = ∼8% 
Zamora-Flores et al44 (2014) US IP general hospital single site CXR in bronchiolitis Before-after (n = 332) Implemented CPG on the basis of AAP guidelines, adjusted for additional risk factors of young children in specific region, to reduce unnecessary costs, and provide more consistent care.     10 mo CXR: pre = 88%; post = 75%; P = .14; AR = 13.0%; RR = 14.8% 
Arlen et al21 (2015) US ED pediatric hospital multisite RBUS and VCUG in UTI Before-after (n = 350) AAP revised practice parameters, which invigorated debate regarding management of vesicoureteral reflux. Electronic medical chart review of before and after guideline revision.     1 y VCUG: pre = 72.1% (124); post = 30.9% (55); P < .001; AR = 41.2%; RR = 57.1%; RBUS: pre = 75.6% (130); post = 58.4% (104); P < .001; AR = 17.2%; RR = 22.8% 
Graf et al22 (2008) US OP pediatric hospital single site CT and/or MRI in nonacute pediatric headache Before-after (n = 725) Study examines period before and after the publication of a CPG (American Academy of Neurology). Unclear whether physicians were aware/unaware or followed/ignored the guidelines.     2 y Neuroimaging use rates: pre 1 = 43% (59); pre 2 = 41% (81); pre 3 = 47% (93); post = 45% (92); P = n.s.; AR = 2%; RR = 4.3% 
Judkins et al23 (2013) Australia OP /GP pediatric hospital single site Renal ultrasound/MCUG/DMSA in UTI Before-after (n = 786) NICE guidelines for UTI published in 2007, promoting less aggressive approach: a reduction in the use of MCUG and antibiotic prophylaxis, and decrease in DMSA scans.     2 y MCUG: pre = 63% (210); post = 31% (102); P < .0001; AR = 32%; RR = 50.8% 
Renal ultrasound: pre = 97% (324); post = 99% (322); P = .07; AI = 2%; RI = 2.1% 
NM: pre = 26% (85); post = 47% (152); P < .001; AI = 21%; RI = 80.8% 
South13 (2009) Australia mixed hospitals multisite RBUS/MCUG/IVP/NM in UTI Before-after (n = not reported) Publication of studies and reviews beginning late 1990s challenged existing UTI treatments. Medicare database queried for use of renal ultrasound, MCU, IVP, NM, and costs before and after.     7 y (Rates per 100 000 children): RBUS (0–4 y.o.): pre = ∼1900; post = ∼2000 RI = 5%; RBUS (5–14 y.o): pre = ∼1000; post = ∼1250; RI = 25% MCUG (0–4 y): pre = ∼600; post = ∼200; RR = 67%; MCUG (5–14 y.o.): pre = ∼100; post = ∼10; RR = 90% 
IVP (0–4 y.o.) pre = ∼16; post = ∼3; RR = 81%; IVP (5–14 y.o.): pre = ∼27; 
Post = ∼3; RR = 89% 
NM (0–4 y.o.) pre = ∼110; post = ∼50; RR = 55%; NM (5–14 y.o.): pre = ∼35; 
Post = ∼12; RR = 66% 
Imaging only: target = combined clinicians and patients and/or families (n = 2)  
Jerardi et al16(2013) US ED/IP pediatric hospital single site RBUS/VCUG in UTI Before-after (n = 214) Changed guideline to recommend VCUG only for patients with abnormal RBUS. Lectures, education materials, order set and EMR documentation changes, and family education materials.   1 y VCUG postnormal RBUS (inappropriate): IP: pre = 92%; post = 0%; AR = 92%; RR = 100% 
ED: pre = 100%; post = 40%; AR = 60%; RR = 60% 
Higgins et al15 (1998) US ED/GP general hospital single site CXR in asthma Before-after (n = 61) Patients’ parents attended 2 group asthma education classes. Instructed parents in guidelines; assigned to a primary care provider. Providers attended up to 5 1-h asthma update lectures.    11 mo Total CXR: pre = 119; POST = 9; RR = 92.4% 
Imaging only: target = patients (only) (n = 3)  
Kalani et al18 (2012) US assumed OP general hospital single site Imaging in IBS Nonrandomized controlled trial (n = not reported) Multidisciplinary behavioral therapy: two 90 min sessions using education, reassurance, dietary intervention, exercise instruction, and guided imagery. Reviewed 3 y pre and 3 y post.     3 y “Use of imaging”: Control: pre = (not reported); post: (not reported); no difference 
Study: pre = (not reported); post: (not reported); P < .003; “reduction” = 50% 
Kalani et al19(2013) US OP general hospital multisite (kaiser) Imaging in IBS Nonrandomized controlled trial (n = 75) Multidisciplinary behavioral therapy: two 90 min sessions using education, reassurance, dietary intervention, exercise instruction, and guided imagery. Reviewed 1 y pre and 4 y post.     4 y GI-related imaging per person: control = 4.23 (1.21); study = 1.03 (0.33); P < .05; RR = 75.7% 
Overall imaging per person: control = 11.08 (4.03); study = 2.93 (0.57); P < .05; RR = 73.6% 
Kalani et al17 (2014) US OP general hospital single site Imaging in IBS Nonrandomized controlled trial (n = 183) Multidisciplinary behavioral therapy: two 90 min sessions using education, reassurance, dietary intervention, exercise instruction, and guided imagery. Reviewed 3 y pre and 4 y post     4 y Mean imaging per person: control = 4.4 (1.51); study = 2.2 (0.85); P < .05; RR = 50% 

A, audit and feedback; AAP, American Academy of Pediatrics; AI, absolute increase in testing; APSA, American Pediatric Surgical Association; AXR, abdominal radiograph; CPG, clinical practice guideline; CPOE, computerized physician order entry; CXR, chest radiograph; DMSA, dimercaptosuccinic acid; E, education; EBG, evidence-based guideline; G, guideline publication; GI, gastrointestinal; GP, general practice setting; I, incentive or penalty; IBS, irritable bowel syndrome; IP, inpatient setting; IVP, intravenous pyelogram; LOE, level of evidence; LRAR, low risk ankle rule; MCUG, micturating cystourethrogram; NEXUS, National Emergency X-Radiography Utilization Study; NM, nuclear medicine; n.s, not significant; OP, outpatient setting; PAS, Pediatric Appendicitis Score; PECARN, Pediatric Emergency Care Applied Research Network; RACP, Royal Australian College of Physicians; RI, relative increase in testing; S, system- and/or process-based; SCAMP, standardized clinical assessment and management plans; TBI, traumatic brain injury.

Ten studies contained reports on interventions to reduce unnecessary pathology testing, with sample sizes ranging from 61 to 3523 (see Table 2). Of these, 9 were from the United States and 1 from the United Kingdom. Most were in ICU patients (n = 8), and the majority (n = 7) were conducted in pediatric hospitals. All studies were single site. All except 1 study45 targeted clinicians only, and none targeted patients only. Follow-up periods ranged from 8 months to 2.5 years, and nearly all (n = 8) were before-after designs, with 2 interrupted time series.46,47 Regarding intervention complexity, all interventions included a system- and/or process-based change component. In addition to this component, 2 studies included education,45,48 1 study included audit and feedback,49 and 4 studies included both education and audit and feedback components alongside a system- and/or process-based change.46,47,50,51 Three studies contained examinations of only a system- and/or process-based change.52,54 

TABLE 2

Included Studies Targeting Pathology (Only): Study Detail and Intervention Characteristics

Author (y) and SettingTest and ConditionDesign (No. Total Sample)Intervention DescriptionLOEIntervention CategoryFollow-up PeriodPrimary Outcomes
IASEG
Pathology only: target = clinicians (only) (n = 9) 
Algaze et al50 (2016) US ICU pediatric hospital single site CV ICU laboratories (CBC, blood chemistry, coagulation panels; blood gases) Before-after (n = 1278) Phase 1: paper checklist for laboratory testing. Phase 2: CPOE decision support restricted laboratory orders to 24-h period. Education, conferences, newsletters, screensavers, and audit and feedback.   1.67 y Mean tests per ICU patient day: CBC: pre = 1.65 (0.72); post 1 = 1.14 (0.69); post 2 = 0.93 (0.53); P < .001; RR = 43.6%; chemistry: pre = 1.35 (0.77); post 1 = 0.84 (0.70); post 2 = 0.46 (0.46); P < .001; RR = 65.9%; coagulation panels: pre = 0.72 (0.59); post 1 = 0.58 (0.62); post 2 = 0.35 (0.41); P < .001; RR = 51.4%; blood gases: pre = 7.09 (2.90); post 1 = 4.31 (2.40); post 2 = 4.18 (1.87); P < .001; RR = 41.0%. 
Johnson et al51 (2016) US IP pediatric hospital single-site Repeat IP laboratories (BMP, CBC) Before-after (n = 3523) “Unnecessary” test as performed within 1 d of previous normal test. Rapid PDSA cycles. Team communication, handover tools, laboratory plans, laboratory charges posted in work areas, and audit and feedback.   1 y Unnecessary BMPs and CBCs: pre = 13.5%; post 2 = 4.5%; AR = 9%; RR = 66.7% 
Repeated within 24 h: pre = 20.9%; post 1 = 13.2%; post 2 = 8.5%; AR = 12.4%; RR = 59.3% 
Pageler et al46 (2013) US ICU pediatric hospital single site Repeat IP laboratories (CBC, blood chemistry, coagulation panels) Interrupted time series (n = 1839) Provider education; audits and feedback; CPOE constraints and alerts for specific laboratory tests to be evaluated daily. Conferences, newsletters, and screensavers. No specific training provided.   1 y Mean monthly rates per patient day: CBC: pre = 1.5 (0.1); post = 1.0 (0.1); P < .001; RR = 33.3% 
Chemistry: pre = 10.6(0.9); post = 6.9(0.6); P < .001; RR = 34.9% 
Coagulation: pre = 3.3 (0.4); post = 1.7 (0.2); P < .001; RR = 48.5% 
Sinitsky et al47 (2016) United Kingdom (England) ICU pediatric hospital single site Repeat IP laboratories (liver function, CBC, coagulation panels, CRP) Interrupted time series (n = not reported) PDSA cycles to develop blood test request form. Table of common tests aided discussion and documented agreed tests. Monthly feedback via e-mail and meetings; stickers at bedside computer.   8 mo Mean tests per bed day: pre = 1.8; post = 1.0; RR = 44.4%; “similar pattern of reduction for CBC, coagulation screens and CRP.” 
Tarrago et al49 (2012) US ICU pediatric hospital single site PICU resource use (all laboratory orders) Before-after (n = 660) Checklist with items related to procedures, medication use, ancillary tests, communication, and guidelines. Paper (then electronic) completed verbally at daily rounds. Feedback via e-mail.    1.5 y Mean orders per patient d: pre = 11.6 (14.2); post 1 = 5.6 (4.2); post 2 = 5.7 (3.3); post 3 = 5.8 (3.9); P < .001; RR = 50% 
Delgado-Corcoran et al48 (2014) US ICU pediatric hospital single site CV ICU laboratories (blood gas, electrolytes, hemoglobin coagulation profile) Before-after (n = 665) PDSA cycles: education; order set and verification; technicians instructed on amount of blood to draw; elimination of routine orders; daily discussion of blood testing needs during rounds.    1 y Blood tests per patient: pre = 38; post = 24; P < .0001; RR = 37%; total blood gases: pre = 7; post = 4; P < .0001; RR = 42.9% 
Single electrolyte: pre = 17; post = 8; P < .0001; RR = 52.9% 
Any hemoglobin: pre = 4; post = 3; P < .0001; RR = 25%: coagulation profile: pre = 3; post = 3; P = .007; RR = 0% 
Cernich et al52 (2014) US OP single site Hypothyroidism in type 1 diabetes (free T4 & thyroid) Before-after (n = not reported) Electronic algorithm developed to allow free T4 and thyroid antibody orders only in cases of abnormal TSH. PDSA cycles; EMR pop-up clarifying patient population and algorithm steps.     1.67 y Free T4 levels: pre = 237; post = (n not reported); RR = 56% 
Thyroid antibody orders: pre = 225; post = (n not reported); RR = 55%. 
Price et al53 (1999) US ICU pediatric hospital single site CV ICU laboratories (arterial blood gas and all laboratories) Before-after (n = 104) Developed 3 pathways with postoperative. Orders; use of diagnostic tests; discharge instructions for parents. Any deviation from pathway was recorded to change and improve processes.     1 y Mean (SD) arterial blood gas tests: ASD: pre = 12.7 (4.1); post = 5.6 (2.7); P < .05; RR = 55.9% 
CoA: pre = 11.2 (4.1); post = 1.7 (1.2); P < .05; RR = 84.8% 
PDA: pre = 2.0 (2.5); post = 0 (0) 
P < 0.05; RR = 100% 
Total laboratory tests: ASD: pre = 102.5 (19.8); post = 58.1 (11.4); P < .05; RR = 43.3% 
CoA: pre = 81.4 (14.2); post = 18.9 (8.6); P = n.s.; RR = 76.8% 
PDA: pre = 43.1 (15.3); post = 11.7 (5.5); P < .05; RR = 72.9% 
Shin et al54 (2013) US ICU single site CV ICU laboratories (CBC, blood chemistry, coagulation panels) Before-after (n = not reported) Intervention implemented to assess daily need for laboratory testing, followed by CPOE system to enforce daily needs assessment.     1.5 y CBC: post 1 reduction = 11%; post 2 reduction = 27%; P < .0001. 
Chemistry: post 1 reduction = 15%; post 2 reduction = 41%; P < .0001; coagulation: post 1 reduction = 20%; post 2 reduction = 33%; P = .0012. 
Pathology only: target = combined clinicians and patients/families (n = 1)  
DeSomma et al45 (2002) US ICU general hospital single site CV ICU laboratories (blood gas, CBC, prothrombin time) Before-after (n = 61) Introduction of intensive care team and surgical techniques. Followed by a postoperative clinical pathway for diagnostic testing, assessments, procedures, and medications. Patient and/or family education on what to expect and were informed of variations from the clinical pathway.    2.5 y Mean (SD) arterial blood gas: pre = 16.9 (7.8); post 1 = 8.5 (8.9); post 2 = 3.2 (1.3); RR = 13.7% 
Pre not reported for below: CBC: post 1 = 2.7 (1.3); post 2 = 1.5 (1.0); RR = 44.4% 
Metabolic profile: post 1 = 2.6 (1.9); post 2 = 0.8 (0.8); RR = 69.2% 
Prothrombin time: post 1 = 2.1 (1.2); post 2 = 1.1 (0.9); RR = 47.6% 
“Complete elimination of routine tests - liver profiles and cardiac enzymes.” 
Overall: post 1 RR = 50%; post 2 RR = additional 40% 
Pathology only: target = patients (only) (n = 0) 
Author (y) and SettingTest and ConditionDesign (No. Total Sample)Intervention DescriptionLOEIntervention CategoryFollow-up PeriodPrimary Outcomes
IASEG
Pathology only: target = clinicians (only) (n = 9) 
Algaze et al50 (2016) US ICU pediatric hospital single site CV ICU laboratories (CBC, blood chemistry, coagulation panels; blood gases) Before-after (n = 1278) Phase 1: paper checklist for laboratory testing. Phase 2: CPOE decision support restricted laboratory orders to 24-h period. Education, conferences, newsletters, screensavers, and audit and feedback.   1.67 y Mean tests per ICU patient day: CBC: pre = 1.65 (0.72); post 1 = 1.14 (0.69); post 2 = 0.93 (0.53); P < .001; RR = 43.6%; chemistry: pre = 1.35 (0.77); post 1 = 0.84 (0.70); post 2 = 0.46 (0.46); P < .001; RR = 65.9%; coagulation panels: pre = 0.72 (0.59); post 1 = 0.58 (0.62); post 2 = 0.35 (0.41); P < .001; RR = 51.4%; blood gases: pre = 7.09 (2.90); post 1 = 4.31 (2.40); post 2 = 4.18 (1.87); P < .001; RR = 41.0%. 
Johnson et al51 (2016) US IP pediatric hospital single-site Repeat IP laboratories (BMP, CBC) Before-after (n = 3523) “Unnecessary” test as performed within 1 d of previous normal test. Rapid PDSA cycles. Team communication, handover tools, laboratory plans, laboratory charges posted in work areas, and audit and feedback.   1 y Unnecessary BMPs and CBCs: pre = 13.5%; post 2 = 4.5%; AR = 9%; RR = 66.7% 
Repeated within 24 h: pre = 20.9%; post 1 = 13.2%; post 2 = 8.5%; AR = 12.4%; RR = 59.3% 
Pageler et al46 (2013) US ICU pediatric hospital single site Repeat IP laboratories (CBC, blood chemistry, coagulation panels) Interrupted time series (n = 1839) Provider education; audits and feedback; CPOE constraints and alerts for specific laboratory tests to be evaluated daily. Conferences, newsletters, and screensavers. No specific training provided.   1 y Mean monthly rates per patient day: CBC: pre = 1.5 (0.1); post = 1.0 (0.1); P < .001; RR = 33.3% 
Chemistry: pre = 10.6(0.9); post = 6.9(0.6); P < .001; RR = 34.9% 
Coagulation: pre = 3.3 (0.4); post = 1.7 (0.2); P < .001; RR = 48.5% 
Sinitsky et al47 (2016) United Kingdom (England) ICU pediatric hospital single site Repeat IP laboratories (liver function, CBC, coagulation panels, CRP) Interrupted time series (n = not reported) PDSA cycles to develop blood test request form. Table of common tests aided discussion and documented agreed tests. Monthly feedback via e-mail and meetings; stickers at bedside computer.   8 mo Mean tests per bed day: pre = 1.8; post = 1.0; RR = 44.4%; “similar pattern of reduction for CBC, coagulation screens and CRP.” 
Tarrago et al49 (2012) US ICU pediatric hospital single site PICU resource use (all laboratory orders) Before-after (n = 660) Checklist with items related to procedures, medication use, ancillary tests, communication, and guidelines. Paper (then electronic) completed verbally at daily rounds. Feedback via e-mail.    1.5 y Mean orders per patient d: pre = 11.6 (14.2); post 1 = 5.6 (4.2); post 2 = 5.7 (3.3); post 3 = 5.8 (3.9); P < .001; RR = 50% 
Delgado-Corcoran et al48 (2014) US ICU pediatric hospital single site CV ICU laboratories (blood gas, electrolytes, hemoglobin coagulation profile) Before-after (n = 665) PDSA cycles: education; order set and verification; technicians instructed on amount of blood to draw; elimination of routine orders; daily discussion of blood testing needs during rounds.    1 y Blood tests per patient: pre = 38; post = 24; P < .0001; RR = 37%; total blood gases: pre = 7; post = 4; P < .0001; RR = 42.9% 
Single electrolyte: pre = 17; post = 8; P < .0001; RR = 52.9% 
Any hemoglobin: pre = 4; post = 3; P < .0001; RR = 25%: coagulation profile: pre = 3; post = 3; P = .007; RR = 0% 
Cernich et al52 (2014) US OP single site Hypothyroidism in type 1 diabetes (free T4 & thyroid) Before-after (n = not reported) Electronic algorithm developed to allow free T4 and thyroid antibody orders only in cases of abnormal TSH. PDSA cycles; EMR pop-up clarifying patient population and algorithm steps.     1.67 y Free T4 levels: pre = 237; post = (n not reported); RR = 56% 
Thyroid antibody orders: pre = 225; post = (n not reported); RR = 55%. 
Price et al53 (1999) US ICU pediatric hospital single site CV ICU laboratories (arterial blood gas and all laboratories) Before-after (n = 104) Developed 3 pathways with postoperative. Orders; use of diagnostic tests; discharge instructions for parents. Any deviation from pathway was recorded to change and improve processes.     1 y Mean (SD) arterial blood gas tests: ASD: pre = 12.7 (4.1); post = 5.6 (2.7); P < .05; RR = 55.9% 
CoA: pre = 11.2 (4.1); post = 1.7 (1.2); P < .05; RR = 84.8% 
PDA: pre = 2.0 (2.5); post = 0 (0) 
P < 0.05; RR = 100% 
Total laboratory tests: ASD: pre = 102.5 (19.8); post = 58.1 (11.4); P < .05; RR = 43.3% 
CoA: pre = 81.4 (14.2); post = 18.9 (8.6); P = n.s.; RR = 76.8% 
PDA: pre = 43.1 (15.3); post = 11.7 (5.5); P < .05; RR = 72.9% 
Shin et al54 (2013) US ICU single site CV ICU laboratories (CBC, blood chemistry, coagulation panels) Before-after (n = not reported) Intervention implemented to assess daily need for laboratory testing, followed by CPOE system to enforce daily needs assessment.     1.5 y CBC: post 1 reduction = 11%; post 2 reduction = 27%; P < .0001. 
Chemistry: post 1 reduction = 15%; post 2 reduction = 41%; P < .0001; coagulation: post 1 reduction = 20%; post 2 reduction = 33%; P = .0012. 
Pathology only: target = combined clinicians and patients/families (n = 1)  
DeSomma et al45 (2002) US ICU general hospital single site CV ICU laboratories (blood gas, CBC, prothrombin time) Before-after (n = 61) Introduction of intensive care team and surgical techniques. Followed by a postoperative clinical pathway for diagnostic testing, assessments, procedures, and medications. Patient and/or family education on what to expect and were informed of variations from the clinical pathway.    2.5 y Mean (SD) arterial blood gas: pre = 16.9 (7.8); post 1 = 8.5 (8.9); post 2 = 3.2 (1.3); RR = 13.7% 
Pre not reported for below: CBC: post 1 = 2.7 (1.3); post 2 = 1.5 (1.0); RR = 44.4% 
Metabolic profile: post 1 = 2.6 (1.9); post 2 = 0.8 (0.8); RR = 69.2% 
Prothrombin time: post 1 = 2.1 (1.2); post 2 = 1.1 (0.9); RR = 47.6% 
“Complete elimination of routine tests - liver profiles and cardiac enzymes.” 
Overall: post 1 RR = 50%; post 2 RR = additional 40% 
Pathology only: target = patients (only) (n = 0) 

A, audit and feedback; ASD, arterial septal defect; BMP, basic metabolic panel; CBC, complete blood count; CoA, coarctation of the aorta; CPOE, computerized physician order entry; CRP, C-reactive protein; CV, cardiovascular; E, education; G, guideline publication; GP, general practice setting; I, incentive or penalty; IP, inpatient setting; LOE, level of evidence; n.s, not significant; OP, outpatient setting; PDA, patent ductus arteriosus; PDSA, plan-do-study-act cycle; S, system/process-based; TSH, thyroid stimulating hormone; T4, thyroxine.

All interventions that exclusively targeted pathology revealed some reduction in testing or no change, and no studies contained reports of an increase in testing. One study51 contained a report of ARs in unnecessary testing of 12% for repeat testing and 9% for basic metabolic panels and complete blood counts. RRs across all pathology studies ranged from a 100% reduction to no change (0% reduction). Studies revealed a greater reduction in testing when baseline levels were higher.

Twenty-two studies contained reports on interventions to reduce both unnecessary pathology testing and imaging, with sample sizes ranging from 51 to 120 539 (Table 3). All except 1 study were from the United States (n = 21). The majority of studies were conducted in inpatient settings (n = 10) or EDs (n = 7), or both inpatient settings and EDs55,57 (n = 3). Again, studies were mostly conducted in pediatric hospitals (n = 19) and at a single site (n = 17). All except 1 study58 were targeted at clinicians alone, and none was targeted at patients alone. Follow-up periods ranged from 9 months to 5.5 years, and most studies were before-after (n = 13) or interrupted time series (n = 7). Regarding intervention complexity, approximately half (n = 13) comprised a single component, usually a system- and/or process-based change (n = 10). Three studies55,59,60 contained reports on effects after an external guideline publication. Multifaceted interventions (n = 9) largely comprised a system- and/or process-based component alongside education (n = 6). For studies in which imaging and pathology were targeted simultaneously, ARs ranged from a 46% reduction to a 22% increase, and RRs ranged from a 100% reduction to a 92% increase.

TABLE 3

Included Studies Targeting Imaging and Pathology: Study Detail and Intervention Characteristics

Author (y) and SettingTest and ConditionDesign (No. Total Sample)Intervention DescriptionLOEIntervention CategoryFollow-up PeriodPrimary Outcomes
IASEG
Imaging & pathology: target = clinicians (only) (n = 21) 
Akenroye et al (2014)61 US ED pediatric hospital single site CXR and RSV testing in bronchiolitis Interrupted time series (n = 2929) AAP guideline adapted and implemented: weekly conferences; e-mail; online copies; monthly performance monitoring; feedback and peer comparison; pocket pamphlet; posters with costs.   1.5 y CXR: pre = 39%; post = 16%; P < .05; AR = 23%; RR = 59.0% 
RSV: pre = 33%; post = 22%; P < .05; AR = 11%; RR = 33.3% 
Shomaker et al62 (2011) US IP pediatric hospital single site CT; ultrasound; plain film; blood cultures; and pleural fluid cultures in empyema Before-after (n = 51) PDSA cycles prioritized areas for improvement; algorithm placed in ED, wards, PICU, intranet; CPOE decision-support and order set; monthly outcomes e-mail; ongoing changes to algorithm.   9 mo CT: pre = 41% (16); post = 0% (0); P = .01; AR = 41%; RR = 100%. Plain films: “No increase” ultrasound: pre = 38%; post = 50%; P = .35; AI = 12%; RI = 31.6%. Blood cultures: pre = 95% (37); post = 100% (12); P = .99; AI = 5%; RI = 5.3% 
Pleural fluid: pre = 82% (32); post = 83% (10); P = .99; AI = 1%; RI = 1.2% 
Perlstein et al63 (2000) US IP pediatric hospital single site CXR; blood gas and RSV testing in bronchiolitis Before-after (n = 1979) Guideline says “don't do” routine laboratory and radiologic studies, respiratory care therapies. Intervention reinforced with daily ward rounds and problems with implementation resolved.    3 y RSV: pre = 89%; post = 43%; P < .001; AR = 46.0%; RR = 51.7%; CXR: pre = 70%; post = 60%; P < .001; AR = 10%; RR = 14.3% 
Blood gases: pre = 6%; post = 9%; p < .001; AI = 3%; RI = 50% 
Fallon et al64 (2016) US ED pediatric hospital single site CT scans and laboratory costs in abdominal trauma Before-after (n = 321) Algorithm developed from literature. Included: posters, online info, education, CPOE decision support. Later, minor revisions made to protocol (eg, age cut-off for unreliable examinations; laboratory orders color coded). Education repeated; new materials.    2.75 y Rate of negative CTs: pre = 77%; post 1 = 68%; post 2 = 51%; P = .003; AR = 26.0%; RR = 33.8% 
Rate of clinically insignificant scans: pre = 86%; post 1 = 78%; post 2 = 68%; P = .03; AR = 18%; RR = 20.9% 
Median laboratory costs: pre = $166; post 1 = $352; post 2 = $139; P = .005; RR = 16.3% 
Guse et al65 (2014) US ED pediatric hospital single site CBC; electrolytes; CXR and CT in syncope Before-after (n = 349) Guideline developed from literature and collaboration. Included: weekly conference; e-mail reminders; displays in ED; online; electronic order set; discharge information; pocket cards.    1 y CBC: pre = 36% (61); post = 16% (29); AR = 20%; RR = 55.6% 
Electrolytes: pre = 29% (50); post = 12% (21); AR = 17%; RR = 58.6% 
CXR: pre = 12% (21); post = 8% (14); AR = 4%; RR = 33.3% 
Head CT: pre = 2% (4); post = 2% (4); no difference 
Mohan et al66 (2016) US ED pediatric hospital single site Multiple laboratories and CXR in chest pain Interrupted time series (n = 1687) Pathway developed from literature and consultation. Electronic order set; intensive staff education; division meeting presentation. Visiting staff received orientation and education.    1 y CXR: pre = 46.1%; post = 35.6%; P < .01; AR = 10.5%; RR = 22.8% 
“No difference in”: troponin, creatine kinase, B-type natriuretic peptide, BMP, CBC, urine drug screen, CRP, sedimentation rate, D-dimer, echocardiogram 
Morris et al67 (2015) US ICU pediatric hospital single site Laboratories and radiology after staffing/structure change Interrupted time series (n = 349) Created “small baby unit” separate from NICU; smaller, darker, quieter. Guidelines and checklists based on child age in hard copy and intranet. Lead physician and nurse as coordinators and clinicians for continuity of care. Training and biweekly talks.    4 y Mean laboratories (blood gas, electrolyte, CBC): pre = 224; post = 82; P < .001; RR = 63.4% 
Mean radiographs (abdo & chest): pre = 45; post = 22; P < .001; RR = 51.1% 
Raucci et al68 (2014) Europe (Italy) ED pediatric hospital single site Echocardiography; CT; MRI and blood workup in syncope Before-after (n = 1073) CPG developed. Educational seminars; electronic guidelines and algorithms; training simulation; daily ED attendance by study team for supervision of implementation, and resolution of problems.    2 y Echo: pre = 5.1%; post = 3%; P = .07; AR = 2.1%; RR = 41.2% 
CT/MRI: pre = 5.5%; post = 1.3%; P < .001; AR = 4.2%; RR = 76.4%. Bloods: pre = 32.1%; post = 13.9%; P < .001; AR = 18.2%; RR = 56.7% 
Fallon et al69 (2014) US ED pediatric hospital single site CT; radiograph; laboratories after staffing/structure change Interrupted time series (n = 472) Implemented protocol where ED physician (not surgery team) primarily manages “level 2” trauma activations with trauma consultation for surgical evaluation or admission.     9 mo Mean CT: pre = 1.4; post = 1.0; P < .001; RR = 28.6%; mean radiographs: pre = 2.4; post = 1.7; P < .001; RR = 29.2%; mean laboratories: pre = 5.1; post = 3.3; P < .001; RR = 35.3% 
Gutierrez et al70 (2013) US IP pediatric hospital multisite Imaging, laboratory and total hospital costs in splenic injury Cross-sectional (n = 1154) APSA Trauma Committee CPG proposed in 1998. Analysis of 20 hospitals with CPG compared with 6 non-CPG hospitals identified through PHIS database.     5 y Imaging: pre = $641; post = $163; P < .001; RR = 74.6% 
Laboratory: pre = $1044; post = $629; P < .001; RR = 39.8% 
Total hospital costs: pre = $10 830; post = $9868; P < .001; RR = 8.9% 
Knott et al71 (2013) US IP pediatric hospital single site Laboratories and CT in perforated appendicitis Before-after (n = 410) Series of prospective trials to develop new protocol. Included details on antibiotic use, laboratory work, CT scans, central venous catheter, diet, nasogastric tube, and total parenteral use.     2.25 y Mean laboratories: pre = 6.7; post = 2.3; P < .0001; RR = 65.7% 
Mean CT scans: pre = 1.3; post = 1.3; P = .86; no difference. 
Leinwand et al72 (2004) US IP/OP/ICU pediatric hospital single site Hematocrits and imaging in liver or splenic injury Before-after (n = 223) CPG developed based on APSA guidelines including length of ICU and hospital stay, follow-up imaging studies.     3.75 y Mean hematocrits: pre = 9.2; post = 4.7; P < .001; RR = 49% 
Mean follow-up imaging: pre = 2.1; post = 0.3; P < .001; RR = 86% 
Mittal et al73 (2014) US IP pediatric hospital multisite CBC; blood/urine culture; and CXR in bronchiolitis Cross-sectional (n = 14 306) Comparison of 28 institutions with CPG institutions versus 5 non-CPG institutions. CPG hospitals had similar guidelines for tests and treatments and recommended against routine tests.     1 y Median use: CBC: control = 37.3; study = 24.2; P = .131; RR = 35.1% 
Blood culture: control = 28.4; study = 21.2; P = .443; RR = 25.4% 
Urine culture: control = 24.3; Study = 18.4; P = .494; RR = 24.3%. CXR: Control = 66.7; study = 53.2; P = .131; RR = 20.2% 
Ogershok et al74 (2001) US IP pediatric hospital single site Chemistry; hematology and radiology after staffing/structure change Before-after (n = 2177) Restructured ward service. Attending physicians were newly hired or recruited from General Pediatrics; received no special training. Attending physicians no longer had dual OP/IP responsibilities; all certified general pediatricians.     1 y Mean chemistry: pre = 7; post = 5; P = .004; RR = 28.6% 
Mean hematology: pre = 3; post = 2; P = .001; RR = 33% 
Mean radiology: pre = 1.0; post = 0.8; P = .001; RR = 20% 
Polites et al56 (2014) US ED/IP general hospital single site CT, ultrasound and CBC in appendicitis Before-after (n = 331) Multidisciplinary team developed algorithm to include surgery consultation before imaging; and no imaging in children with unequivocal history.     3 y CT: pre = 38.5% (52); post = 18.4% (36); P < .001; AR = 20.1%; RR = 52.2% 
CBC: pre = 98.5% (133); post = 92.9% (182); P = .018; AR = 5.6%; RR = 5.7% 
Ultrasound: pre = 74.8% (101); post = 73.5% (144); P = .784; AR = 1.3%; RR = 1.7% 
Singer et al75 (2014) US ED pediatric hospital single site Radiology and laboratories after staffing/structure change Interrupted time series (n = 34 961) Pediatric ED opened. In addition to certified pediatric attending physicians, there are 1–4 house staff including pediatric and EM residents, rather than ED staffed by general emergency physicians.     11 mo Any imaging: pre = 42.5%; post = 39.3%; AR = 3.2%; RR = 7.5% 
CT: pre = 8.9%; post = 7.6%; AR = 1.3%; RR = 14.6% 
MRI: pre = 0.6%; post = 0.5%; AR = 0.1%; RR = 16.7% 
Any bloods: pre = 33.1%; post = 30.1%; AR = 3.0%; RR = 9.1% 
Toomey et al76 (2016) US IP single site CT and RSV testing in bronchiolitis Before-after (n = 385) Implemented new Standard of Care Guidelines to decrease admission rate, length of stay, and nonindicated therapies and procedures.     2 y RSV testing: ‘Decrease’ = 36% (change in CT use not reported) 
Warner et al57 (1998) US IP/ED pediatric hospital single site Multiple laboratories; AXR and ultrasound in appendicitis Before-after (n = 242) Developed and implemented clinical pathway on the basis of current resource use, literature review, and standard recommendations to promote early engagement of surgical teams.     1 y CBC: pre = ∼97%; post = ∼57%; P < .001; AR = ∼40%; RR = ∼41.2% 
Renal, serum electrolytes, blood urea nitrogen, creatinine: pre = ∼56%; post = ∼18%; P < .001; AR = ∼38%; RR = ∼67.9% 
AXR: pre = ∼57%; Post = ∼17%; P < .001; AR = ∼40%; RR = 70.2% 
Urinalysis: pre = ∼89%; post = ∼61%; P < .001; AR = ∼28%; RR = ∼31.5% 
Liver enzymes, serum amylase, pelvic ultrasound: pre = ∼40%; post = ∼25%; P < .05; AR = ∼15%; RR = ∼37.5% 
McCulloh et al59 (2014) US IP pediatric hospital multisite CXR; RSV; rapid flu; bloods; urine and lumbar in bronchiolitis Before-after (n = 1233) Pre- and post–2006 AAP Bronchiolitis CPG. No formalized care pathways used. Research teams at each site selected quality measures for assessment of impact on the basis of the AAP CPG.     2 y CXR: pre = 72.9% (385); post = 66.7% (470); P = .02; AR = 6.2%; RR = 8.5% 
RSV: pre = 91.3% (482); post = 84.0% (592); P < .0001; AR = 7.3%; RR = 8.0% 
Flu test: pre = 23.3% (123); post = 44.8% (316); P < .0001; AI = 21.5%; RI = 92.3% 
CBC: pre = 42.6% (225); post = 42.4% (299); P = .94; AR = 0.2%; RR = 0.5% 
Serum chemistry: pre = 29.7% (157); post = 31.2% (220); P = .58; AI = 1.5%; RI = 5.1% 
Blood gas: pre = 9.5% (50); post = 8.5% (60); P = .56; AR = 1.0%; RR = 10.5% 
Blood cultures: pre = 31.8% (168); post = 28.2% (199); P = .17 
AR = 3.6%; RR = 11.3% 
Urinalysis/culture: pre = 22.0% (116); post = 19.9% (40); P = .37; AR = 2.1%; RR = 9.5% 
Lumbar puncture: pre = 8.9% (47); post = 6.5% (46); P = .12; AR = 2.4%; RR = 27.0% 
Parikh et al60 (2014) US IP pediatric hospital multisite CXR; CBC; and RSV in bronchiolitis Interrupted time series (n = 37 907) Pre- and post–2006 AAP Bronchiolitis CPG that recommended supportive care with limited diagnostic testing and treatment. Use rates determined from billing data from PHIS database.     5.5 y CBC: pre = 34.8%; Post 1 = 34.9%; post 2 = 29.3%; P < .001; AR = 5.5%; RR = 15.8% 
CXR: pre = 61.3%; post 1 = 60.7%; post 2 = 52.1%; P < .001; AR = 9.2%; RR = 15.0% 
RSV: pre = 61.3%; post 1 = 59.9%; post 2 = 41.4%; P < .001; AR = 19.9%; RR = 32.5% 
Parikh et al55 (2016) US ED/IP pediatric hospital multisite CXR; blood culture; CBC and CRP in community-acquired pneumonia Interrupted time series (n = 120 539) Pre- and post–PIDS and IDSA 2011 guidelines discouraging blood tests, radiologic studies, and repeat testing for children hospitalized with community acquired pneumonia. Data from PHIS database.     3 y ED CRP: pre = 5.4%; post = 4.9%; P < .05; AR = 0.5%; RR = 9.3% 
Blood: pre = 14.6%; post = 8.6%; P = n.s.; AR = 6%; RR = 41.1% 
CBC: pre = 19.2%; post = 14.0%; P = n.s.; AR = 5.2%; RR = 27.1% 
CXR: pre = 85.4%; post = 81.1%; P = n.s.; AR = 4.3%; RR = 5.0% 
Inpatient blood: pre = 50.6%; post = 41.4%; P < .05; AR = 9.2%; RR = 18.2% 
CBC: pre = 65.2%; post = 62.2%, P = n.s.; AR = 3%; RR = 4.6% 
CRP: pre = 25.7%; post = 23.5%; P = n.s.; AR = 2.2%; RR = 8.6% 
CXR: pre = 89.4%; post = 83.9%; P = n.s.; AR = 5.5%; RR = 6.2 
Repeat inpatient CXR: pre = 25.5% post = 17.7%; P < .05; AR = 7.8%; RR = 30.6% 
Blood: pre = 6.5%; post = 5.8%; P = n.s.; AR = 0.7%; RR = 10.8% 
CBC: Pre = 23.4%; post = 16.0%; P = n.s.; AR = 7.4%; RR = 31.6% 
CRP: pre = 12.5%; post = 7.3%; P = n.s.; AR = 5.2%; RR = 41.6% 
Imaging and pathology: target = combined clinicians and patients/families (n = 1)  
Wazeka et al58 (2001) US IP general hospital single site Laboratories and radiology in asthma Before-after (n = 1004) Developed pathway from NIH guidelines and clinical experience. Hard copy in nursing stations and patient charts; lectures; periodic reviews; patient education and discharge planning.    4 y Mean laboratories per admission: pre = 13; post = 6.5; P < .0006; RR = 50% (mean radiologic studies per admission not reported). Radiology costs: pre = $31 514; post = $30 484; P = n.s.; RR = 3.3% 
Imaging and pathology: target = patients (only) (n = 0) 
Author (y) and SettingTest and ConditionDesign (No. Total Sample)Intervention DescriptionLOEIntervention CategoryFollow-up PeriodPrimary Outcomes
IASEG
Imaging & pathology: target = clinicians (only) (n = 21) 
Akenroye et al (2014)61 US ED pediatric hospital single site CXR and RSV testing in bronchiolitis Interrupted time series (n = 2929) AAP guideline adapted and implemented: weekly conferences; e-mail; online copies; monthly performance monitoring; feedback and peer comparison; pocket pamphlet; posters with costs.   1.5 y CXR: pre = 39%; post = 16%; P < .05; AR = 23%; RR = 59.0% 
RSV: pre = 33%; post = 22%; P < .05; AR = 11%; RR = 33.3% 
Shomaker et al62 (2011) US IP pediatric hospital single site CT; ultrasound; plain film; blood cultures; and pleural fluid cultures in empyema Before-after (n = 51) PDSA cycles prioritized areas for improvement; algorithm placed in ED, wards, PICU, intranet; CPOE decision-support and order set; monthly outcomes e-mail; ongoing changes to algorithm.   9 mo CT: pre = 41% (16); post = 0% (0); P = .01; AR = 41%; RR = 100%. Plain films: “No increase” ultrasound: pre = 38%; post = 50%; P = .35; AI = 12%; RI = 31.6%. Blood cultures: pre = 95% (37); post = 100% (12); P = .99; AI = 5%; RI = 5.3% 
Pleural fluid: pre = 82% (32); post = 83% (10); P = .99; AI = 1%; RI = 1.2% 
Perlstein et al63 (2000) US IP pediatric hospital single site CXR; blood gas and RSV testing in bronchiolitis Before-after (n = 1979) Guideline says “don't do” routine laboratory and radiologic studies, respiratory care therapies. Intervention reinforced with daily ward rounds and problems with implementation resolved.    3 y RSV: pre = 89%; post = 43%; P < .001; AR = 46.0%; RR = 51.7%; CXR: pre = 70%; post = 60%; P < .001; AR = 10%; RR = 14.3% 
Blood gases: pre = 6%; post = 9%; p < .001; AI = 3%; RI = 50% 
Fallon et al64 (2016) US ED pediatric hospital single site CT scans and laboratory costs in abdominal trauma Before-after (n = 321) Algorithm developed from literature. Included: posters, online info, education, CPOE decision support. Later, minor revisions made to protocol (eg, age cut-off for unreliable examinations; laboratory orders color coded). Education repeated; new materials.    2.75 y Rate of negative CTs: pre = 77%; post 1 = 68%; post 2 = 51%; P = .003; AR = 26.0%; RR = 33.8% 
Rate of clinically insignificant scans: pre = 86%; post 1 = 78%; post 2 = 68%; P = .03; AR = 18%; RR = 20.9% 
Median laboratory costs: pre = $166; post 1 = $352; post 2 = $139; P = .005; RR = 16.3% 
Guse et al65 (2014) US ED pediatric hospital single site CBC; electrolytes; CXR and CT in syncope Before-after (n = 349) Guideline developed from literature and collaboration. Included: weekly conference; e-mail reminders; displays in ED; online; electronic order set; discharge information; pocket cards.    1 y CBC: pre = 36% (61); post = 16% (29); AR = 20%; RR = 55.6% 
Electrolytes: pre = 29% (50); post = 12% (21); AR = 17%; RR = 58.6% 
CXR: pre = 12% (21); post = 8% (14); AR = 4%; RR = 33.3% 
Head CT: pre = 2% (4); post = 2% (4); no difference 
Mohan et al66 (2016) US ED pediatric hospital single site Multiple laboratories and CXR in chest pain Interrupted time series (n = 1687) Pathway developed from literature and consultation. Electronic order set; intensive staff education; division meeting presentation. Visiting staff received orientation and education.    1 y CXR: pre = 46.1%; post = 35.6%; P < .01; AR = 10.5%; RR = 22.8% 
“No difference in”: troponin, creatine kinase, B-type natriuretic peptide, BMP, CBC, urine drug screen, CRP, sedimentation rate, D-dimer, echocardiogram 
Morris et al67 (2015) US ICU pediatric hospital single site Laboratories and radiology after staffing/structure change Interrupted time series (n = 349) Created “small baby unit” separate from NICU; smaller, darker, quieter. Guidelines and checklists based on child age in hard copy and intranet. Lead physician and nurse as coordinators and clinicians for continuity of care. Training and biweekly talks.    4 y Mean laboratories (blood gas, electrolyte, CBC): pre = 224; post = 82; P < .001; RR = 63.4% 
Mean radiographs (abdo & chest): pre = 45; post = 22; P < .001; RR = 51.1% 
Raucci et al68 (2014) Europe (Italy) ED pediatric hospital single site Echocardiography; CT; MRI and blood workup in syncope Before-after (n = 1073) CPG developed. Educational seminars; electronic guidelines and algorithms; training simulation; daily ED attendance by study team for supervision of implementation, and resolution of problems.    2 y Echo: pre = 5.1%; post = 3%; P = .07; AR = 2.1%; RR = 41.2% 
CT/MRI: pre = 5.5%; post = 1.3%; P < .001; AR = 4.2%; RR = 76.4%. Bloods: pre = 32.1%; post = 13.9%; P < .001; AR = 18.2%; RR = 56.7% 
Fallon et al69 (2014) US ED pediatric hospital single site CT; radiograph; laboratories after staffing/structure change Interrupted time series (n = 472) Implemented protocol where ED physician (not surgery team) primarily manages “level 2” trauma activations with trauma consultation for surgical evaluation or admission.     9 mo Mean CT: pre = 1.4; post = 1.0; P < .001; RR = 28.6%; mean radiographs: pre = 2.4; post = 1.7; P < .001; RR = 29.2%; mean laboratories: pre = 5.1; post = 3.3; P < .001; RR = 35.3% 
Gutierrez et al70 (2013) US IP pediatric hospital multisite Imaging, laboratory and total hospital costs in splenic injury Cross-sectional (n = 1154) APSA Trauma Committee CPG proposed in 1998. Analysis of 20 hospitals with CPG compared with 6 non-CPG hospitals identified through PHIS database.     5 y Imaging: pre = $641; post = $163; P < .001; RR = 74.6% 
Laboratory: pre = $1044; post = $629; P < .001; RR = 39.8% 
Total hospital costs: pre = $10 830; post = $9868; P < .001; RR = 8.9% 
Knott et al71 (2013) US IP pediatric hospital single site Laboratories and CT in perforated appendicitis Before-after (n = 410) Series of prospective trials to develop new protocol. Included details on antibiotic use, laboratory work, CT scans, central venous catheter, diet, nasogastric tube, and total parenteral use.     2.25 y Mean laboratories: pre = 6.7; post = 2.3; P < .0001; RR = 65.7% 
Mean CT scans: pre = 1.3; post = 1.3; P = .86; no difference. 
Leinwand et al72 (2004) US IP/OP/ICU pediatric hospital single site Hematocrits and imaging in liver or splenic injury Before-after (n = 223) CPG developed based on APSA guidelines including length of ICU and hospital stay, follow-up imaging studies.     3.75 y Mean hematocrits: pre = 9.2; post = 4.7; P < .001; RR = 49% 
Mean follow-up imaging: pre = 2.1; post = 0.3; P < .001; RR = 86% 
Mittal et al73 (2014) US IP pediatric hospital multisite CBC; blood/urine culture; and CXR in bronchiolitis Cross-sectional (n = 14 306) Comparison of 28 institutions with CPG institutions versus 5 non-CPG institutions. CPG hospitals had similar guidelines for tests and treatments and recommended against routine tests.     1 y Median use: CBC: control = 37.3; study = 24.2; P = .131; RR = 35.1% 
Blood culture: control = 28.4; study = 21.2; P = .443; RR = 25.4% 
Urine culture: control = 24.3; Study = 18.4; P = .494; RR = 24.3%. CXR: Control = 66.7; study = 53.2; P = .131; RR = 20.2% 
Ogershok et al74 (2001) US IP pediatric hospital single site Chemistry; hematology and radiology after staffing/structure change Before-after (n = 2177) Restructured ward service. Attending physicians were newly hired or recruited from General Pediatrics; received no special training. Attending physicians no longer had dual OP/IP responsibilities; all certified general pediatricians.     1 y Mean chemistry: pre = 7; post = 5; P = .004; RR = 28.6% 
Mean hematology: pre = 3; post = 2; P = .001; RR = 33% 
Mean radiology: pre = 1.0; post = 0.8; P = .001; RR = 20% 
Polites et al56 (2014) US ED/IP general hospital single site CT, ultrasound and CBC in appendicitis Before-after (n = 331) Multidisciplinary team developed algorithm to include surgery consultation before imaging; and no imaging in children with unequivocal history.     3 y CT: pre = 38.5% (52); post = 18.4% (36); P < .001; AR = 20.1%; RR = 52.2% 
CBC: pre = 98.5% (133); post = 92.9% (182); P = .018; AR = 5.6%; RR = 5.7% 
Ultrasound: pre = 74.8% (101); post = 73.5% (144); P = .784; AR = 1.3%; RR = 1.7% 
Singer et al75 (2014) US ED pediatric hospital single site Radiology and laboratories after staffing/structure change Interrupted time series (n = 34 961) Pediatric ED opened. In addition to certified pediatric attending physicians, there are 1–4 house staff including pediatric and EM residents, rather than ED staffed by general emergency physicians.     11 mo Any imaging: pre = 42.5%; post = 39.3%; AR = 3.2%; RR = 7.5% 
CT: pre = 8.9%; post = 7.6%; AR = 1.3%; RR = 14.6% 
MRI: pre = 0.6%; post = 0.5%; AR = 0.1%; RR = 16.7% 
Any bloods: pre = 33.1%; post = 30.1%; AR = 3.0%; RR = 9.1% 
Toomey et al76 (2016) US IP single site CT and RSV testing in bronchiolitis Before-after (n = 385) Implemented new Standard of Care Guidelines to decrease admission rate, length of stay, and nonindicated therapies and procedures.     2 y RSV testing: ‘Decrease’ = 36% (change in CT use not reported) 
Warner et al57 (1998) US IP/ED pediatric hospital single site Multiple laboratories; AXR and ultrasound in appendicitis Before-after (n = 242) Developed and implemented clinical pathway on the basis of current resource use, literature review, and standard recommendations to promote early engagement of surgical teams.     1 y CBC: pre = ∼97%; post = ∼57%; P < .001; AR = ∼40%; RR = ∼41.2% 
Renal, serum electrolytes, blood urea nitrogen, creatinine: pre = ∼56%; post = ∼18%; P < .001; AR = ∼38%; RR = ∼67.9% 
AXR: pre = ∼57%; Post = ∼17%; P < .001; AR = ∼40%; RR = 70.2% 
Urinalysis: pre = ∼89%; post = ∼61%; P < .001; AR = ∼28%; RR = ∼31.5% 
Liver enzymes, serum amylase, pelvic ultrasound: pre = ∼40%; post = ∼25%; P < .05; AR = ∼15%; RR = ∼37.5% 
McCulloh et al59 (2014) US IP pediatric hospital multisite CXR; RSV; rapid flu; bloods; urine and lumbar in bronchiolitis Before-after (n = 1233) Pre- and post–2006 AAP Bronchiolitis CPG. No formalized care pathways used. Research teams at each site selected quality measures for assessment of impact on the basis of the AAP CPG.     2 y CXR: pre = 72.9% (385); post = 66.7% (470); P = .02; AR = 6.2%; RR = 8.5% 
RSV: pre = 91.3% (482); post = 84.0% (592); P < .0001; AR = 7.3%; RR = 8.0% 
Flu test: pre = 23.3% (123); post = 44.8% (316); P < .0001; AI = 21.5%; RI = 92.3% 
CBC: pre = 42.6% (225); post = 42.4% (299); P = .94; AR = 0.2%; RR = 0.5% 
Serum chemistry: pre = 29.7% (157); post = 31.2% (220); P = .58; AI = 1.5%; RI = 5.1% 
Blood gas: pre = 9.5% (50); post = 8.5% (60); P = .56; AR = 1.0%; RR = 10.5% 
Blood cultures: pre = 31.8% (168); post = 28.2% (199); P = .17 
AR = 3.6%; RR = 11.3% 
Urinalysis/culture: pre = 22.0% (116); post = 19.9% (40); P = .37; AR = 2.1%; RR = 9.5% 
Lumbar puncture: pre = 8.9% (47); post = 6.5% (46); P = .12; AR = 2.4%; RR = 27.0% 
Parikh et al60 (2014) US IP pediatric hospital multisite CXR; CBC; and RSV in bronchiolitis Interrupted time series (n = 37 907) Pre- and post–2006 AAP Bronchiolitis CPG that recommended supportive care with limited diagnostic testing and treatment. Use rates determined from billing data from PHIS database.     5.5 y CBC: pre = 34.8%; Post 1 = 34.9%; post 2 = 29.3%; P < .001; AR = 5.5%; RR = 15.8% 
CXR: pre = 61.3%; post 1 = 60.7%; post 2 = 52.1%; P < .001; AR = 9.2%; RR = 15.0% 
RSV: pre = 61.3%; post 1 = 59.9%; post 2 = 41.4%; P < .001; AR = 19.9%; RR = 32.5% 
Parikh et al55 (2016) US ED/IP pediatric hospital multisite CXR; blood culture; CBC and CRP in community-acquired pneumonia Interrupted time series (n = 120 539) Pre- and post–PIDS and IDSA 2011 guidelines discouraging blood tests, radiologic studies, and repeat testing for children hospitalized with community acquired pneumonia. Data from PHIS database.     3 y ED CRP: pre = 5.4%; post = 4.9%; P < .05; AR = 0.5%; RR = 9.3% 
Blood: pre = 14.6%; post = 8.6%; P = n.s.; AR = 6%; RR = 41.1% 
CBC: pre = 19.2%; post = 14.0%; P = n.s.; AR = 5.2%; RR = 27.1% 
CXR: pre = 85.4%; post = 81.1%; P = n.s.; AR = 4.3%; RR = 5.0% 
Inpatient blood: pre = 50.6%; post = 41.4%; P < .05; AR = 9.2%; RR = 18.2% 
CBC: pre = 65.2%; post = 62.2%, P = n.s.; AR = 3%; RR = 4.6% 
CRP: pre = 25.7%; post = 23.5%; P = n.s.; AR = 2.2%; RR = 8.6% 
CXR: pre = 89.4%; post = 83.9%; P = n.s.; AR = 5.5%; RR = 6.2 
Repeat inpatient CXR: pre = 25.5% post = 17.7%; P < .05; AR = 7.8%; RR = 30.6% 
Blood: pre = 6.5%; post = 5.8%; P = n.s.; AR = 0.7%; RR = 10.8% 
CBC: Pre = 23.4%; post = 16.0%; P = n.s.; AR = 7.4%; RR = 31.6% 
CRP: pre = 12.5%; post = 7.3%; P = n.s.; AR = 5.2%; RR = 41.6% 
Imaging and pathology: target = combined clinicians and patients/families (n = 1)  
Wazeka et al58 (2001) US IP general hospital single site Laboratories and radiology in asthma Before-after (n = 1004) Developed pathway from NIH guidelines and clinical experience. Hard copy in nursing stations and patient charts; lectures; periodic reviews; patient education and discharge planning.    4 y Mean laboratories per admission: pre = 13; post = 6.5; P < .0006; RR = 50% (mean radiologic studies per admission not reported). Radiology costs: pre = $31 514; post = $30 484; P = n.s.; RR = 3.3% 
Imaging and pathology: target = patients (only) (n = 0) 

A, audit and feedback; AAP, American Academy of Pediatrics; AI, absolute increase in testing; APSA, American Pediatric Surgical Association; AXR, abdominal radiograph; BMP, basic metabolic panel; CBC, complete blood count; CPG, clinical practice guideline; CPOE, computerized physician order entry; CXR, chest radiograph; CRP, C-reactive protein; E, education; G, guideline publication; GP, general practice setting; I, incentive or penalty; IDSA, Infectious Diseases Society of America; IP, inpatient setting; LOE, level of evidence; NIH, National Institutes of Health; n.s, not significant; OP, outpatient setting; PDSA, plan-do-study-act cycle; PHIS, Pediatric Health Information System; PIDS, Pediatric Infectious Diseases Society; RI, relative increase in testing; RSV, respiratory syncytial virus; S, system- and/or process-based.

Overall, there is evidence to suggest that intervention complexity was related to reduced testing (Fig 2), with a combination of 3 components (system- and/or process-based, education, and audit and feedback) appearing more effective (mean RR = 45.0%; SD = 28.3%) than 1 (mean RR = 28.6%; SD = 34.9%) or 2 components (mean RR = 32.0%; SD = 30.3%). Of the single component interventions, the most effective type was education alone (mean RR = 57.9%; SD = 13.6%); however, this mean is calculated from only 3 studies. Studies contained reports of varying effectiveness of system- and/or process-based interventions (mean RR = 35.0%; SD = 33.3%) and guideline publication (mean RR = 15.8%; SD = 34.7%).

FIGURE 2

Box plots of relative changes in testing stratified by combinations of intervention components. Negative values represent a reduction in testing; positive values represent an increase in testing. There were no other combinations of intervention components. System- and/or process-based (n = 24); education (n = 3); guideline publication (n = 7); audit and feedback plus system- and/or process-based (n = 3); education plus system- and/or process-based (n = 14); audit and feedback plus system- and/or process-based plus education (n = 13).

FIGURE 2

Box plots of relative changes in testing stratified by combinations of intervention components. Negative values represent a reduction in testing; positive values represent an increase in testing. There were no other combinations of intervention components. System- and/or process-based (n = 24); education (n = 3); guideline publication (n = 7); audit and feedback plus system- and/or process-based (n = 3); education plus system- and/or process-based (n = 14); audit and feedback plus system- and/or process-based plus education (n = 13).

Interventions were more effective if they targeted only imaging (mean RR = 41.8%; SD = 38.4) or only pathology testing (mean RR = 48.8%; SD = 20.9) rather than both simultaneously (mean RR = 21.6%; SD = 29.2). Interventions in which both clinicians and patients were targeted appeared more effective (mean RR = 61.9%; SD = 34.3) than those in which clinicians alone were targeted (mean RR = 30.0%; SD = 32.0). The only studies in which patients alone were targeted was a series of 3 studies on irritable bowel syndrome in adolescents,17,19 in which a mean RR of 57.9% (SD = 13.6) was reported.

The type of test was also associated with differences in reductions, with greater RR seen in routine follow-up imaging (mean RR = 76.1%; SD = 12.1%) and invasive voiding cystourethrogram (VCUG) tests (mean RR = 69.3%; SD = 20.0%) compared with respiratory syncytial virus testing (mean RR = 11.5%; SD = 52.8%). In studies aimed at reducing CT scans for appendicitis, CT scan use decreased but abdominal ultrasound use increased (mean increase = 15.0%; SD 23.5%).

Setting of care was also associated with varying reductions in unnecessary testing. Consistent reductions were observed in the ICU (mean RR = 47.6%; SD = 21.0%) and ED (mean RR = 32.3%; SD = 25.5%), with no studies containing reports of an increase in testing in these settings. In contrast, 5 studies contained reports of an increase in testing in inpatient and outpatient settings. Two of these studies20,28 involved multiple sites where each site implemented the suite of interventions separately. Some sites had reductions, whereas others had an increase in LVC.28 Two studies23,59 contained reports on the effect of publication of external guidelines with no individual site modification and/or implementation. Differences in site characteristics and/or how the interventions were implemented may account for the different outcomes. Reductions in testing were smaller and more variable for inpatient (mean RR = 25.7%; SD = 34.1%) and multiple setting studies (mean RR = 26.7%; SD = 48.7%). Greater reductions were observed for single-site (mean RR = 35.6%; SD = 31.4%) versus multisite interventions (mean RR = 24.6%; SD = 35.6%) and for general hospitals (mean RR = 41.7%; SD = 38.9%) versus pediatric hospitals (mean RR = 31.5%; SD = 30.6%).

Most studies contained reports on at least 1 balancing measure (ie, secondary effects of the intervention such as length of stay in hospital or ED, admission rates, mortality, or cost). All authors reported either no change or an improvement in these outcomes after the intervention. Eighteen studies contained reports of a reduction in length of stay, and an additional 17 studies contained reports of no change. Seven contained reports of a reduced rate of admission to hospital, whereas 11 studies contained reports of no change. Only 1 study62 contained a report of a reduction in readmissions, and 14 contained reports of no change. Of the 21 studies that contained reports on the effect of the intervention on hospital or patient costs, all contained reports of a cost saving as a result of conducting fewer tests. However, no authors conducted a cost-effectiveness analysis for the intervention as a whole. Two studies67,74 contained reports of lower mortality in the postintervention period, and 9 contained reports of no change.

Overall quality of included studies was poor. The κ interrater reliability for the study risk of bias ratings was 0.82, with all discrepancies subsequently resolved through discussion. Similarity in risk of bias ratings across the studies was largely because of the study design; the majority of studies were before-after design or interrupted-time series designs and were graded fairly equally. Only 1 RCT was conducted, and it was rated as a low risk of bias. All other studies were rated as high risk of bias on at least 2 of the 9 Cochrane Effective Practice and Organization of Care domains, although approximately one-third (n = 20) of studies were rated high risk on 3 or more domains. The majority of studies did not contain enough information to provide a clear assessment, and many domains were rated as unclear risk. Study level of evidence based on Oxford Centre for Evidence-Based Medicine ratings is presented for each study alongside intervention details in Tables 13.

The majority of studies designed to evaluate interventions to reduce LVC in pediatric imaging and pathology reveal positive effects ranging from small to large. Characteristics of more effective interventions include multifaceted as opposed to single-component interventions, interventions that target imaging or pathology only (and not both), and interventions that include both families and their treating clinicians. Greater reductions were seen in general compared with pediatric hospitals, likely because general hospitals usually had higher baseline levels of testing than pediatric hospitals. No studies contained reports on our secondary outcome of cost-effectiveness; however, studies generally contained reports of an improvement (or no adverse effect) on balancing measures such as length of stay, admissions, and hospital or patient costs.

The authors of 4 studies15,16,45,58 implemented an intervention targeting both clinicians and families. The authors of 1 of these studies45 aimed to reduce blood testing after surgical closure of atrial septal defects in a single general hospital. A clinical pathway for postoperative care was introduced, and patients and families were educated on what care to expect and were informed of significant variations from the clinical pathway. They found promising reductions in unnecessary blood testing (RR range: 13.7%–69.2%) that were sustained up to 2.5 years post–initial intervention. Even more effective were the 2 studies15,16 in which unnecessary imaging was targeted and clinicians and patients were included in the intervention; 1 of these studies16 contained a report of a 100% reduction in VCUG testing after a normal renal bladder ultrasound (RBUS) in suspected urinary tract infection (UTI) in inpatient care. The researchers achieved this at a single pediatric hospital through the use of a multifaceted intervention including education for staff and families, lectures, electronic medical record (EMR) order set changes, and audit and feedback. Further comparative studies are needed to determine if family involvement increases efficacy and sustainability of interventions. Communicating risks of LVC practices to families (eg, radiation doses in abdominal radiographs, increased length of stay, and exposure to nosocomial infections after false-positive blood test results) may be a powerful tool in prompting families to refuse or at least question clinician-driven, LVC practices.

Effective methods to reduce unnecessary imaging depended on the condition. For example, common to effective methods to reduce abdominal CTs in children with suspected appendicitis was an early surgical consult. The authors of 1 study15 reported a comparatively high (92.4%) RR in use of chest radiographs in asthma versus 29.1% and 35.4% reported in the other 2 studies31,33 that contained reports on this. The more effective of the 3 studies15 included patient asthma education sessions and assigned patients to a primary care provider who was trained in national asthma guidelines.

Similarly, the authors of 4 studies67,69,74,75 examined changes in resource use after a staffing and/or structural change (eg, a dedicated pediatric ED or staffing changes), although the authors of 1 study67 saw greater reductions than was reported in other studies, and they implemented a staffing and/or structure change with supporting education. The authors opened a dedicated “small baby unit” separate from the NICU, with processes promoting continuity of care and education for staff. Education comprised ongoing training and talks (ie, 16 hours of baseline education followed by quarterly 3-hour case discussions, informal talks twice per week, and weekly pharmacy and nutrition rounds). This study achieved a 63% reduction in laboratory testing and 51% decrease in abdominal and chest radiographs over a 4-year follow-up period.

Overall, the quality of the studies was poor. The majority of studies were quality improvement studies, which are difficult to assess with traditional evidence grading scales yet may still offer ideas as to how best to reduce LVC. Only 1 RCT was conducted,20 and this had a low risk of bias. RCTs can be difficult to conduct in complex health care systems. Recent evidence suggests that ITS designs may offer results similar to those arising from RCTs but at a fraction of the cost and time.77 Although several before-after studies revealed positive effects in reducing unnecessary imaging and pathology testing, results are likely to be overinflated because they do not account for secular trends in care patterns or potential confounders of the effect of the intervention on the LVC practice. Replication of these effects should be demonstrated in ITS or RCTs before claims of effectiveness can be confirmed. In this respect, the pediatric literature contrasts with the adult literature in which more RCTs have been conducted.9,78 In Kobewka et al’s9 systematic review of interventions to reduce unnecessary pathology testing, 14 of 109 studies were RCTs. However, similar to our review, Kobewka et al9 found that interventions were more effective if they were multicomponent and their conductors targeted fewer rather than more tests. Sustainability was also an issue in adult studies; only 15 of 109 studies contained reports of outcomes beyond 12 months. Unlike the pediatric literature, no authors targeted clinicians and patients in their interventions.

Our review has a number of strengths. We conducted a comprehensive search across published and gray literature and only included studies with a follow-up period of >6 months to assess sustainability. We rigorously assessed study risk of bias and had high interrater reliability in doing so. We note some limitations. First, the overall risk of bias was high because of poor study quality. Second, we limited our screening to English language papers and so excluded ∼6 papers published in another language. However, we scanned over 10 000 texts and included 64 studies and believe it is unlikely that inclusion of a handful of non-English language studies would have changed our conclusions in any meaningful way. We were unable to pool results because of the heterogeneous nature of the interventions. We thus attempted to estimate, when data permitted, relative and absolute changes in LVC, pre- and post-interventions. Finally, most studies were conducted in single sites, predominantly in US hospitals, possibly because of the availability of large health data sets to analyze, together with incentives (eg, hospital rankings) to reduce wasteful care. This reduces the external validity of the findings given that health systems and practices vary widely between countries and between hospitals.

There is no single solution to reducing LVC. Our systematic review suggests that future researchers should rigorously evaluate interventions that are multifaceted, include audit and feedback, system-based changes and education, and target 1 low-value practice at a time. Aligning with key recommendations from the 2017 National Quality Forum report79 on improving diagnostic quality, interventions to reduce LVC should be codesigned with patients and clinicians (a relatively novel approach in the LVC literature), tailored to the needs of their environment, and take advantage of electronic health records to report on outcomes. Evaluations should report on both effectiveness and cost-effectiveness outcomes, ensure long-term follow-up to assess sustainability of the intervention(s), and ideally be conducted across multiple sites to provide evidence of generalizability.

     
  • AR

    absolute reduction

  •  
  • CT

    computed tomography

  •  
  • ED

    emergency department

  •  
  • EMR

    electronic medical record

  •  
  • LVC

    low-value care

  •  
  • NICE

    National Institute for Health and Care Excellence

  •  
  • RBUS

    renal bladder ultrasound

  •  
  • RCT

    randomized controlled trial

  •  
  • RR

    relative reduction

  •  
  • UTI

    urinary tract infection

  •  
  • VCUG

    voiding cystourethrogram

Dr Hiscock conceptualized and designed the study, interpreted the data, and drafted and revised the article critically for important intellectual content; Ms Neely extracted, analyzed, and interpreted the data and drafted the manuscript; Ms Warren extracted and interpreted the data and drafted the manuscript; Mr Soon extracted the data and drafted the manuscript; Dr Georgiou interpreted the data and revised the article critically for important intellectual content; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

This trial has been registered with the PROSPERO international prospective register of systematic reviews (identifier CRD42016047960).

FUNDING: The Health Services Research Unit of The Royal Children’s Hospital is funded by a Royal Children’s Hospital Foundation grant (2015-521). Dr Hiscock is supported by a National Health and Medical Research Council Career Development Fellowship (1068947). Mr Soon is supported by the Policy and Advocacy Department of the Royal Australasian College of Physicians. The Murdoch Children’s Research Institute is supported by the Victorian Government’s Operational Infrastructure Program. The funding organizations had no role in the design and conduct of the study; collection, management, analysis, or interpretation of the data; preparation, review, or approval of the manuscript; or the decision to submit the manuscript for publication.

COMPANION PAPER: A companion to this article can be found online at www.pediatrics.org/cgi/doi/10.1542/peds.2017-3859.

1
Scott
IA
,
Duckett
SJ
.
In search of professional consensus in defining and reducing low-value care.
Med J Aust
.
2015
;
203
(
4
):
179
181
[PubMed]
2
Levinson
W
,
Kallewaard
M
,
Bhatia
RS
,
Wolfson
D
,
Shortt
S
,
Kerr
EA
;
Choosing Wisely International Working Group
.
‘Choosing Wisely’: a growing international campaign.
BMJ Qual Saf
.
2015
;
24
(
2
):
167
174
[PubMed]
3
Lallemand
NC
. Health policy brief: Reducing waste in health care. December 13, 2012. Available at: https://www.healthaffairs.org/do/10.1377/hpb20121213.959735/full/
4
Hong
AS
,
Ross-Degnan
D
,
Zhang
F
,
Wharam
JF
.
Small decline in low-value back imaging associated with the ‘Choosing Wisely’ campaign, 2012-14.
Health Aff (Millwood)
.
2017
;
36
(
4
):
671
679
[PubMed]
5
Brehaut
JC
,
Colquhoun
HL
,
Eva
KW
, et al
.
Practice feedback interventions: 15 suggestions for optimizing effectiveness.
Ann Intern Med
.
2016
;
164
(
6
):
435
441
[PubMed]
6
Ivers
N
,
Jamtvedt
G
,
Flottorp
S
, et al
.
Audit and feedback: effects on professional practice and healthcare outcomes.
Cochrane Database Syst Rev
.
2012
;(
6
):
CD000259
[PubMed]
7
Moher
D
,
Liberati
A
,
Tetzlaff
J
,
Altman
DG
;
PRISMA Group
.
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement.
BMJ
.
2009
;
339
:
b2535
[PubMed]
8
National Institute for Health and Care Excellence
. Cost saving and resource planning. Available at: https://www.nice.org.uk/savingsAndProductivity/collection?page=1&pageSize=2000&type=Do+not+do&published. Accessed November 6, 2017
9
Kobewka
DM
,
Ronksley
PE
,
McKay
JA
,
Forster
AJ
,
van Walraven
C
.
Influence of educational, audit and feedback, system based, and incentive and penalty interventions to reduce laboratory test utilization: a systematic review.
Clin Chem Lab Med
.
2015
;
53
(
2
):
157
183
[PubMed]
10
Thomas
J
,
Brunton
J
,
Graziosi
S
.
EPPI-Reviewer 4: Software for Research Synthesis. EPPI-Centre Software
.
London, United Kingdom
:
Social Science Research Unit, UCL Institute of Education
;
2010
11
Effective Practice and Organisation of Care
. Suggested risk of bias criteria for EPOC reviews. Available at: http://epoc.cochrane.org/sites/epoc.cochrane.org/files/public/uploads/Resources-for-authors2017/suggested_risk_of_bias_criteria_for_epoc_reviews.pdf
12
Oxford Centre for Evidence-Based Medicine
. Oxford Centre for Evidence-Based Medicine 2011 levels of evidence. Available at: http://www.cebm.net/wp-content/uploads/2014/06/CEBM-Levels-of-Evidence-2.1.pdf. Accessed November 6, 2017
13
South
M
.
Radiological investigations following urinary tract infection: changes in Australian practice.
Arch Dis Child
.
2009
;
94
(
12
):
927
930
[PubMed]
14
Boutis
K
,
Grootendorst
P
,
Willan
A
, et al
.
Effect of the low risk ankle rule on the frequency of radiography in children with ankle injuries.
CMAJ
.
2013
;
185
(
15
):
E731
E738
[PubMed]
15
Higgins
JC
,
Kiser
WR
,
McClenathan
S
,
Tynan
NL
.
Influence of an interventional program on resource use and cost in pediatric asthma.
Am J Manag Care
.
1998
;
4
(
10
):
1465
1469
[PubMed]
16
Jerardi
KE
,
Elkeeb
D
,
Weiser
J
,
Brinkman
WB
.
Rapid implementation of evidence-based guidelines for imaging after first urinary tract infection.
Pediatrics
.
2013
;
132
(
3
). Available at: www.pediatrics.org/cgi/content/full/132/3/e749
[PubMed]
17
Kalani
A
,
Gilroy
M
,
Del Pozo
J
,
Orr
T
,
Lawson
M
.
Reduced healthcare utilization achieved by a multidisciplinary behavioral intervention for adolescents with irritable bowel syndrome (IBS): four-year healthcare utilization data including diagnostic imaging studies, number, and nature of healthcare visits.
Am J Gastroenterol
.
2014
;
109
(
suppl 2
):
S544
S593
18
Kalani
A
,
Griem
A
,
Crebassa
L
, et al
.
A simple intervention program markedly decreases resource utilization for adolescents with irritable bowel syndrome (IBS).
Am J Gastroenterol
.
2012
;
107
(
Suppl 1
):
S700
S701
19
Kalani
A
,
Griem
A
,
Crebassa
L
,
Mow
W
,
Lawson
MJ
.
Recognizing the “crisis year” in adolescent irritable bowel syndrome (IBS): an effective program that can reduce healthcare utilization.
Gastroenterol
.
2013
;
144
(
5 suppl 1
):
S93
S94
20
Chien
AT
,
Ganeshan
S
,
Schuster
MA
, et al
.
The effect of price information on the ordering of images and procedures.
Pediatrics
.
2017
;
139
(
2
):
e20161507
[PubMed]
21
Arlen
AM
,
Merriman
LS
,
Kirsch
JM
, et al
.
Early effect of American Academy of Pediatrics urinary tract infection guidelines on radiographic imaging and diagnosis of vesicoureteral reflux in the emergency room setting.
J Urol
.
2015
;
193
(
suppl 5
):
1760
1765
[PubMed]
22
Graf
WD
,
Kayyali
HR
,
Alexander
JJ
,
Simon
SD
,
Morriss
MC
.
Neuroimaging-use trends in nonacute pediatric headache before and after clinical practice parameters.
Pediatrics
.
2008
;
122
(
5
). Available at: www.pediatrics.org/cgi/content/full/122/5/e1001
[PubMed]
23
Judkins
A
,
Pascoe
E
,
Payne
D
.
Management of urinary tract infection in a tertiary children’s hospital before and after publication of the NICE guidelines.
Arch Dis Child
.
2013
;
98
(
7
):
521
525
[PubMed]
24
Akenroye
AT
,
Stack
AM
.
The development and evaluation of an evidence-based guideline programme to improve care in a paediatric emergency department.
Emerg Med J
.
2016
;
33
(
2
):
109
117
[PubMed]
25
Kwon
H
,
Jung
JY
.
Effectiveness of a radiation reduction campaign targeting children with gastrointestinal symptoms in a pediatric emergency department.
Medicine (Baltimore)
.
2017
;
96
(
3
):
e5907
[PubMed]
26
Mittal
V
,
Darnell
C
,
Walsh
B
, et al
.
Inpatient bronchiolitis guideline implementation and resource utilization.
Pediatrics
.
2014
;
133
(
3
). Available at: www.pediatrics.org/cgi/content/full/133/3/e730
[PubMed]
27
Nigrovic
LE
,
Stack
AM
,
Mannix
RC
, et al
.
Quality improvement effort to reduce cranial CTs for children with minor blunt head trauma.
Pediatrics
.
2015
;
136
(
1
). Available at: www.pediatrics.org/cgi/content/full/136/1/e227
[PubMed]
28
Ralston
SL
,
Garber
MD
,
Rice-Conboy
E
, et al;
Value in Inpatient Pediatrics Network Quality Collaborative for Improving Hospital Compliance with AAP Bronchiolitis Guideline (BQIP)
.
A multicenter collaborative to reduce unnecessary care in inpatient bronchiolitis.
Pediatrics
.
2016
;
137
(
1
):e20150851
[PubMed]
29
Shah
SR
,
Sinclair
KA
,
Theut
SB
,
Johnson
KM
,
Holcomb
GW
 III
,
St Peter
SD
.
Computed tomography utilization for the diagnosis of acute appendicitis in children decreases with a diagnostic algorithm.
Ann Surg
.
2016
;
264
(
3
):
474
481
[PubMed]
30
Goldberg
J
,
McClaine
RJ
,
Cook
B
, et al
.
Use of a mild traumatic brain injury guideline to reduce inpatient hospital imaging and charges.
J Pediatr Surg
.
2011
;
46
(
9
):
1777
1783
[PubMed]
31
Buckmaster
A
,
Boon
R
.
Reduce the rads: a quality assurance project on reducing unnecessary chest x-rays in children with asthma.
J Paediatr Child Health
.
2005
;
41
(
3
):
107
111
[PubMed]
32
Depinet
H
,
von Allmen
D
,
Towbin
A
,
Hornung
R
,
Ho
M
,
Alessandrini
E
.
Risk stratification to decrease unnecessary diagnostic imaging for acute appendicitis.
Pediatrics
.
2016
;
138
(
3
):
e20154031
[PubMed]
33
Gildenhuys
J
,
Lee
M
,
Isbister
GK
.
Does implementation of a paediatric asthma clinical practice guideline worksheet change clinical practice?
Int J Emerg Med
.
2009
;
2
(
1
):
33
39
[PubMed]
34
Russell
WS
,
Schuh
AM
,
Hill
JG
, et al
.
Clinical practice guidelines for pediatric appendicitis evaluation can decrease computed tomography utilization while maintaining diagnostic accuracy.
Pediatr Emerg Care
.
2013
;
29
(
5
):
568
573
[PubMed]
35
Browning
JG
,
Reed
MJ
,
Wilkinson
AG
,
Beattie
T
.
Imaging infants with head injury: effect of a change in policy.
Emerg Med J
.
2005
;
22
(
1
):
33
36
[PubMed]
36
Bowen
S
,
Johnson
K
,
Reed
MH
,
Zhang
L
,
Curry
L
.
The effect of incorporating guidelines into a computerized order entry system for diagnostic imaging.
J Am Coll Radiol
.
2011
;
8
(
4
):
251
258
[PubMed]
37
Luther
G
,
Miller
PE
,
Mahan
ST
,
Waters
PM
,
Bae
DS
.
Decreasing resource utilization using standardized clinical assessment and management plans (SCAMPs) [published online ahead of print September 15, 2016].
J Pediatr Orthop
. doi:
[PubMed]
38
Price
MB
,
Grant
MJC
,
Welkie
K
.
Financial impact of elimination of routine chest radiographs in a pediatric intensive care unit.
Crit Care Med
.
1999
;
27
(
8
):
1588
1593
[PubMed]
39
Reed
MJ
,
Browning
JG
,
Wilkinson
AG
,
Beattie
T
.
Can we abolish skull x rays for head injury?
Arch Dis Child
.
2005
;
90
(
8
):
859
864
[PubMed]
40
Rosati
SF
,
Maarouf
R
,
Wolfe
L
, et al
.
Implementation of pediatric cervical spine clearance guidelines at a combined trauma center: twelve-month impact.
J Trauma Acute Care Surg
.
2015
;
78
(
6
):
1117
1121
[PubMed]
41
Ruffing
RP
,
Trokel
M
,
Weiner
S
.
An interdisciplinary intervention to decrease abdominal CT use in children [abstract].
Acad Emerg Med
.
2012
;
19
(
4 suppl 1
):
S342
S343
42
Stylianos
S
.
Compliance with evidence-based guidelines in children with isolated spleen or liver injury: a prospective study.
J Pediatr Surg
.
2002
;
37
(
3
):
453
456
[PubMed]
43
Thompson
TM
,
Lovvorn
J
,
Lynch
A
,
Hunter
E
.
Reasons for repeat imaging in the pediatric emergency department (ED) and the impact of a state wide trauma image repository system.
J Investig Med
.
2013
;
61
(
2
):
454
44
Zamora-Flores
D
,
Busen
NH
,
Smout
R
,
Velasquez
O
.
Implementing a clinical practice guideline for the treatment of bronchiolitis in a high-risk Hispanic pediatric population.
J Pediatr Health Care
.
2015
;
29
(
2
):
169
180
[PubMed]
45
DeSomma
M
,
Divekar
A
,
Galloway
AC
,
Colvin
SB
,
Artman
M
,
Auslender
M
.
Impact of a clinical pathway on the postoperative care of children undergoing surgical closure of atrial septal defects.
Appl Nurs Res
.
2002
;
15
(
4
):
243
248
[PubMed]
46
Pageler
NM
,
Franzon
D
,
Longhurst
CA
, et al
.
Embedding time-limited laboratory orders within computerized provider order entry reduces laboratory utilization.
Pediatr Crit Care Med
.
2013
;
14
(
4
):
413
419
[PubMed]
47
Sinitsky
L
,
Shepherd
D
,
Cooshneea
M
,
Brierley
J
.
Reducing the number of unnecessary liver function tests requested on the paediatric intensive care unit.
Arch Dis Child
.
2016
;
101
(
suppl 1
):
A314
A315
48
Delgado-Corcoran
C
,
Bodily
S
,
Frank
DU
,
Witte
MK
,
Castillo
R
,
Bratton
SL
.
Reducing blood testing in pediatric patients after heart surgery: a quality improvement project.
Pediatr Crit Care Med
.
2014
;
15
(
8
):
756
761
[PubMed]
49
Tarrago
R
,
Nowak
JE
,
Leonard
CS
,
Payne
NR
.
Reductions in invasive device use and care costs after institution of a daily safety checklist in a pediatric critical care unit.
Jt Comm J Qual Patient Saf
.
2014
;
40
(
6
):
270
278
[PubMed]
50
Algaze
CA
,
Wood
M
,
Pageler
NM
,
Sharek
PJ
,
Longhurst
CA
,
Shin
AY
.
Use of a checklist and clinical decision support tool reduces laboratory use and improves cost.
Pediatrics
.
2016
;
137
(
1
):
e20143019
[PubMed]
51
Johnson
DP
,
Lind
C
,
Parker
SE
, et al
.
Toward high-value care: a quality improvement initiative to reduce unnecessary repeat complete blood counts and basic metabolic panels on a pediatric hospitalist service.
Hosp Pediatr
.
2016
;
6
(
1
):
1
8
[PubMed]
52
Cernich
JT
,
Hamilton
M
,
Mitre
N
.
Increased use of lab algorithm to reduce cost of screening for hypothyroidism in pediatric type 1 diabetes [abstract].
Diabetes
.
2014
;
63
(
Suppl 1
):
A322
A323
53
Price
MB
,
Jones
A
,
Hawkins
JA
,
McGough
EC
,
Lambert
L
,
Dean
JM
.
Critical pathways for postoperative care after simple congenital heart surgery.
Am J Manag Care
.
1999
;
5
(
2
):
185
192
[PubMed]
54
Shin
AY
,
Algaze
C
,
Wood
M
, et al
.
Impact of clinical resource management: laboratory optimization is associated with reduced cost and improved patient outcomes [abstract].
Pediatr Crit Care Med
.
2013
;
14
(
5 suppl 1
):
S108
55
Parikh
K
,
Hall
M
,
Blaschke
AJ
, et al
.
Aggregate and hospital-level impact of national guidelines on diagnostic resource utilization for children with pneumonia at children’s hospitals.
J Hosp Med
.
2016
;
11
(
5
):
317
323
[PubMed]
56
Polites
SF
,
Mohamed
MI
,
Habermann
EB
, et al
.
A simple algorithm reduces computed tomography use in the diagnosis of appendicitis in children.
Surgery
.
2014
;
156
(
2
):
448
454
[PubMed]
57
Warner
BW
,
Kulick
RM
,
Stoops
MM
,
Mehta
S
,
Stephan
M
,
Kotagal
UR
.
An evidenced-based clinical pathway for acute appendicitis decreases hospital duration and cost.
J Pediatr Surg
.
1998
;
33
(
9
):
1371
1375
[PubMed]
58
Wazeka
A
,
Valacer
DJ
,
Cooper
M
,
Caplan
DW
,
DiMaio
M
.
Impact of a pediatric asthma clinical pathway on hospital cost and length of stay.
Pediatr Pulmonol
.
2001
;
32
(
3
):
211
216
[PubMed]
59
McCulloh
RJ
,
Smitherman
SE
,
Koehn
KL
,
Alverson
BK
.
Assessing the impact of national guidelines on the management of children hospitalized for acute bronchiolitis.
Pediatr Pulmonol
.
2014
;
49
(
7
):
688
694
[PubMed]
60
Parikh
K
,
Hall
M
,
Teach
SJ
.
Bronchiolitis management before and after the AAP guidelines.
Pediatrics
.
2014
;
133
(
1
). Available at: www.pediatrics.org/cgi/content/full/133/1/e1
[PubMed]
61
Akenroye
AT
,
Baskin
MN
,
Samnaliev
M
,
Stack
AM
.
Impact of a bronchiolitis guideline on ED resource use and cost: a segmented time-series analysis.
Pediatrics
.
2014
;
133
(
1
). Available at: www.pediatrics.org/cgi/content/full/133/1/e227
[PubMed]
62
Shomaker
KL
,
Weiner
T
,
Esther
CR
 Jr
.
Impact of an evidence-based algorithm on quality of care in pediatric parapneumonic effusion and empyema.
Pediatr Pulmonol
.
2011
;
46
(
7
):
722
728
[PubMed]
63
Perlstein
PH
,
Kotagal
UR
,
Schoettker
PJ
, et al
.
Sustaining the implementation of an evidence-based guideline for bronchiolitis.
Arch Pediatr Adolesc Med
.
2000
;
154
(
10
):
1001
1007
[PubMed]
64
Fallon
SC
,
Delemos
D
,
Akinkuotu
A
,
Christopher
D
,
Naik-Mathuria
BJ
.
The use of an institutional pediatric abdominal trauma protocol improves resource use.
J Trauma Acute Care Surg
.
2016
;
80
(
1
):
57
63
[PubMed]
65
Guse
SE
,
Neuman
MI
,
O’Brien
M
, et al
.
Implementing a guideline to improve management of syncope in the emergency department.
Pediatrics
.
2014
;
134
(
5
). Available at: www.pediatrics.org/cgi/content/full/134/5/e1413
[PubMed]
66
Mohan
S
,
Nandi
D
,
Stephens
P
,
MʼFarrej
M
,
Vogel
RL
,
Bonafide
CP
.
Implementation of a clinical pathway for chest pain in a pediatric emergency department [published online ahead of print September 20, 2016].
Pediatr Emerg Care
. doi:
[PubMed]
67
Morris
M
,
Cleary
JP
,
Soliman
A
.
Small baby unit improves quality and outcomes in extremely low birth weight infants.
Pediatrics
.
2015
;
136
(
4
). Available at: www.pediatrics.org/cgi/content/full/136/4/e1007
[PubMed]
68
Raucci
U
,
Scateni
S
,
Tozzi
AE
, et al
.
The availability and the adherence to pediatric guidelines for the management of syncope in the emergency department.
J Pediatr
.
2014
;
165
(
5
):
967.e1
972.e1
[PubMed]
69
Fallon
SC
,
Delemos
D
,
Christopher
D
,
Frost
M
,
Wesson
DE
,
Naik-Mathuria
B
.
Trauma surgeon becomes consultant: evaluation of a protocol for management of intermediate-level trauma patients.
J Pediatr Surg
.
2014
;
49
(
1
):
178
182; discussion 182–183
[PubMed]
70
Gutierrez
IM
,
Zurakowski
D
,
Chen
Q
,
Mooney
DP
.
Clinical practice guidelines (CPGs) reduce costs in the management of isolated splenic injuries at pediatric trauma centers.
Langenbecks Arch Surg
.
2013
;
398
(
2
):
313
315
[PubMed]
71
Knott
EM
,
Gasior
AC
,
Ostlie
DJ
,
Holcomb
GW
 III
,
St Peter
SD
.
Decreased resource utilization since initiation of institutional clinical pathway for care of children with perforated appendicitis.
J Pediatr Surg
.
2013
;
48
(
6
):
1395
1398
[PubMed]
72
Leinwand
MJ
,
Atkinson
CC
,
Mooney
DP
.
Application of the APSA evidence-based guidelines for isolated liver or spleen injuries: a single institution experience.
J Pediatr Surg
.
2004
;
39
(
3
):
487
490
[PubMed]
73
Mittal
V
,
Hall
M
,
Morse
R
, et al
.
Impact of inpatient bronchiolitis clinical practice guideline implementation on testing and treatment.
J Pediatr
.
2014
;
165
(
3
):
570.e3
576.e3
[PubMed]
74
Ogershok
PR
,
Li
X
,
Palmer
HC
,
Moore
RS
,
Weisse
ME
,
Ferrari
ND
.
Restructuring an academic pediatric inpatient service using concepts developed by hospitalists.
Clin Pediatr (Phila)
.
2001
;
40
(
12
):
653
660; discussion 661–662
75
Singer
AJ
,
Garra
G
,
Thode
HC
 Jr
.
Changes in practice patterns with the opening of a dedicated pediatric emergency department.
Pediatr Emerg Care
.
2014
;
30
(
10
):
705
709
[PubMed]
76
Toomey
S
.
1222: implementation of bronchiolitis guidelines provide consistent quality of care.
Crit Care Med
.
2016
;
44
(
12
):
381
77
Fretheim
A
,
Zhang
F
,
Ross-Degnan
D
, et al
.
A reanalysis of cluster randomized trials showed interrupted time-series studies were valuable in health system evaluation.
J Clin Epidemiol
.
2015
;
68
(
3
):
324
333
[PubMed]
78
Naaktgeboren
C
,
Heus
P
,
van Dulmen
S
,
Weenink
JW
,
Kool
T
,
Hooft
L
.
Preventing overdiagnosis through the deimplementation of low-value diagnostic tests: a systematic review.
In:
4th International Conference on Preventing Overdiagnosis
;
September 20–22, 2016
;
Barcelona, Spain
79
National Quality Forum
. Improving Diagnostic Quality and Safety: Final Report. 2017. Available at: www.qualityforum.org/Publications/2017/09/Improving_Diagnostic_Quality_and_Safety_Final_Report.aspx

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

Supplementary data