Video Abstract

Video Abstract

Close modal
BACKGROUND AND OBJECTIVES:

Use of commercial direct-to-consumer (DTC) telemedicine outside of the pediatric medical home is increasing among children, and acute respiratory infections (ARIs) are the most commonly diagnosed condition at DTC telemedicine visits. Our objective was to compare the quality of antibiotic prescribing for ARIs among children across 3 settings: DTC telemedicine, urgent care, and the primary care provider (PCP) office.

METHODS:

In a retrospective cohort study using 2015–2016 claims data from a large national commercial health plan, we identified ARI visits by children (0–17 years old), excluding visits with comorbidities that could affect antibiotic decisions. Visits were matched on age, sex, chronic medical complexity, state, rurality, health plan type, and ARI diagnosis category. Within the matched sample, we compared the percentage of ARI visits with any antibiotic prescribing and the percentage of ARI visits with guideline-concordant antibiotic management.

RESULTS:

There were 4604 DTC telemedicine, 38 408 urgent care, and 485 201 PCP visits for ARIs in the matched sample. Antibiotic prescribing was higher for DTC telemedicine visits than for other settings (52% of DTC telemedicine visits versus 42% urgent care and 31% PCP visits; P < .001 for both comparisons). Guideline-concordant antibiotic management was lower at DTC telemedicine visits than at other settings (59% of DTC telemedicine visits versus 67% urgent care and 78% PCP visits; P < .001 for both comparisons).

CONCLUSIONS:

At DTC telemedicine visits, children with ARIs were more likely to receive antibiotics and less likely to receive guideline-concordant antibiotic management compared to children at PCP visits and urgent care visits.

What’s Known on This Subject:

Authors of previous studies identified concerns about antibiotic prescribing for adults using direct-to-consumer (DTC) telemedicine. No study has examined antibiotic prescribing during pediatric DTC telemedicine visits.

What This Study Adds:

Compared with urgent care and primary care visits, children receiving care for acute respiratory infections during DTC telemedicine visits are more likely to receive antibiotics and less likely to receive guideline-concordant antibiotic management.

Many commercial companies offer direct-to-consumer (DTC) telemedicine visits in which patients are connected to physicians outside of the medical home through audio-only or audio-video conferencing on their personal devices.1 DTC telemedicine companies highlight the convenience of this care and report high patient satisfaction.2,4 Access to telemedicine has expanded, with 96% of large business insurance plans now offering coverage for employees and their families.5 As a result, millions of children now have access to DTC telemedicine, and use is rapidly increasing.6 

The American Academy of Pediatrics discourages the use of DTC telemedicine outside of the medical home for acute pediatric care, citing concerns such as limited physical examination capabilities, lack of ongoing patient-provider relationships, and lack of access to patient records, all of which may lead to lower-quality care.7 The American Academy of Pediatrics further notes that these issues may be particularly important with children who may have difficulty verbalizing their symptoms.7 The American Telemedicine Association echoed many of these concerns,8 adding that the use of personal devices may also impact technical quality and privacy of care. The American Telemedicine Association further suggests that DTC telemedicine outside of the medical home should not be used for children <2 years old.8 Both organizations emphasize the need for telemedicine visits to adhere to the same standards expected of in-person care, including adherence to professional guidelines for appropriate examination, diagnostic testing, and treatment.7,8 

To date, there is limited evidence regarding the quality of care during pediatric DTC telemedicine visits. Early studies of adult patients using DTC telemedicine compared with physician office visits have identified more inappropriate antibiotic prescriptions for bronchitis9 and more broad-spectrum antibiotic use.10 The quality of DTC telemedicine care for children may differ for several reasons. First, there is heightened reliance on the physical examination among children who cannot fully articulate symptoms.7 Second, care requires knowledge of pediatric-specific guidelines.11,13 Third, rates of guideline-concordant antibiotic management at physician visits is higher for children than for adults14; therefore, the relative difference in quality of care at DTC telemedicine visits may be different.

To fill this gap in knowledge about the quality of commercial DTC telemedicine care for children, we used data from a large national health plan to compare the quality of antibiotic prescribing for acute respiratory infections (ARIs) at pediatric DTC telemedicine, urgent care, and primary care provider (PCP) visits. We focus on ARIs because authors of previous studies identified concerns about antibiotic prescribing during pediatric ARI visits in other settings,14,19 and because ARIs account for the majority of DTC telemedicine visits.6 

We performed a retrospective cohort study using 2015–2016 claims data from a large national commercial health plan, which provides coverage for roughly 4 million children annually from all 9 census divisions and contracts with a single national DTC telemedicine vendor. Children age 0 to 17 years were included, with no minimum enrollment duration requirement. We excluded children who lacked pharmaceutical coverage during the month of their visit to ensure record of pharmacy claims.

ARI visits were identified by using previously published methods on the basis of International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes.15,20,21 Consistent with previous work,14,15,20,21 ARI diagnoses included diagnoses for which antibiotics may be warranted (sinusitis [461.x], pneumonia [481.x–483.x, 485.x–486.x], streptococcal pharyngitis [034.x], acute otitis media [382.x]) and diagnoses for which antibiotics are not warranted (eg, viral upper respiratory infection [URI], bronchiolitis, viral pharyngitis, serous otitis media [381.x, 460.x, 462–466.x, 480.x, 490.x]). We excluded bacterial pneumonia visits because of the small number of DTC telemedicine visits identified (n = 11). We converted International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM) codes to ICD-9-CM codes using a modified crosswalk created by the Centers for Medicare and Medicaid Services (see Supplemental Table 5).22 

To examine the first ARI visit within an episode of illness, we identified “index” ARI visits with no ARI visits at any location in the previous 21 days, which is consistent with previous studies.10,23 Subsequent ARI visits within 21 days of an index visit were considered follow-up care. We excluded episodes as detailed in Supplemental Fig 2. Using a previously published methodology, we excluded episodes for which the index visit included a diagnosis indicating chronic conditions (eg, history of transplant) or additional codiagnoses (eg, urinary tract infection) that could impact antibiotic prescribing decisions (n = 60 947; 4.7%) (Supplemental Table 6).15,21 These exclusion criteria were originally developed for adult populations. Although not all of these exclusion criteria are reasons to deviate from pediatric ARI guidelines, we applied them to study a set of visits consistent with other studies.

ARI episodes were categorized as DTC telemedicine, urgent care, or PCP on the basis of the index visit setting. We identified setting using place of service codes, Current Procedural Terminology codes, insurer provider specialty codes, and organizational national provider identifiers and tax identification numbers for urgent care and DTC telemedicine companies through previously described methods (additional details in Supplemental Information).24 ARI episodes that began with index visits to settings other than DTC telemedicine, urgent care, or PCP offices (eg, emergency department, retail-based clinics) were not included.

Our 2 primary outcomes, compared across settings, were (1) the proportion of visits that resulted in an antibiotic prescription and (2) the proportion of visits that resulted in antibiotic management that was concordant with guidelines for the visit diagnosis.

We identified all filled prescriptions on the day of the index visit and subsequent 2 days and assigned these to the index visit, consistent with previous methods.1,10,25 We included all systemic antibiotics (ie, oral, intramuscular) and excluded topical, ophthalmic, or otic antibiotics. We determined guideline concordance of antibiotics with visit diagnosis, assigning each visit to 1 of 5 categories (Supplemental Table 7).21,26 “Antibiotic prescribed” visits included visits categorized as guideline-concordant antibiotic use, guideline–non-concordant antibiotic use, and unnecessary antibiotic use. “Guideline-concordant antibiotic management” included visits categorized as guideline-concordant antibiotic use for sinusitis, otitis media, or streptococcal pharyngitis (ie, amoxicillin or amoxicillin-clavulanic acid for sinusitis,12 amoxicillin or amoxicillin-clavulanic acid for otitis media,11 and penicillin or amoxicillin for streptococcal pharyngitis27), no antibiotic use for sinusitis or otitis media (also guideline-concordant management for these diagnoses11,12), or guideline-concordant nonuse of antibiotics for diagnoses that do not warrant antibiotics (eg, bronchiolitis13).

As secondary outcomes, we compared the occurrence of streptococcal testing within 1 day for children diagnosed with streptococcal pharyngitis27 as well as return visits within 2 and 21 days of an index visit to any setting (eg, PCP, DTC telemedicine, urgent care, retail clinic, emergency department). We did not have streptococcal testing results.

Patient age, sex, and high-deductible health plan (HDHP) status were determined from insurance enrollment files. Using the patients’ zip code, we determined census region and rural-urban commuting area codes.28 We determined patient chronic medical complexity using the Pediatric Medical Complexity Algorithm Version 2 applied to 2014–2016 inpatient and outpatient claims.29 This algorithm is used to identify children with 3 levels of chronic medical complexity: complex chronic disease, noncomplex chronic disease, and no chronic disease.29 

To address differences in patients and conditions managed in the 3 settings, we used coarsened-exact matching,30 which prunes observations to achieve covariate balance between groups. Instead of matching to a fixed number of visits, coarsened-exact matching matches DTC telemedicine visits to many matched urgent care visits and PCP visits and then weights each stratum or matched set.30 Specifically, we matched DTC telemedicine visits to urgent care visits and separately to PCP visits on the basis of age (in years), sex, chronic medical complexity, state, rural or urban area, HDHP status, and diagnosis category (eg, otitis media, viral URI).

Across ARI visits for the 3 settings of care, we described patient characteristics and diagnoses. In the unmatched sample and the matched weighted sample, we examined antibiotic prescribing and guideline-concordant antibiotic management for DTC telemedicine visits compared with urgent care and compared with PCP offices using χ2 tests. We include both unmatched and matched results because of potential subjectivity in diagnoses, which may then drive variability in antibiotic prescribing.16 In the matched weighted sample, we also compared streptococcal testing and return visits by setting using χ2 tests. We adjusted for multiple comparisons in our main results using the Benjamini-Hochberg false discovery rate,31 yielding a P value of .0342.

To examine whether differences in antibiotic prescribing varied by patient characteristics, we performed further analysis that was focused on 2 variables that may impact guideline adherence: child age and chronic medical conditions. Within each category of child age and medical complexity, we examined antibiotic testing and guideline concordance for DTC telemedicine compared with urgent care and PCP offices using χ2 tests and then tested the significance of an interaction term between age or medical complexity and visit setting using logistic regression.

We performed 2 sensitivity analyses to examine the robustness of our findings to specific assumptions. First, we conducted a sensitivity analysis in which we redefined guideline-concordant antibiotic management so that it no longer included “no antibiotic use” for sinusitis and otitis. Because guidelines allow for observation without antibiotic use in specific circumstances for sinusitis and otitis media.11,12 we considered no antibiotic use to be a guideline-concordant management option for these diagnoses in our main analysis but not in this sensitivity analysis. Second, we examined antibiotic prescribing and guideline-concordant management after excluding ARI visits with a follow-up visit within 2 days because visits with such early follow-up might result in miscategorization of the site of antibiotic prescribing.

Analyses were conducted in SAS version 9.4 (SAS Institute, Inc, Cary, NC). Deidentified data were used in this analysis, which was determined to be exempt by Harvard Medical School’s Institutional Review Board.

In the unmatched sample, there were 5431 index ARI visits to DTC telemedicine, 87 555 to urgent care, and 1 132 116 to PCPs in 2015–2016 that met inclusion criteria. Compared with urgent care and PCP visits, DTC telemedicine visits were more likely to involve children who were older, enrolled in HDHPs, and in the South (Table 1). Among DTC telemedicine visits, 5% were for children <2 years old. At DTC telemedicine visits, children were more likely to receive a diagnosis of viral ARI (68% vs 54% urgent care; 59% PCP visits) or sinusitis (18% vs 8% urgent care; 10% PCP visits) and less likely to receive a diagnosis of otitis media (8% vs 26% urgent care; 23% PCP).

TABLE 1

Characteristics of ARI Visits Before and After Matching, 2015–2016

Pediatric Visits
Before MatchingAfter Matchinga
DTC Telemedicine, No. (%)Urgent Care, No. (%)Standardized Difference,b %PCP, No. (%)Standardized Difference,b %DTC Telemedicine, No. (%)Urgent Care, No. (%)Standardized Difference,b %PCP, No. (%)Standardized Difference,b %
Visits, N 5431 87 555 — 1 132 116 — 4604 38 408 — 485 201 — 
Child age, y           
 0–1 245 (5) 8234 (9) 59.0 195 692 (17) 89.6 224 (5) 1956 (5) 4.7 24 192 (5) 2.5 
 2–5 1305 (24) 24 845 (28) 21.5 390 699 (35) 46.4 1132 (25) 9528 (25) 1.2 121 932 (25) 2.9 
 6–11 1930 (36) 30 018 (34) 5.6 331 716 (29) 30.1 1614 (35) 13 392 (35) 0.8 168 036 (35) 1.9 
 12–17 1951 (36) 24 458 (28) 39.3 214 009 (19) 110.7 1634 (35) 13 532 (35) 1.1 171 041 (35) 1.0 
Sex           
 Female 2755 (51) 44 181 (50) 1.1 555 277 (49) 6.7 2339 (51) 19 513 (51) 246 500 (51) 
 Male 2676 (49) 43 374 (50) — 576 839 (51) — 2265 (49) 18 895 (49) — 238 701 (49) — 
Chronic medical complexityc           
 No chronic conditions 3993 (74) 65 931 (75) 9.5 835 897 (74) 1.6 3542 (77) 29 548 (77) 373 280 (77) 
 Noncomplex chronic condition 1062 (20) 14 823 (17) 18.5 197 934 (17) 14.3 803 (17) 6699 (17) 84 626 (17) 
 Complex chronic condition 376 (7) 6801 (8) 11.9 98 285 (9) 22.2 259 (6) 2161 (6) 27 295 (6) 
Rural or urband           
 Metropolitan 4967 (91) 81 028 (93) 15.7 1 052 233 (93) 22.7 4415 (96) 36 839 (96) 0.5 465 266 (96) 0.1 
 Micropolitan 284 (5) 4421 (5) 3.7 48 849 (4) 22.1 140 (3) 1161 (3) 0.6 14 771 (3) 0.1 
 Small town 109 (2) 1449 (2) 21.4 20 540 (2) 10.8 34 (1) 284 (1) 3583 (1) 
 Rural 71 (1) 657 (1) 71.7 10 494 (1) 41.3 15 (0) 125 (0) 1581 (0) 
Region           
 Northeast 627 (12) 5691 (7) 79.7 346 792 (31) 90.0 490 (11) 4088 (11) 51 640 (11) 
 Midwest 929 (17) 16 393 (19) 10.7 161 878 (14) 22.9 701 (15) 5833 (15) 0.3 73 876 (15) 
 South 2943 (54) 42 860 (49) 21.0 445 160 (39) 62.3 2587 (56) 21 757 (57) 1.9 275 657 (57) 2.5 
 West 932 (17) 22 611 (26) 45.8 178 286 (16) 10.6 826 (18) 6730 (18) 2.9 84 029 (17) 4.3 
HDHP           
 No 4575 (84) 81 720 (93) 136.2 1 062 214 (94) 164.3 4119 (89) 34 362 (89) 434 088 (89) 
 Yes 856 (16) 5835 (7) — 69 902 (6) — 485 (11) 4046 (11) — 51 113 (11) — 
Diagnosis categorye           
 Sinusitis 966 (18) 7101 (8) 122.1 113 392 (10) 85.9 646 (14) 5389 (14) 68 080 (14) 
 Streptococcal pharyngitis 326 (6) 11 745 (13) 65.6 103 066 (9) 37.5 250 (5) 2086 (5) 26 347 (5) 
 Otitis media 429 (8) 22 838 (26) 96.9 263 743 (23) 86.4 369 (8) 3078 (8) 38 888 (8) 
 Viral ARIf 3710 (68) 46 979 (54) 59.1 664 912 (59) 39.5 3339 (73) 27 855 (73) 351 887 (73) 
Pediatric Visits
Before MatchingAfter Matchinga
DTC Telemedicine, No. (%)Urgent Care, No. (%)Standardized Difference,b %PCP, No. (%)Standardized Difference,b %DTC Telemedicine, No. (%)Urgent Care, No. (%)Standardized Difference,b %PCP, No. (%)Standardized Difference,b %
Visits, N 5431 87 555 — 1 132 116 — 4604 38 408 — 485 201 — 
Child age, y           
 0–1 245 (5) 8234 (9) 59.0 195 692 (17) 89.6 224 (5) 1956 (5) 4.7 24 192 (5) 2.5 
 2–5 1305 (24) 24 845 (28) 21.5 390 699 (35) 46.4 1132 (25) 9528 (25) 1.2 121 932 (25) 2.9 
 6–11 1930 (36) 30 018 (34) 5.6 331 716 (29) 30.1 1614 (35) 13 392 (35) 0.8 168 036 (35) 1.9 
 12–17 1951 (36) 24 458 (28) 39.3 214 009 (19) 110.7 1634 (35) 13 532 (35) 1.1 171 041 (35) 1.0 
Sex           
 Female 2755 (51) 44 181 (50) 1.1 555 277 (49) 6.7 2339 (51) 19 513 (51) 246 500 (51) 
 Male 2676 (49) 43 374 (50) — 576 839 (51) — 2265 (49) 18 895 (49) — 238 701 (49) — 
Chronic medical complexityc           
 No chronic conditions 3993 (74) 65 931 (75) 9.5 835 897 (74) 1.6 3542 (77) 29 548 (77) 373 280 (77) 
 Noncomplex chronic condition 1062 (20) 14 823 (17) 18.5 197 934 (17) 14.3 803 (17) 6699 (17) 84 626 (17) 
 Complex chronic condition 376 (7) 6801 (8) 11.9 98 285 (9) 22.2 259 (6) 2161 (6) 27 295 (6) 
Rural or urband           
 Metropolitan 4967 (91) 81 028 (93) 15.7 1 052 233 (93) 22.7 4415 (96) 36 839 (96) 0.5 465 266 (96) 0.1 
 Micropolitan 284 (5) 4421 (5) 3.7 48 849 (4) 22.1 140 (3) 1161 (3) 0.6 14 771 (3) 0.1 
 Small town 109 (2) 1449 (2) 21.4 20 540 (2) 10.8 34 (1) 284 (1) 3583 (1) 
 Rural 71 (1) 657 (1) 71.7 10 494 (1) 41.3 15 (0) 125 (0) 1581 (0) 
Region           
 Northeast 627 (12) 5691 (7) 79.7 346 792 (31) 90.0 490 (11) 4088 (11) 51 640 (11) 
 Midwest 929 (17) 16 393 (19) 10.7 161 878 (14) 22.9 701 (15) 5833 (15) 0.3 73 876 (15) 
 South 2943 (54) 42 860 (49) 21.0 445 160 (39) 62.3 2587 (56) 21 757 (57) 1.9 275 657 (57) 2.5 
 West 932 (17) 22 611 (26) 45.8 178 286 (16) 10.6 826 (18) 6730 (18) 2.9 84 029 (17) 4.3 
HDHP           
 No 4575 (84) 81 720 (93) 136.2 1 062 214 (94) 164.3 4119 (89) 34 362 (89) 434 088 (89) 
 Yes 856 (16) 5835 (7) — 69 902 (6) — 485 (11) 4046 (11) — 51 113 (11) — 
Diagnosis categorye           
 Sinusitis 966 (18) 7101 (8) 122.1 113 392 (10) 85.9 646 (14) 5389 (14) 68 080 (14) 
 Streptococcal pharyngitis 326 (6) 11 745 (13) 65.6 103 066 (9) 37.5 250 (5) 2086 (5) 26 347 (5) 
 Otitis media 429 (8) 22 838 (26) 96.9 263 743 (23) 86.4 369 (8) 3078 (8) 38 888 (8) 
 Viral ARIf 3710 (68) 46 979 (54) 59.1 664 912 (59) 39.5 3339 (73) 27 855 (73) 351 887 (73) 

—, not applicable.

a

Sample characteristics before and after matching. In comparison of matched samples, each stratum of matched sets is weighted to account for differences in the number of matched urgent care and PCP visits. Counts within the matched categories may exceed column total because of rounding of weighted frequencies. Visits were matched by age (in years) and state, but results are reported in the table by larger categories of age group and census region.

b

Standardized differences comparing the characteristics of urgent care or PCP visits (in the column to the left) to DTC telemedicine, before and after matching.

c

Patient chronic medical complexity was determined by using the Pediatric Medical Complexity Algorithm applied to all 2014–2016 inpatient and outpatient claims for each patient.29 This algorithm is used to identify children with 3 levels of chronic medical complexity: complex chronic disease, (ie, significant multisystem chronic disease, progressive conditions with decreased life expectancy, technology dependence, or malignancy), noncomplex chronic disease (ie, chronic conditions such as asthma that do not meet the above criteria), and no chronic disease.

d

Rural or urban status was determined by using the US Department of Agriculture rural-urban commuting area codes.

e

Because a small percentage of visits have >1 of these diagnoses (1.1%), percentages for diagnosis category may sum to >100%.

f

Viral ARI category consists of diagnoses for which antibiotics are not warranted (eg, viral URI, bronchiolitis, viral pharyngitis, and serous otitis media) in the absence of another diagnosis.

In the unmatched cohort (Table 2), DTC telemedicine visits were associated with substantially increased antibiotic prescribing (52% vs 46% urgent care; 33% PCP; P < .001 for both) and decreased guideline-concordant antibiotic management compared with other settings (60% vs 71% urgent care; 80% PCP; P < .001 for both).

TABLE 2

Antibiotic Prescribing and Guideline-Concordant Antibiotic Management in Matched Sample, 2015–2016

Unmatched VisitsMatched Visits
DTC TelemedicineUrgent CarePaPCPPbDTC TelemedicineUrgent CarePaPCPPb
Total ARI visits, N 5431 87 555 — 1 132 116 — 4604 38 408 — 485 201 — 
Any antibiotics prescribed,cn (%)d 2846 (52) 40 594 (46) <.001 376 638 (33) <.001 2381 (52) 16 198 (42) <.001 152 687 (31) <.001 
Guideline-concordant management,en (%)d 3246 (60) 61 842 (71) <.001 907 450 (80) <.001 2712 (59) 25 788 (67) <.001 376 259 (78) <.001 
Specifics category of guideline concordance and nonconcordance among ARI visits, n (%)d           
 Guideline-concordant use 761 (14) 18 613 (21) <.001 189 970 (17) <.001 561 (12) 4260 (11) .026 52 776 (11) .005 
 Guideline nonconcordant use 369 (7) 7514 (9) <.001 83 297 (7) .11 271 (6) 2507 (7) .095 31 138 (6) .14 
 Unnecessary antibiotic use 1716 (32) 14 467 (17) <.001 103 371 (9) <.001 1549 (34) 9431 (25) <.001 68 773 (14) <.001 
 No antibiotic use 591 (11) 14 449 (17) <.001 193 937 (17) <.001 433 (9) 3787 (10) .33 49 401 (10) .083 
 Guideline-concordant nonuse 1994 (37) 32 512 (37) .54 561 541 (50) <.001 1790 (39) 18 424 (48) <.001 283 114 (58) <.001 
Unmatched VisitsMatched Visits
DTC TelemedicineUrgent CarePaPCPPbDTC TelemedicineUrgent CarePaPCPPb
Total ARI visits, N 5431 87 555 — 1 132 116 — 4604 38 408 — 485 201 — 
Any antibiotics prescribed,cn (%)d 2846 (52) 40 594 (46) <.001 376 638 (33) <.001 2381 (52) 16 198 (42) <.001 152 687 (31) <.001 
Guideline-concordant management,en (%)d 3246 (60) 61 842 (71) <.001 907 450 (80) <.001 2712 (59) 25 788 (67) <.001 376 259 (78) <.001 
Specifics category of guideline concordance and nonconcordance among ARI visits, n (%)d           
 Guideline-concordant use 761 (14) 18 613 (21) <.001 189 970 (17) <.001 561 (12) 4260 (11) .026 52 776 (11) .005 
 Guideline nonconcordant use 369 (7) 7514 (9) <.001 83 297 (7) .11 271 (6) 2507 (7) .095 31 138 (6) .14 
 Unnecessary antibiotic use 1716 (32) 14 467 (17) <.001 103 371 (9) <.001 1549 (34) 9431 (25) <.001 68 773 (14) <.001 
 No antibiotic use 591 (11) 14 449 (17) <.001 193 937 (17) <.001 433 (9) 3787 (10) .33 49 401 (10) .083 
 Guideline-concordant nonuse 1994 (37) 32 512 (37) .54 561 541 (50) <.001 1790 (39) 18 424 (48) <.001 283 114 (58) <.001 

—, not applicable.

a

P value determined from χ2 test for DTC telemedicine versus urgent care visits.

b

P value determined from χ2 test for DTC telemedicine versus PCP visits.

c

Visits with antibiotic prescribing include those categorized as guideline-concordant use, guideline nonconcordant use, and unnecessary antibiotic use.

d

Percentages reported are the proportion of visits in each setting meeting the specified criteria divided by the total number of ARI visits at that setting (indicated by the top row).

e

Visits with guideline-concordant antibiotic management include those categorized as guideline-concordant antibiotic use, no antibiotic use (for acute otitis media and sinusitis), and guideline-concordant nonuse of antibiotics.

After matching, the weighted sample included 4604 DTC telemedicine, 38 408 urgent care, and 485 201 PCP visits with improved balance on observed characteristics, including child age, geographic region, and visit diagnosis (Table 1, Supplemental Table 8).

At DTC telemedicine visits, children were more likely to receive an antibiotic compared with matched visits at other settings (52% vs 42% urgent care; 31% PCP; P < .001 for both comparisons; Table 2). At DTC telemedicine visits, children were less likely to receive guideline-concordant antibiotic management (59% vs 67% urgent care; 78% PCP; P < .001 for both comparisons). The difference in guideline-concordant antibiotic management rates was primarily driven by antibiotic prescribing for visits with viral ARI diagnoses that do not warrant antibiotics. For such visits, antibiotics were appropriately not prescribed in only 54% of DTC telemedicine visits compared with 66% of urgent care visits (P < .001) and 80% of PCP visits (P < .001; Table 3).

TABLE 3

Antibiotic Prescribing Outcomes by Diagnosis

Pediatric Visits
DTC TelemedicineUrgent CarePaPCPPb
Total ARI Visits, N 4604 38 408 — 485 201 — 
Summative categories      
 Visits for which antibiotics may be indicated, n 1265 10 553 — 133 314 — 
  Any antibiotic use, n (%) 832 (66) 6766 (64) .25 83 914 (63) .038 
  Guideline-concordant antibiotic management,cn (%) 922 (73) 7364 (70) .023 93 145 (70) .020 
  Sinusitis visits, n 646 5389 — 68 080 — 
   Any antibiotic use, n (%) 410 (63) 3415 (63) .96 43 219 (63) .99 
   Guideline-concordant antibiotic management, n (%) 471 (73) 3837 (71) .36 49 611 (73) .98 
  Streptococcal pharyngitis visits, n 250 2086 — 26 347 — 
   Any antibiotic use, n (%) 178 (71) 1403 (67) .21 17 315 (66) .069 
   Guideline-concordant antibiotic management, n (%) 136 (54) 974 (47) .021 11 424 (43) <.001 
  Otitis media visits, n 369 3078 — 38 888 — 
   Any antibiotic use, n (%) 244 (66) 1948 (63) .28 23 380 (60) .019 
   Guideline-concordant antibiotic management, n (%) 315 (85) 2553 (83) .24 32 109 (83) .16 
 Visits where antibiotics not indicated, n 3339 27 855 — 351 887 — 
  Guideline-concordant antibiotic management, n (%) 1790 (54) 18 424 (66) <.001 283 114 (80) <.001 
  Bronchiolitis visits, n 51 263 — 4037 — 
   Guideline-concordant antibiotic management, n (%) 31 (61) 182 (69) .25 3197 (79) .001 
  Bronchitis visits, n 363 2686 — 27 308 — 
   Guideline-concordant antibiotic management, n (%) 160 (44) 1114 (41) .344 11 270 (41) .28 
  Visits for other viral URIs, n 2925 24 906 — 320 542 — 
   Guideline-concordant antibiotic management, n (%) 1599 (55) 17 128 (69) <.001 268 647 (84) <.001 
 Visits where antibiotics prescribed, n 2381 16 198 — 152 687 — 
  Guideline-concordant antibiotic management, n (%) 561 (24) 4260 (26) .005 52 776 (35) <.001 
Pediatric Visits
DTC TelemedicineUrgent CarePaPCPPb
Total ARI Visits, N 4604 38 408 — 485 201 — 
Summative categories      
 Visits for which antibiotics may be indicated, n 1265 10 553 — 133 314 — 
  Any antibiotic use, n (%) 832 (66) 6766 (64) .25 83 914 (63) .038 
  Guideline-concordant antibiotic management,cn (%) 922 (73) 7364 (70) .023 93 145 (70) .020 
  Sinusitis visits, n 646 5389 — 68 080 — 
   Any antibiotic use, n (%) 410 (63) 3415 (63) .96 43 219 (63) .99 
   Guideline-concordant antibiotic management, n (%) 471 (73) 3837 (71) .36 49 611 (73) .98 
  Streptococcal pharyngitis visits, n 250 2086 — 26 347 — 
   Any antibiotic use, n (%) 178 (71) 1403 (67) .21 17 315 (66) .069 
   Guideline-concordant antibiotic management, n (%) 136 (54) 974 (47) .021 11 424 (43) <.001 
  Otitis media visits, n 369 3078 — 38 888 — 
   Any antibiotic use, n (%) 244 (66) 1948 (63) .28 23 380 (60) .019 
   Guideline-concordant antibiotic management, n (%) 315 (85) 2553 (83) .24 32 109 (83) .16 
 Visits where antibiotics not indicated, n 3339 27 855 — 351 887 — 
  Guideline-concordant antibiotic management, n (%) 1790 (54) 18 424 (66) <.001 283 114 (80) <.001 
  Bronchiolitis visits, n 51 263 — 4037 — 
   Guideline-concordant antibiotic management, n (%) 31 (61) 182 (69) .25 3197 (79) .001 
  Bronchitis visits, n 363 2686 — 27 308 — 
   Guideline-concordant antibiotic management, n (%) 160 (44) 1114 (41) .344 11 270 (41) .28 
  Visits for other viral URIs, n 2925 24 906 — 320 542 — 
   Guideline-concordant antibiotic management, n (%) 1599 (55) 17 128 (69) <.001 268 647 (84) <.001 
 Visits where antibiotics prescribed, n 2381 16 198 — 152 687 — 
  Guideline-concordant antibiotic management, n (%) 561 (24) 4260 (26) .005 52 776 (35) <.001 

—, not applicable.

a

P value determined from χ2 test for DTC telemedicine versus urgent care visits.

b

P value determined from χ2 test for DTC telemedicine versus PCP visits.

c

Visits with guideline-concordant antibiotic management include those categorized as guideline-concordant antibiotic use, no antibiotic use (for acute otitis media and sinusitis visits), and guideline-concordant nonuse of antibiotics.

In the matched sample, among visits with a diagnosis of streptococcal pharyngitis, streptococcal testing was performed for 4% of DTC telemedicine visits compared with 75% of urgent care and 68% of PCP visits (P < .001 for both; Table 4). Subsequent visits within 2 days occurred for a slightly higher percentage of DTC telemedicine visits (5% vs 2% urgent care; 1% PCP; P < .001 for both).

TABLE 4

Streptococcal Testing and Return Visits in Matched Sample, 2015–2016

Pediatric Visits
DTC TelemedicineUrgent CarePaPCPPb
Visits with diagnosis of streptococcal pharyngitis, N 250 2086  26 347  
 Streptococcal testing performed, n (%) 9 (4) 1557 (75) <.001 17 818 (68) <.001 
Total ARI visits, N 4604 38 408 — 485 201 — 
 Return ARI visit within 2 d, n (%) 226 (5) 869 (2) <.001 5875 (1) <.001 
 Return ARI visit within 21 d, n (%) 525 (11) 3719 (10) <.001 45 629 (9) <.001 
Pediatric Visits
DTC TelemedicineUrgent CarePaPCPPb
Visits with diagnosis of streptococcal pharyngitis, N 250 2086  26 347  
 Streptococcal testing performed, n (%) 9 (4) 1557 (75) <.001 17 818 (68) <.001 
Total ARI visits, N 4604 38 408 — 485 201 — 
 Return ARI visit within 2 d, n (%) 226 (5) 869 (2) <.001 5875 (1) <.001 
 Return ARI visit within 21 d, n (%) 525 (11) 3719 (10) <.001 45 629 (9) <.001 

—, not applicable.

a

P value determined from χ2 test for DTC telemedicine versus urgent care visits.

b

P value determined from χ2 test for DTC telemedicine versus PCP visits.

Compared with urgent care and PCP visits, DTC telemedicine visits had significantly increased antibiotic prescribing and decreased guideline-concordant antibiotic management within each category of age and chronic medical complexity (Fig 1). The 1 exception was that DTC telemedicine and urgent care results were not statistically different for the youngest children (0–1 year old), for which the sample size was small. Interaction terms indicated that the degree of difference between DTC telemedicine and PCP visits varied significantly with age for antibiotics prescribed (interaction term P < .001) and guideline concordance (interaction term P = .004); other interaction terms were not significant.

FIGURE 1

Antibiotic prescribing and guideline-concordant antibiotic management for DTC telemedicine compared with urgent care (UC) and PCP visits, stratified by child age and chronic medical complexity. A, Antibiotic prescribing associated with DTC telemedicine visits compared with UC and PCP visits, stratified by 4 levels of child age (left) and 3 levels of chronic medical complexity (right). B, Guideline-concordant antibiotic management associated with DTC telemedicine visits compared with UC and PCP visits, stratified by 4 levels of child age (left) and 3 levels of chronic medical complexity (right). Within each age or medical complexity category, the values for UC and PCP visits differed significantly from DTC telemedicine with 1 exception: DTC telemedicine and UC results were not statistically different for the youngest children (0–1 year old). Interaction terms indicated that the degree of difference between DTC telemedicine and PCP visits varied significantly with age for antibiotics prescribed (interaction term P < .001) and guideline concordance (interaction term P = .004); other interaction terms were not significant.

FIGURE 1

Antibiotic prescribing and guideline-concordant antibiotic management for DTC telemedicine compared with urgent care (UC) and PCP visits, stratified by child age and chronic medical complexity. A, Antibiotic prescribing associated with DTC telemedicine visits compared with UC and PCP visits, stratified by 4 levels of child age (left) and 3 levels of chronic medical complexity (right). B, Guideline-concordant antibiotic management associated with DTC telemedicine visits compared with UC and PCP visits, stratified by 4 levels of child age (left) and 3 levels of chronic medical complexity (right). Within each age or medical complexity category, the values for UC and PCP visits differed significantly from DTC telemedicine with 1 exception: DTC telemedicine and UC results were not statistically different for the youngest children (0–1 year old). Interaction terms indicated that the degree of difference between DTC telemedicine and PCP visits varied significantly with age for antibiotics prescribed (interaction term P < .001) and guideline concordance (interaction term P = .004); other interaction terms were not significant.

Close modal

When no longer classifying “no antibiotic use” as guideline-concordant for otitis media and sinusitis, rates of guideline-concordant antibiotic management remained lower overall at DTC telemedicine (51% vs 59% urgent care; 69% PCP; P < .001 for both; Supplemental Table 9). Analysis excluding index visits with follow-up visits within 2 days did not differ meaningfully from main results.

Children treated for ARIs at commercial DTC telemedicine visits were substantially more likely to receive an antibiotic and less likely to receive guideline-concordant antibiotic management than children presenting to urgent care or primary care. These quality differences were observed in metrics agnostic to visit diagnosis (antibiotic prescribing rate) and conditional on diagnosis (guideline-concordant antibiotic management). These differences were primarily driven by increased use of antibiotics for visits receiving viral diagnoses.

These differences in antibiotic prescribing for children contrast with previous studies of DTC telemedicine quality among adult patients in which quality differences have been smaller or nonexistent.9,10 For example, using data from the same health plan, we observed similar antibiotic management for adults seeking care for ARIs in these 3 settings.32 Professional groups such as the American Academy of Pediatrics and the American Telemedicine Association have previously raised concerns about DTC telemedicine care outside of the medical home for pediatric patients.7,8 Our results support these concerns and underscore the importance of pediatric-specific evaluation and guidelines. Targeted improvement initiatives have the potential to substantially improve the quality of care delivered via DTC telemedicine.33 Pediatric-specific guidelines and metrics should be incorporated into such efforts.

Several mechanisms could contribute to increased antibiotic prescribing and decreased guideline concordance in pediatric DTC telemedicine visits. First, the potentially rich information that telemedicine can provide in other models of telemedicine, such as when the patient is at a facility designed for telemedicine consultation, is relatively limited in DTC telemedicine. Specifically, the information transmitted in DTC telemedicine is limited in that personal devices used by parents for DTC telemedicine visits rarely incorporate peripheral attachments designed to enhance remote visits (eg, tele-otoscopy) and also do not incorporate trained telepresenters (ie, nurses or medical assistants with the patient). The limited availability of otoscopy and streptococcal testing in DTC telemedicine may contribute to the different diagnosis mix at DTC telemedicine (increased viral ARIs and sinusitis and decreased otitis media and streptococcal pharyngitis) compared with in-person settings. Additionally, personal devices may vary in the quality of microphones, cameras, and Wi-Fi, and some DTC telemedicine visits are completed by using only telephone. These issues reduce the data available to the treating DTC telemedicine provider, which may be of greater concern in pediatric care because of the more limited ability of children to communicate symptoms. Second, there may be differential expectations for antibiotics among children and parents who use DTC telemedicine versus in-person care. Although we matched on a number of observed variables, parental expectations may still differ across settings in ways that we were unable to address. Third, DTC telemedicine visits outside of the medical home lack 3 types of continuity: informational (lack of medical records), relationship (lack of ongoing relationship between the provider and patient), and clinical management (lack of opportunity to manage the patient’s care over time),34 which has the potential to impact quality of care. However, this is unlikely to be the only explanation given that authors of previous evaluations of retail-based clinics, which are also a model of in-person care with limited continuity, have found comparable quality to physician office visits.15,19 Fourth, the majority of commercial DTC telemedicine visits are for adults, such that DTC telemedicine providers may have variable pediatric training or experience, which could impact providers’ knowledge of and adherence to guideline-based pediatric care.35 Prior studies indicate that only 9.6% of urgent care centers employ any pediatricians,36 and pediatricians may also be underrepresented in DTC telemedicine. Together, these issues may increase clinical uncertainty during pediatric DTC telemedicine visits, prompting physicians to prescribe antibiotics “just to be safe.”37,38 

We note additional quality concerns raised by our analysis beyond antibiotic use. First, the American Telemedicine Association has recommended that commercial DTC telemedicine should not be used for children <2 years old.8 We note that 5% of studied DTC telemedicine visits for ARIs were within this age group but that gaps in quality existed for children of all ages. Second, accurate diagnosis of otitis media in children requires visualization of the tympanic membranes,11 and accurate diagnosis of streptococcal pharyngitis requires streptococcal testing.27 Making these diagnoses via telemedicine in the absence of reliable visualization and testing could in itself be considered a quality concern. Third, increased rates of follow-up visits after DTC telemedicine raise questions about the effectiveness and system-level efficiency of these visits.

Quality of care for ARIs may be better in other models of acute telemedicine care. For example, acute telemedicine visits in which peripheral devices such as tele-stethoscopes or tele-otoscopes (such as offered at telemedicine kiosks39) are integrated may allow for enhanced examination compared with DTC telemedicine. Using telemedicine technology to connect patients to their PCP’s office would allow telemedicine use within the context of continuity of information, relationships, and care management.7 School-based or day care–based models of acute telemedicine can be used to integrate peripherals and telepresenters.40,41 Finally, pediatric-specific improvement initiatives may also result in improved quality.33 Examining the variation in quality across different models of acute telemedicine care may inform optimal use of technology for acute pediatric concerns.

Although our focus was DTC telemedicine, the quality of antibiotic prescribing was also lower at urgent care visits compared with PCP visits (59% vs 67% guideline concordant). Urgent care clinics have grown rapidly in the United States,42 and urgent care and DTC telemedicine share some characteristics that might impede quality, such as limited continuity. Across all 3 settings, our findings emphasize the need for improvement in guideline-concordant antibiotic use and ongoing antibiotic stewardship efforts in outpatient settings.43,48 

Our analysis had several limitations. As an analysis of insurance claims, we did not have additional sociodemographic or clinical data and thus could not account for variables such as patient race, severity of illness, time constraints, or family expectations, which might drive differences in prescribing.38,49 Given the data available, we could not independently confirm the diagnosis made. The differences in diagnosis mix in the unmatched data could be due to true differences in case mix (perhaps due to self-selection to specific settings on the part of parents) or biases in diagnosis (perhaps due to differences in information available to clinicians or differences in desire to select a diagnosis that justifies prescription of antibiotics). Insurance claims also do not have information on patient allergies. Although we would not expect substantial differences in rates of allergies across settings, we may be underestimating overall guideline concordance in the absence of allergy data. Also, because some children were enrolled for brief periods, we may not be capturing all children with higher medical complexity. However, we confirmed that identified rates of chronic complex disease in the full population of pediatric health plan beneficiaries (6.4%) were relatively similar to rates identified in other population studies (6% in previous claims analysis29). We used claims data rather than electronic health record or provider-reported data. Although this means we examined antibiotics filled rather than prescribed, in-person visit antibiotic use appeared consistent with previous studies of antibiotic use. Specifically, our rates of antibiotic use among unmatched PCP visits were comparable to previous studies for diagnoses for which antibiotics may be indicated (59% vs 50%–80% in previous literature) and diagnoses for which antibiotics are not indicated (16% vs 20%–21% in previous literature).14,20,50 We did not include retail-based clinics in our analyses. Authors of other studies have found that among adults, retail-based clinics have higher rates of guideline-concordant care relative to PCPs.15,19 Physician specialty was not available for DTC telemedicine visits, so we could not examine whether pediatric-specific physician training explained any quality differences. We note also that our analysis is limited to a specific health plan and visits by its members to their contracted DTC telemedicine vendor, and our analysis also did not include Medicaid beneficiaries. Antibiotic prescribing among other DTC telemedicine companies, models, and populations may differ. Understanding the impact of physician training and telemedicine models on quality of care during DTC telemedicine should be a priority in future work.

Children who receive care via DTC telemedicine visits were significantly more likely to receive antibiotics and less likely to receive guideline-concordant antibiotic management than children who visit either urgent care or PCPs.

Dr Ray designed the study, supervised analysis, interpreted the data, and drafted the manuscript; Ms Shi conducted the analyses and interpreted the data; Dr Poon assisted with analysis and interpreted the data; Drs Gidengil and Uscher-Pines assisted with the study design and interpreted the data; Dr Mehrotra designed the study, acquired the data, supervised analysis, and interpreted the data; and all authors critically revised the manuscript, approved the final manuscript as submitted, and agree to be accountable for the work.

FUNDING: Supported in part by grants from the Eunice Kennedy Shriver National Institute of Child Health and Human Development (K23HD088642; Dr Ray) and gifts from Melvin Hall (Dr. Mehrotra). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication. Funded by the National Institutes of Health (NIH).

COMPANION PAPER: A companion to this article can be found online at www.pediatrics.org/cgi/doi/10.1542/peds.2019-0631.

     
  • ARI

    acute respiratory infection

  •  
  • DTC

    direct-to-consumer

  •  
  • HDHP

    high-deductible health plan

  •  
  • ICD-9-CM

    International Classification of Diseases, Ninth Revision, Clinical Modification

  •  
  • ICD-10-CM

    International Classification of Diseases, 10th Revision, Clinical Modification

  •  
  • PCP

    primary care provider

  •  
  • URI

    upper respiratory infection

1
Ashwood
JS
,
Mehrotra
A
,
Cowling
D
,
Uscher-Pines
L
.
Direct-to-consumer telehealth may increase access to care but does not decrease spending.
Health Aff (Millwood)
.
2017
;
36
(
3
):
485
491
[PubMed]
2
Teladoc
. The quality care you need with the convenience you want. Available at: https://www.teladoc.com/. Accessed April 2, 2018
3
Doctor on Demand
. See a doctor 24/7, anytime. Available at: www.doctorondemand.com. Accessed April 2, 2018
4
MDLive
. Always there. Visit a doctor, counselor, psychiatrist or dermatologist by secure video or phone. Available at: https://www.mdlive.com/. Accessed April 2, 2018.
5
National Business Group on Health
. NBGH press release: large U.S. employers project health care benefit costs to surpass $14,000 per employee in 2018, National Business Group on Health survey finds. 2017. Available at: https://www.businessgrouphealth.org/news/nbgh-news/press-releases/press-release-details/?ID=334. Accessed February 28, 2019.
6
Ray
KN
,
Shi
Z
,
Poon
SJ
,
Uscher-Pines
L
,
Mehrotra
A
.
Use of commercial direct-to-consumer telemedicine by children [published online ahead of print January 10, 2019].
Acad Pediatr
.
7
Conners
GP
,
Kressly
SJ
,
Perrin
JM
,
Richerson
JE
,
Sankrithi
UM
;
Committee on Practice and Ambulatory Medicine
;
Committee on Pediatric Emergency Medicine
;
Section on Telehealth Care
;
Section on Emergency Medicine
;
Subcommittee on Urgent Care
;
Task Force on Pediatric Practice Change
.
Nonemergency acute care: when it’s not the medical home.
Pediatrics
.
2017
;
139
(
5
):
e20170629
[PubMed]
8
McSwain
SD
,
Bernard
J
,
Burke
BL
 Jr
, et al
.
American telemedicine association operating procedures for pediatric telehealth.
Telemed J E Health
.
2017
;
23
(
9
):
699
706
[PubMed]
9
Uscher-Pines
L
,
Mulcahy
A
,
Cowling
D
,
Hunter
G
,
Burns
R
,
Mehrotra
A
.
Access and quality of care in direct-to-consumer telemedicine.
Telemed J E Health
.
2016
;
22
(
4
):
282
287
[PubMed]
10
Uscher-Pines
L
,
Mulcahy
A
,
Cowling
D
,
Hunter
G
,
Burns
R
,
Mehrotra
A
.
Antibiotic prescribing for acute respiratory infections in direct-to-consumer telemedicine visits.
JAMA Intern Med
.
2015
;
175
(
7
):
1234
1235
[PubMed]
11
Lieberthal
AS
,
Carroll
AE
,
Chonmaitree
T
, et al
.
The diagnosis and management of acute otitis media [published correction appears in Pediatrics. 2014;133(2):346].
Pediatrics
.
2013
;
131
(
3
). Available at: www.pediatrics.org/cgi/content/full/131/3/e964
[PubMed]
12
Wald
ER
,
Applegate
KE
,
Bordley
C
, et al;
American Academy of Pediatrics
.
Clinical practice guideline for the diagnosis and management of acute bacterial sinusitis in children aged 1 to 18 years.
Pediatrics
.
2013
;
132
(
1
). Available at: www.pediatrics.org/cgi/content/full/132/1/e262
[PubMed]
13
Ralston
SL
,
Lieberthal
AS
,
Meissner
HC
, et al;
American Academy of Pediatrics
.
Clinical practice guideline: the diagnosis, management, and prevention of bronchiolitis [published correction appears in Pediatrics. 2014;136(4):782].
Pediatrics
.
2014
;
134
(
5
). Available at: www.pediatrics.org/cgi/content/full/134/5/e1474
[PubMed]
14
Fleming-Dutra
KE
,
Hersh
AL
,
Shapiro
DJ
, et al
.
Prevalence of inappropriate antibiotic prescriptions among US ambulatory care visits, 2010-2011.
JAMA
.
2016
;
315
(
17
):
1864
1873
[PubMed]
15
Mehrotra
A
,
Gidengil
CA
,
Setodji
CM
,
Burns
RM
,
Linder
JA
.
Antibiotic prescribing for respiratory infections at retail clinics, physician practices, and emergency departments.
Am J Manag Care
.
2015
;
21
(
4
):
294
302
[PubMed]
16
Gerber
JS
,
Prasad
PA
,
Russell Localio
A
, et al
.
Variation in antibiotic prescribing across a pediatric primary care network.
J Pediatric Infect Dis Soc
.
2015
;
4
(
4
):
297
304
[PubMed]
17
Dooling
KL
,
Shapiro
DJ
,
Van Beneden
C
,
Hersh
AL
,
Hicks
LA
.
Overprescribing and inappropriate antibiotic selection for children with pharyngitis in the United States, 1997-2010.
JAMA Pediatr
.
2014
;
168
(
11
):
1073
1074
[PubMed]
18
Hersh
AL
,
Shapiro
DJ
,
Pavia
AT
,
Shah
SS
.
Antibiotic prescribing in ambulatory pediatrics in the United States.
Pediatrics
.
2011
;
128
(
6
):
1053
1061
[PubMed]
19
Palms
DL
,
Hicks
LA
,
Bartoces
M
, et al
.
Comparison of antibiotic prescribing in retail clinics, urgent care centers, emergency departments, and traditional ambulatory care settings in the United States.
JAMA Intern Med
.
2018
;
178
(
9
):
1267
1269
[PubMed]
20
Grijalva
CG
,
Nuorti
JP
,
Griffin
MR
.
Antibiotic prescription rates for acute respiratory tract infections in US ambulatory settings.
JAMA
.
2009
;
302
(
7
):
758
766
[PubMed]
21
Gidengil
CA
,
Mehrotra
A
,
Beach
S
,
Setodji
C
,
Hunter
G
,
Linder
JA
.
What drives variation in antibiotic prescribing for acute respiratory infections?
J Gen Intern Med
.
2016
;
31
(
8
):
918
924
[PubMed]
22
Roth
J
;
National Bureau of Economic Research
. CMS ICD-9-CM to and from ICD-10-CM and ICD-10-PCD crosswalk or general equivalence mappings. Available at: www.nber.org/data/icd9-icd-10-cm-and-pcs-crosswalk-general-equivalence-mapping.html. Accessed April 11, 2018
23
Mehrotra
A
,
Paone
S
,
Martich
GD
,
Albert
SM
,
Shevchik
GJ
.
A comparison of care at e-visits and physician office visits for sinusitis and urinary tract infection.
JAMA Intern Med
.
2013
;
173
(
1
):
72
74
[PubMed]
24
Poon
SJ
,
Schuur
JD
,
Mehrotra
A
.
Trends in visits to acute care venues for treatment of low-acuity conditions in the United States from 2008 to 2015.
JAMA Intern Med
.
2018
;
178
(
10
):
1342
1349
[PubMed]
25
Mehrotra
A
,
Paone
S
,
Martich
GD
,
Albert
SM
,
Shevchik
GJ
.
Characteristics of patients who seek care via eVisits instead of office visits.
Telemed J E Health
.
2013
;
19
(
7
):
515
519
[PubMed]
26
Gidengil
CA
,
Linder
JA
,
Beach
S
,
Setodji
CM
,
Hunter
G
,
Mehrotra
A
.
Using clinical vignettes to assess quality of care for acute respiratory infections.
Inquiry
.
2016
;
53
:
0046958016636531
[PubMed]
27
Shulman
ST
,
Bisno
AL
,
Clegg
HW
, et al
.
Clinical practice guideline for the diagnosis and management of group A streptococcal pharyngitis: 2012 update by the Infectious Diseases Society of America [published correction appears in Clin Infect Dis. 2014;58(10):1496].
Clin Infect Dis
.
2012
;
55
(
10
):
1279
1282
[PubMed]
28
United States Department of Agriculture
. Rural-urban commuting area codes. Available at: https://www.ers.usda.gov/data-products/rural-urban-commuting-area-codes/. Accessed April 2, 2018
29
Simon
TD
,
Cawthon
ML
,
Popalisky
J
,
Mangione-Smith
R
;
Center of Excellence on Quality of Care Measures for Children With Complex Needs (COE4CCN)
.
Development and validation of the pediatric medical complexity algorithm (PMCA) version 2.0.
Hosp Pediatr
.
2017
;
7
(
7
):
373
377
[PubMed]
30
Iacus
SM
,
King
G
,
Poorro
G
.
Causal inference without balance checking: coarsened exact matching.
Polit Anal
.
2012
;
20
(
1
):
1
24
31
Benjamini
Y
,
Hochberg
Y
.
Controlling the false discovery rate: a practical and powerful approach to multiple testing.
J R Stat Soc B
.
1995
;
57
(
1
):
289
300
32
Shi
Z
,
Mehrotra
A
,
Gidengil
CA
,
Poon
SJ
,
Uscher-Pines
L
,
Ray
KN
.
Quality of care for acute respiratory infections during direct-to-consumer telemedicine visits for adults.
Health Aff (Millwood)
.
2018
;
37
(
12
):
2014
2023
33
Gali
K
,
Faiman
M
,
Romm
S
. Ensuring clinical quality in telemedicine. NEJM Catalyst. 2018. Available at: https://catalyst.nejm.org/clinical-quality-telemedicine-online-care/. Accessed October 12, 2018
34
Haggerty
JL
,
Reid
RJ
,
Freeman
GK
,
Starfield
BH
,
Adair
CE
,
McKendry
R
.
Continuity of care: a multidisciplinary review.
BMJ
.
2003
;
327
(
7425
):
1219
1221
[PubMed]
35
Guttmann
A
,
Manuel
D
,
Dick
PT
,
To
T
,
Lam
K
,
Stukel
TA
.
Volume matters: physician practice characteristics and immunization coverage among young children insured through a universal health plan.
Pediatrics
.
2006
;
117
(
3
):
595
602
[PubMed]
36
Weinick
RM
,
Bristol
SJ
,
DesRoches
CM
.
Urgent care centers in the U.S.: findings from a national survey.
BMC Health Serv Res
.
2009
;
9
:
79
[PubMed]
37
Horwood
J
,
Cabral
C
,
Hay
AD
,
Ingram
J
.
Primary care clinician antibiotic prescribing decisions in consultations for children with RTIs: a qualitative interview study.
Br J Gen Pract
.
2016
;
66
(
644
):
e207
e213
[PubMed]
38
Zetts
RM
,
Stoesz
A
,
Smith
BA
,
Hyun
DY
.
Outpatient antibiotic use and the need for increased antibiotic stewardship efforts.
Pediatrics
.
2018
;
141
(
6
):
e20174124
[PubMed]
39
Pledger
M
. First 25 Ohio-developed telemedicine kiosks open in Northeast Ohio RiteAids: HealthSpot. 2015. Available at: www.cleveland.com/business/index.ssf/2015/07/the_first_25_ohio-developed_te.html. Accessed April 2, 2018
40
McConnochie
KM
,
Wood
NE
,
Herendeen
NE
,
ten Hoopen
CB
,
Roghmann
KJ
.
Telemedicine in urban and suburban childcare and elementary schools lightens family burdens.
Telemed J E Health
.
2010
;
16
(
5
):
533
542
[PubMed]
41
North
SW
,
McElligot
J
,
Douglas
G
,
Martin
A
.
Improving access to care through the patient-centered medical home.
Pediatr Ann
.
2014
;
43
(
2
):
e33
e38
[PubMed]
42
Urgent Care Association of America
. Benchmarking report summary: headlines on growth. 2016. Available at: https://www.ucaoa.org/Portals/80/pdfs/benchmarking/2016BenchmarkReport.pdf. Accessed April 12, 2018
43
Sanchez
GV
,
Fleming-Dutra
KE
,
Roberts
RM
,
Hicks
LA
.
Core elements of outpatient antibiotic stewardship.
MMWR Recomm Rep
.
2016
;
65
(
6
):
1
12
[PubMed]
44
Gerber
JS
,
Prasad
PA
,
Fiks
AG
, et al
.
Effect of an outpatient antimicrobial stewardship intervention on broad-spectrum antibiotic prescribing by primary care pediatricians: a randomized trial.
JAMA
.
2013
;
309
(
22
):
2345
2352
[PubMed]
45
Tonkin-Crine
SK
,
Tan
PS
,
van Hecke
O
, et al
.
Clinician-targeted interventions to influence antibiotic prescribing behaviour for acute respiratory infections in primary care: an overview of systematic reviews.
Cochrane Database Syst Rev
.
2017
;(
9
):
CD012252
46
McDonagh
M
,
Peterson
K
,
Winthrop
K
,
Cantor
A
,
Holzhammer
B
,
Buckley
DI
.
Improving Antibiotic Prescribing for Uncomplicated Acute Respiratory Tract Infections: Comparative Effectiveness Reviews, No. 163
.
Rockville, MD
:
Agency for Healthcare Research and Quality (US)
;
2016
47
Meeker
D
,
Linder
JA
,
Fox
CR
, et al
.
Effect of behavioral interventions on inappropriate antibiotic prescribing among primary care practices: a randomized clinical trial.
JAMA
.
2016
;
315
(
6
):
562
570
[PubMed]
48
Linder
JA
,
Meeker
D
,
Fox
CR
, et al
.
Effects of behavioral interventions on inappropriate antibiotic prescribing in primary care 12 months after stopping interventions.
JAMA
.
2017
;
318
(
14
):
1391
1392
[PubMed]
49
Gerber
JS
,
Prasad
PA
,
Localio
AR
, et al
.
Racial differences in antibiotic prescribing by primary care pediatricians.
Pediatrics
.
2013
;
131
(
4
):
677
684
[PubMed]
50
Steinman
MA
,
Gonzales
R
,
Linder
JA
,
Landefeld
CS
.
Changing use of antibiotics in community-based outpatient practice, 1991-1999.
Ann Intern Med
.
2003
;
138
(
7
):
525
533
[PubMed]

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

Supplementary data