Video Abstract
Use of commercial direct-to-consumer (DTC) telemedicine outside of the pediatric medical home is increasing among children, and acute respiratory infections (ARIs) are the most commonly diagnosed condition at DTC telemedicine visits. Our objective was to compare the quality of antibiotic prescribing for ARIs among children across 3 settings: DTC telemedicine, urgent care, and the primary care provider (PCP) office.
In a retrospective cohort study using 2015–2016 claims data from a large national commercial health plan, we identified ARI visits by children (0–17 years old), excluding visits with comorbidities that could affect antibiotic decisions. Visits were matched on age, sex, chronic medical complexity, state, rurality, health plan type, and ARI diagnosis category. Within the matched sample, we compared the percentage of ARI visits with any antibiotic prescribing and the percentage of ARI visits with guideline-concordant antibiotic management.
There were 4604 DTC telemedicine, 38 408 urgent care, and 485 201 PCP visits for ARIs in the matched sample. Antibiotic prescribing was higher for DTC telemedicine visits than for other settings (52% of DTC telemedicine visits versus 42% urgent care and 31% PCP visits; P < .001 for both comparisons). Guideline-concordant antibiotic management was lower at DTC telemedicine visits than at other settings (59% of DTC telemedicine visits versus 67% urgent care and 78% PCP visits; P < .001 for both comparisons).
At DTC telemedicine visits, children with ARIs were more likely to receive antibiotics and less likely to receive guideline-concordant antibiotic management compared to children at PCP visits and urgent care visits.
Authors of previous studies identified concerns about antibiotic prescribing for adults using direct-to-consumer (DTC) telemedicine. No study has examined antibiotic prescribing during pediatric DTC telemedicine visits.
Compared with urgent care and primary care visits, children receiving care for acute respiratory infections during DTC telemedicine visits are more likely to receive antibiotics and less likely to receive guideline-concordant antibiotic management.
Many commercial companies offer direct-to-consumer (DTC) telemedicine visits in which patients are connected to physicians outside of the medical home through audio-only or audio-video conferencing on their personal devices.1 DTC telemedicine companies highlight the convenience of this care and report high patient satisfaction.2,–4 Access to telemedicine has expanded, with 96% of large business insurance plans now offering coverage for employees and their families.5 As a result, millions of children now have access to DTC telemedicine, and use is rapidly increasing.6
The American Academy of Pediatrics discourages the use of DTC telemedicine outside of the medical home for acute pediatric care, citing concerns such as limited physical examination capabilities, lack of ongoing patient-provider relationships, and lack of access to patient records, all of which may lead to lower-quality care.7 The American Academy of Pediatrics further notes that these issues may be particularly important with children who may have difficulty verbalizing their symptoms.7 The American Telemedicine Association echoed many of these concerns,8 adding that the use of personal devices may also impact technical quality and privacy of care. The American Telemedicine Association further suggests that DTC telemedicine outside of the medical home should not be used for children <2 years old.8 Both organizations emphasize the need for telemedicine visits to adhere to the same standards expected of in-person care, including adherence to professional guidelines for appropriate examination, diagnostic testing, and treatment.7,8
To date, there is limited evidence regarding the quality of care during pediatric DTC telemedicine visits. Early studies of adult patients using DTC telemedicine compared with physician office visits have identified more inappropriate antibiotic prescriptions for bronchitis9 and more broad-spectrum antibiotic use.10 The quality of DTC telemedicine care for children may differ for several reasons. First, there is heightened reliance on the physical examination among children who cannot fully articulate symptoms.7 Second, care requires knowledge of pediatric-specific guidelines.11,–13 Third, rates of guideline-concordant antibiotic management at physician visits is higher for children than for adults14; therefore, the relative difference in quality of care at DTC telemedicine visits may be different.
To fill this gap in knowledge about the quality of commercial DTC telemedicine care for children, we used data from a large national health plan to compare the quality of antibiotic prescribing for acute respiratory infections (ARIs) at pediatric DTC telemedicine, urgent care, and primary care provider (PCP) visits. We focus on ARIs because authors of previous studies identified concerns about antibiotic prescribing during pediatric ARI visits in other settings,14,–19 and because ARIs account for the majority of DTC telemedicine visits.6
Methods
Study Population
We performed a retrospective cohort study using 2015–2016 claims data from a large national commercial health plan, which provides coverage for roughly 4 million children annually from all 9 census divisions and contracts with a single national DTC telemedicine vendor. Children age 0 to 17 years were included, with no minimum enrollment duration requirement. We excluded children who lacked pharmaceutical coverage during the month of their visit to ensure record of pharmacy claims.
Identifying ARI Visits
ARI visits were identified by using previously published methods on the basis of International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes.15,20,21 Consistent with previous work,14,15,20,21 ARI diagnoses included diagnoses for which antibiotics may be warranted (sinusitis [461.x], pneumonia [481.x–483.x, 485.x–486.x], streptococcal pharyngitis [034.x], acute otitis media [382.x]) and diagnoses for which antibiotics are not warranted (eg, viral upper respiratory infection [URI], bronchiolitis, viral pharyngitis, serous otitis media [381.x, 460.x, 462–466.x, 480.x, 490.x]). We excluded bacterial pneumonia visits because of the small number of DTC telemedicine visits identified (n = 11). We converted International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM) codes to ICD-9-CM codes using a modified crosswalk created by the Centers for Medicare and Medicaid Services (see Supplemental Table 5).22
To examine the first ARI visit within an episode of illness, we identified “index” ARI visits with no ARI visits at any location in the previous 21 days, which is consistent with previous studies.10,23 Subsequent ARI visits within 21 days of an index visit were considered follow-up care. We excluded episodes as detailed in Supplemental Fig 2. Using a previously published methodology, we excluded episodes for which the index visit included a diagnosis indicating chronic conditions (eg, history of transplant) or additional codiagnoses (eg, urinary tract infection) that could impact antibiotic prescribing decisions (n = 60 947; 4.7%) (Supplemental Table 6).15,21 These exclusion criteria were originally developed for adult populations. Although not all of these exclusion criteria are reasons to deviate from pediatric ARI guidelines, we applied them to study a set of visits consistent with other studies.
Identifying Setting of Care
ARI episodes were categorized as DTC telemedicine, urgent care, or PCP on the basis of the index visit setting. We identified setting using place of service codes, Current Procedural Terminology codes, insurer provider specialty codes, and organizational national provider identifiers and tax identification numbers for urgent care and DTC telemedicine companies through previously described methods (additional details in Supplemental Information).24 ARI episodes that began with index visits to settings other than DTC telemedicine, urgent care, or PCP offices (eg, emergency department, retail-based clinics) were not included.
Outcomes
Our 2 primary outcomes, compared across settings, were (1) the proportion of visits that resulted in an antibiotic prescription and (2) the proportion of visits that resulted in antibiotic management that was concordant with guidelines for the visit diagnosis.
We identified all filled prescriptions on the day of the index visit and subsequent 2 days and assigned these to the index visit, consistent with previous methods.1,10,25 We included all systemic antibiotics (ie, oral, intramuscular) and excluded topical, ophthalmic, or otic antibiotics. We determined guideline concordance of antibiotics with visit diagnosis, assigning each visit to 1 of 5 categories (Supplemental Table 7).21,26 “Antibiotic prescribed” visits included visits categorized as guideline-concordant antibiotic use, guideline–non-concordant antibiotic use, and unnecessary antibiotic use. “Guideline-concordant antibiotic management” included visits categorized as guideline-concordant antibiotic use for sinusitis, otitis media, or streptococcal pharyngitis (ie, amoxicillin or amoxicillin-clavulanic acid for sinusitis,12 amoxicillin or amoxicillin-clavulanic acid for otitis media,11 and penicillin or amoxicillin for streptococcal pharyngitis27), no antibiotic use for sinusitis or otitis media (also guideline-concordant management for these diagnoses11,12), or guideline-concordant nonuse of antibiotics for diagnoses that do not warrant antibiotics (eg, bronchiolitis13).
As secondary outcomes, we compared the occurrence of streptococcal testing within 1 day for children diagnosed with streptococcal pharyngitis27 as well as return visits within 2 and 21 days of an index visit to any setting (eg, PCP, DTC telemedicine, urgent care, retail clinic, emergency department). We did not have streptococcal testing results.
Patient and Visit Characteristics
Patient age, sex, and high-deductible health plan (HDHP) status were determined from insurance enrollment files. Using the patients’ zip code, we determined census region and rural-urban commuting area codes.28 We determined patient chronic medical complexity using the Pediatric Medical Complexity Algorithm Version 2 applied to 2014–2016 inpatient and outpatient claims.29 This algorithm is used to identify children with 3 levels of chronic medical complexity: complex chronic disease, noncomplex chronic disease, and no chronic disease.29
Matching
To address differences in patients and conditions managed in the 3 settings, we used coarsened-exact matching,30 which prunes observations to achieve covariate balance between groups. Instead of matching to a fixed number of visits, coarsened-exact matching matches DTC telemedicine visits to many matched urgent care visits and PCP visits and then weights each stratum or matched set.30 Specifically, we matched DTC telemedicine visits to urgent care visits and separately to PCP visits on the basis of age (in years), sex, chronic medical complexity, state, rural or urban area, HDHP status, and diagnosis category (eg, otitis media, viral URI).
Statistical Analysis
Across ARI visits for the 3 settings of care, we described patient characteristics and diagnoses. In the unmatched sample and the matched weighted sample, we examined antibiotic prescribing and guideline-concordant antibiotic management for DTC telemedicine visits compared with urgent care and compared with PCP offices using χ2 tests. We include both unmatched and matched results because of potential subjectivity in diagnoses, which may then drive variability in antibiotic prescribing.16 In the matched weighted sample, we also compared streptococcal testing and return visits by setting using χ2 tests. We adjusted for multiple comparisons in our main results using the Benjamini-Hochberg false discovery rate,31 yielding a P value of .0342.
To examine whether differences in antibiotic prescribing varied by patient characteristics, we performed further analysis that was focused on 2 variables that may impact guideline adherence: child age and chronic medical conditions. Within each category of child age and medical complexity, we examined antibiotic testing and guideline concordance for DTC telemedicine compared with urgent care and PCP offices using χ2 tests and then tested the significance of an interaction term between age or medical complexity and visit setting using logistic regression.
We performed 2 sensitivity analyses to examine the robustness of our findings to specific assumptions. First, we conducted a sensitivity analysis in which we redefined guideline-concordant antibiotic management so that it no longer included “no antibiotic use” for sinusitis and otitis. Because guidelines allow for observation without antibiotic use in specific circumstances for sinusitis and otitis media.11,12 we considered no antibiotic use to be a guideline-concordant management option for these diagnoses in our main analysis but not in this sensitivity analysis. Second, we examined antibiotic prescribing and guideline-concordant management after excluding ARI visits with a follow-up visit within 2 days because visits with such early follow-up might result in miscategorization of the site of antibiotic prescribing.
Analyses were conducted in SAS version 9.4 (SAS Institute, Inc, Cary, NC). Deidentified data were used in this analysis, which was determined to be exempt by Harvard Medical School’s Institutional Review Board.
Results
In the unmatched sample, there were 5431 index ARI visits to DTC telemedicine, 87 555 to urgent care, and 1 132 116 to PCPs in 2015–2016 that met inclusion criteria. Compared with urgent care and PCP visits, DTC telemedicine visits were more likely to involve children who were older, enrolled in HDHPs, and in the South (Table 1). Among DTC telemedicine visits, 5% were for children <2 years old. At DTC telemedicine visits, children were more likely to receive a diagnosis of viral ARI (68% vs 54% urgent care; 59% PCP visits) or sinusitis (18% vs 8% urgent care; 10% PCP visits) and less likely to receive a diagnosis of otitis media (8% vs 26% urgent care; 23% PCP).
Characteristics of ARI Visits Before and After Matching, 2015–2016
| Pediatric Visits | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Before Matching | After Matchinga | |||||||||
| DTC Telemedicine, No. (%) | Urgent Care, No. (%) | Standardized Difference,b % | PCP, No. (%) | Standardized Difference,b % | DTC Telemedicine, No. (%) | Urgent Care, No. (%) | Standardized Difference,b % | PCP, No. (%) | Standardized Difference,b % | |
| Visits, N | 5431 | 87 555 | — | 1 132 116 | — | 4604 | 38 408 | — | 485 201 | — |
| Child age, y | ||||||||||
| 0–1 | 245 (5) | 8234 (9) | 59.0 | 195 692 (17) | 89.6 | 224 (5) | 1956 (5) | 4.7 | 24 192 (5) | 2.5 |
| 2–5 | 1305 (24) | 24 845 (28) | 21.5 | 390 699 (35) | 46.4 | 1132 (25) | 9528 (25) | 1.2 | 121 932 (25) | 2.9 |
| 6–11 | 1930 (36) | 30 018 (34) | 5.6 | 331 716 (29) | 30.1 | 1614 (35) | 13 392 (35) | 0.8 | 168 036 (35) | 1.9 |
| 12–17 | 1951 (36) | 24 458 (28) | 39.3 | 214 009 (19) | 110.7 | 1634 (35) | 13 532 (35) | 1.1 | 171 041 (35) | 1.0 |
| Sex | ||||||||||
| Female | 2755 (51) | 44 181 (50) | 1.1 | 555 277 (49) | 6.7 | 2339 (51) | 19 513 (51) | 0 | 246 500 (51) | 0 |
| Male | 2676 (49) | 43 374 (50) | — | 576 839 (51) | — | 2265 (49) | 18 895 (49) | — | 238 701 (49) | — |
| Chronic medical complexityc | ||||||||||
| No chronic conditions | 3993 (74) | 65 931 (75) | 9.5 | 835 897 (74) | 1.6 | 3542 (77) | 29 548 (77) | 0 | 373 280 (77) | 0 |
| Noncomplex chronic condition | 1062 (20) | 14 823 (17) | 18.5 | 197 934 (17) | 14.3 | 803 (17) | 6699 (17) | 0 | 84 626 (17) | 0 |
| Complex chronic condition | 376 (7) | 6801 (8) | 11.9 | 98 285 (9) | 22.2 | 259 (6) | 2161 (6) | 0 | 27 295 (6) | 0 |
| Rural or urband | ||||||||||
| Metropolitan | 4967 (91) | 81 028 (93) | 15.7 | 1 052 233 (93) | 22.7 | 4415 (96) | 36 839 (96) | 0.5 | 465 266 (96) | 0.1 |
| Micropolitan | 284 (5) | 4421 (5) | 3.7 | 48 849 (4) | 22.1 | 140 (3) | 1161 (3) | 0.6 | 14 771 (3) | 0.1 |
| Small town | 109 (2) | 1449 (2) | 21.4 | 20 540 (2) | 10.8 | 34 (1) | 284 (1) | 0 | 3583 (1) | 0 |
| Rural | 71 (1) | 657 (1) | 71.7 | 10 494 (1) | 41.3 | 15 (0) | 125 (0) | 0 | 1581 (0) | 0 |
| Region | ||||||||||
| Northeast | 627 (12) | 5691 (7) | 79.7 | 346 792 (31) | 90.0 | 490 (11) | 4088 (11) | 0 | 51 640 (11) | 0 |
| Midwest | 929 (17) | 16 393 (19) | 10.7 | 161 878 (14) | 22.9 | 701 (15) | 5833 (15) | 0.3 | 73 876 (15) | 0 |
| South | 2943 (54) | 42 860 (49) | 21.0 | 445 160 (39) | 62.3 | 2587 (56) | 21 757 (57) | 1.9 | 275 657 (57) | 2.5 |
| West | 932 (17) | 22 611 (26) | 45.8 | 178 286 (16) | 10.6 | 826 (18) | 6730 (18) | 2.9 | 84 029 (17) | 4.3 |
| HDHP | ||||||||||
| No | 4575 (84) | 81 720 (93) | 136.2 | 1 062 214 (94) | 164.3 | 4119 (89) | 34 362 (89) | 0 | 434 088 (89) | 0 |
| Yes | 856 (16) | 5835 (7) | — | 69 902 (6) | — | 485 (11) | 4046 (11) | — | 51 113 (11) | — |
| Diagnosis categorye | ||||||||||
| Sinusitis | 966 (18) | 7101 (8) | 122.1 | 113 392 (10) | 85.9 | 646 (14) | 5389 (14) | 0 | 68 080 (14) | 0 |
| Streptococcal pharyngitis | 326 (6) | 11 745 (13) | 65.6 | 103 066 (9) | 37.5 | 250 (5) | 2086 (5) | 0 | 26 347 (5) | 0 |
| Otitis media | 429 (8) | 22 838 (26) | 96.9 | 263 743 (23) | 86.4 | 369 (8) | 3078 (8) | 0 | 38 888 (8) | 0 |
| Viral ARIf | 3710 (68) | 46 979 (54) | 59.1 | 664 912 (59) | 39.5 | 3339 (73) | 27 855 (73) | 0 | 351 887 (73) | 0 |
| Pediatric Visits | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Before Matching | After Matchinga | |||||||||
| DTC Telemedicine, No. (%) | Urgent Care, No. (%) | Standardized Difference,b % | PCP, No. (%) | Standardized Difference,b % | DTC Telemedicine, No. (%) | Urgent Care, No. (%) | Standardized Difference,b % | PCP, No. (%) | Standardized Difference,b % | |
| Visits, N | 5431 | 87 555 | — | 1 132 116 | — | 4604 | 38 408 | — | 485 201 | — |
| Child age, y | ||||||||||
| 0–1 | 245 (5) | 8234 (9) | 59.0 | 195 692 (17) | 89.6 | 224 (5) | 1956 (5) | 4.7 | 24 192 (5) | 2.5 |
| 2–5 | 1305 (24) | 24 845 (28) | 21.5 | 390 699 (35) | 46.4 | 1132 (25) | 9528 (25) | 1.2 | 121 932 (25) | 2.9 |
| 6–11 | 1930 (36) | 30 018 (34) | 5.6 | 331 716 (29) | 30.1 | 1614 (35) | 13 392 (35) | 0.8 | 168 036 (35) | 1.9 |
| 12–17 | 1951 (36) | 24 458 (28) | 39.3 | 214 009 (19) | 110.7 | 1634 (35) | 13 532 (35) | 1.1 | 171 041 (35) | 1.0 |
| Sex | ||||||||||
| Female | 2755 (51) | 44 181 (50) | 1.1 | 555 277 (49) | 6.7 | 2339 (51) | 19 513 (51) | 0 | 246 500 (51) | 0 |
| Male | 2676 (49) | 43 374 (50) | — | 576 839 (51) | — | 2265 (49) | 18 895 (49) | — | 238 701 (49) | — |
| Chronic medical complexityc | ||||||||||
| No chronic conditions | 3993 (74) | 65 931 (75) | 9.5 | 835 897 (74) | 1.6 | 3542 (77) | 29 548 (77) | 0 | 373 280 (77) | 0 |
| Noncomplex chronic condition | 1062 (20) | 14 823 (17) | 18.5 | 197 934 (17) | 14.3 | 803 (17) | 6699 (17) | 0 | 84 626 (17) | 0 |
| Complex chronic condition | 376 (7) | 6801 (8) | 11.9 | 98 285 (9) | 22.2 | 259 (6) | 2161 (6) | 0 | 27 295 (6) | 0 |
| Rural or urband | ||||||||||
| Metropolitan | 4967 (91) | 81 028 (93) | 15.7 | 1 052 233 (93) | 22.7 | 4415 (96) | 36 839 (96) | 0.5 | 465 266 (96) | 0.1 |
| Micropolitan | 284 (5) | 4421 (5) | 3.7 | 48 849 (4) | 22.1 | 140 (3) | 1161 (3) | 0.6 | 14 771 (3) | 0.1 |
| Small town | 109 (2) | 1449 (2) | 21.4 | 20 540 (2) | 10.8 | 34 (1) | 284 (1) | 0 | 3583 (1) | 0 |
| Rural | 71 (1) | 657 (1) | 71.7 | 10 494 (1) | 41.3 | 15 (0) | 125 (0) | 0 | 1581 (0) | 0 |
| Region | ||||||||||
| Northeast | 627 (12) | 5691 (7) | 79.7 | 346 792 (31) | 90.0 | 490 (11) | 4088 (11) | 0 | 51 640 (11) | 0 |
| Midwest | 929 (17) | 16 393 (19) | 10.7 | 161 878 (14) | 22.9 | 701 (15) | 5833 (15) | 0.3 | 73 876 (15) | 0 |
| South | 2943 (54) | 42 860 (49) | 21.0 | 445 160 (39) | 62.3 | 2587 (56) | 21 757 (57) | 1.9 | 275 657 (57) | 2.5 |
| West | 932 (17) | 22 611 (26) | 45.8 | 178 286 (16) | 10.6 | 826 (18) | 6730 (18) | 2.9 | 84 029 (17) | 4.3 |
| HDHP | ||||||||||
| No | 4575 (84) | 81 720 (93) | 136.2 | 1 062 214 (94) | 164.3 | 4119 (89) | 34 362 (89) | 0 | 434 088 (89) | 0 |
| Yes | 856 (16) | 5835 (7) | — | 69 902 (6) | — | 485 (11) | 4046 (11) | — | 51 113 (11) | — |
| Diagnosis categorye | ||||||||||
| Sinusitis | 966 (18) | 7101 (8) | 122.1 | 113 392 (10) | 85.9 | 646 (14) | 5389 (14) | 0 | 68 080 (14) | 0 |
| Streptococcal pharyngitis | 326 (6) | 11 745 (13) | 65.6 | 103 066 (9) | 37.5 | 250 (5) | 2086 (5) | 0 | 26 347 (5) | 0 |
| Otitis media | 429 (8) | 22 838 (26) | 96.9 | 263 743 (23) | 86.4 | 369 (8) | 3078 (8) | 0 | 38 888 (8) | 0 |
| Viral ARIf | 3710 (68) | 46 979 (54) | 59.1 | 664 912 (59) | 39.5 | 3339 (73) | 27 855 (73) | 0 | 351 887 (73) | 0 |
—, not applicable.
Sample characteristics before and after matching. In comparison of matched samples, each stratum of matched sets is weighted to account for differences in the number of matched urgent care and PCP visits. Counts within the matched categories may exceed column total because of rounding of weighted frequencies. Visits were matched by age (in years) and state, but results are reported in the table by larger categories of age group and census region.
Standardized differences comparing the characteristics of urgent care or PCP visits (in the column to the left) to DTC telemedicine, before and after matching.
Patient chronic medical complexity was determined by using the Pediatric Medical Complexity Algorithm applied to all 2014–2016 inpatient and outpatient claims for each patient.29 This algorithm is used to identify children with 3 levels of chronic medical complexity: complex chronic disease, (ie, significant multisystem chronic disease, progressive conditions with decreased life expectancy, technology dependence, or malignancy), noncomplex chronic disease (ie, chronic conditions such as asthma that do not meet the above criteria), and no chronic disease.
Rural or urban status was determined by using the US Department of Agriculture rural-urban commuting area codes.
Because a small percentage of visits have >1 of these diagnoses (1.1%), percentages for diagnosis category may sum to >100%.
Viral ARI category consists of diagnoses for which antibiotics are not warranted (eg, viral URI, bronchiolitis, viral pharyngitis, and serous otitis media) in the absence of another diagnosis.
In the unmatched cohort (Table 2), DTC telemedicine visits were associated with substantially increased antibiotic prescribing (52% vs 46% urgent care; 33% PCP; P < .001 for both) and decreased guideline-concordant antibiotic management compared with other settings (60% vs 71% urgent care; 80% PCP; P < .001 for both).
Antibiotic Prescribing and Guideline-Concordant Antibiotic Management in Matched Sample, 2015–2016
| Unmatched Visits | Matched Visits | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| DTC Telemedicine | Urgent Care | Pa | PCP | Pb | DTC Telemedicine | Urgent Care | Pa | PCP | Pb | |
| Total ARI visits, N | 5431 | 87 555 | — | 1 132 116 | — | 4604 | 38 408 | — | 485 201 | — |
| Any antibiotics prescribed,c n (%)d | 2846 (52) | 40 594 (46) | <.001 | 376 638 (33) | <.001 | 2381 (52) | 16 198 (42) | <.001 | 152 687 (31) | <.001 |
| Guideline-concordant management,e n (%)d | 3246 (60) | 61 842 (71) | <.001 | 907 450 (80) | <.001 | 2712 (59) | 25 788 (67) | <.001 | 376 259 (78) | <.001 |
| Specifics category of guideline concordance and nonconcordance among ARI visits, n (%)d | ||||||||||
| Guideline-concordant use | 761 (14) | 18 613 (21) | <.001 | 189 970 (17) | <.001 | 561 (12) | 4260 (11) | .026 | 52 776 (11) | .005 |
| Guideline nonconcordant use | 369 (7) | 7514 (9) | <.001 | 83 297 (7) | .11 | 271 (6) | 2507 (7) | .095 | 31 138 (6) | .14 |
| Unnecessary antibiotic use | 1716 (32) | 14 467 (17) | <.001 | 103 371 (9) | <.001 | 1549 (34) | 9431 (25) | <.001 | 68 773 (14) | <.001 |
| No antibiotic use | 591 (11) | 14 449 (17) | <.001 | 193 937 (17) | <.001 | 433 (9) | 3787 (10) | .33 | 49 401 (10) | .083 |
| Guideline-concordant nonuse | 1994 (37) | 32 512 (37) | .54 | 561 541 (50) | <.001 | 1790 (39) | 18 424 (48) | <.001 | 283 114 (58) | <.001 |
| Unmatched Visits | Matched Visits | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| DTC Telemedicine | Urgent Care | Pa | PCP | Pb | DTC Telemedicine | Urgent Care | Pa | PCP | Pb | |
| Total ARI visits, N | 5431 | 87 555 | — | 1 132 116 | — | 4604 | 38 408 | — | 485 201 | — |
| Any antibiotics prescribed,c n (%)d | 2846 (52) | 40 594 (46) | <.001 | 376 638 (33) | <.001 | 2381 (52) | 16 198 (42) | <.001 | 152 687 (31) | <.001 |
| Guideline-concordant management,e n (%)d | 3246 (60) | 61 842 (71) | <.001 | 907 450 (80) | <.001 | 2712 (59) | 25 788 (67) | <.001 | 376 259 (78) | <.001 |
| Specifics category of guideline concordance and nonconcordance among ARI visits, n (%)d | ||||||||||
| Guideline-concordant use | 761 (14) | 18 613 (21) | <.001 | 189 970 (17) | <.001 | 561 (12) | 4260 (11) | .026 | 52 776 (11) | .005 |
| Guideline nonconcordant use | 369 (7) | 7514 (9) | <.001 | 83 297 (7) | .11 | 271 (6) | 2507 (7) | .095 | 31 138 (6) | .14 |
| Unnecessary antibiotic use | 1716 (32) | 14 467 (17) | <.001 | 103 371 (9) | <.001 | 1549 (34) | 9431 (25) | <.001 | 68 773 (14) | <.001 |
| No antibiotic use | 591 (11) | 14 449 (17) | <.001 | 193 937 (17) | <.001 | 433 (9) | 3787 (10) | .33 | 49 401 (10) | .083 |
| Guideline-concordant nonuse | 1994 (37) | 32 512 (37) | .54 | 561 541 (50) | <.001 | 1790 (39) | 18 424 (48) | <.001 | 283 114 (58) | <.001 |
—, not applicable.
P value determined from χ2 test for DTC telemedicine versus urgent care visits.
P value determined from χ2 test for DTC telemedicine versus PCP visits.
Visits with antibiotic prescribing include those categorized as guideline-concordant use, guideline nonconcordant use, and unnecessary antibiotic use.
Percentages reported are the proportion of visits in each setting meeting the specified criteria divided by the total number of ARI visits at that setting (indicated by the top row).
Visits with guideline-concordant antibiotic management include those categorized as guideline-concordant antibiotic use, no antibiotic use (for acute otitis media and sinusitis), and guideline-concordant nonuse of antibiotics.
Antibiotic Prescribing in Matched Visits
After matching, the weighted sample included 4604 DTC telemedicine, 38 408 urgent care, and 485 201 PCP visits with improved balance on observed characteristics, including child age, geographic region, and visit diagnosis (Table 1, Supplemental Table 8).
At DTC telemedicine visits, children were more likely to receive an antibiotic compared with matched visits at other settings (52% vs 42% urgent care; 31% PCP; P < .001 for both comparisons; Table 2). At DTC telemedicine visits, children were less likely to receive guideline-concordant antibiotic management (59% vs 67% urgent care; 78% PCP; P < .001 for both comparisons). The difference in guideline-concordant antibiotic management rates was primarily driven by antibiotic prescribing for visits with viral ARI diagnoses that do not warrant antibiotics. For such visits, antibiotics were appropriately not prescribed in only 54% of DTC telemedicine visits compared with 66% of urgent care visits (P < .001) and 80% of PCP visits (P < .001; Table 3).
Antibiotic Prescribing Outcomes by Diagnosis
| Pediatric Visits | |||||
|---|---|---|---|---|---|
| DTC Telemedicine | Urgent Care | Pa | PCP | Pb | |
| Total ARI Visits, N | 4604 | 38 408 | — | 485 201 | — |
| Summative categories | |||||
| Visits for which antibiotics may be indicated, n | 1265 | 10 553 | — | 133 314 | — |
| Any antibiotic use, n (%) | 832 (66) | 6766 (64) | .25 | 83 914 (63) | .038 |
| Guideline-concordant antibiotic management,c n (%) | 922 (73) | 7364 (70) | .023 | 93 145 (70) | .020 |
| Sinusitis visits, n | 646 | 5389 | — | 68 080 | — |
| Any antibiotic use, n (%) | 410 (63) | 3415 (63) | .96 | 43 219 (63) | .99 |
| Guideline-concordant antibiotic management, n (%) | 471 (73) | 3837 (71) | .36 | 49 611 (73) | .98 |
| Streptococcal pharyngitis visits, n | 250 | 2086 | — | 26 347 | — |
| Any antibiotic use, n (%) | 178 (71) | 1403 (67) | .21 | 17 315 (66) | .069 |
| Guideline-concordant antibiotic management, n (%) | 136 (54) | 974 (47) | .021 | 11 424 (43) | <.001 |
| Otitis media visits, n | 369 | 3078 | — | 38 888 | — |
| Any antibiotic use, n (%) | 244 (66) | 1948 (63) | .28 | 23 380 (60) | .019 |
| Guideline-concordant antibiotic management, n (%) | 315 (85) | 2553 (83) | .24 | 32 109 (83) | .16 |
| Visits where antibiotics not indicated, n | 3339 | 27 855 | — | 351 887 | — |
| Guideline-concordant antibiotic management, n (%) | 1790 (54) | 18 424 (66) | <.001 | 283 114 (80) | <.001 |
| Bronchiolitis visits, n | 51 | 263 | — | 4037 | — |
| Guideline-concordant antibiotic management, n (%) | 31 (61) | 182 (69) | .25 | 3197 (79) | .001 |
| Bronchitis visits, n | 363 | 2686 | — | 27 308 | — |
| Guideline-concordant antibiotic management, n (%) | 160 (44) | 1114 (41) | .344 | 11 270 (41) | .28 |
| Visits for other viral URIs, n | 2925 | 24 906 | — | 320 542 | — |
| Guideline-concordant antibiotic management, n (%) | 1599 (55) | 17 128 (69) | <.001 | 268 647 (84) | <.001 |
| Visits where antibiotics prescribed, n | 2381 | 16 198 | — | 152 687 | — |
| Guideline-concordant antibiotic management, n (%) | 561 (24) | 4260 (26) | .005 | 52 776 (35) | <.001 |
| Pediatric Visits | |||||
|---|---|---|---|---|---|
| DTC Telemedicine | Urgent Care | Pa | PCP | Pb | |
| Total ARI Visits, N | 4604 | 38 408 | — | 485 201 | — |
| Summative categories | |||||
| Visits for which antibiotics may be indicated, n | 1265 | 10 553 | — | 133 314 | — |
| Any antibiotic use, n (%) | 832 (66) | 6766 (64) | .25 | 83 914 (63) | .038 |
| Guideline-concordant antibiotic management,c n (%) | 922 (73) | 7364 (70) | .023 | 93 145 (70) | .020 |
| Sinusitis visits, n | 646 | 5389 | — | 68 080 | — |
| Any antibiotic use, n (%) | 410 (63) | 3415 (63) | .96 | 43 219 (63) | .99 |
| Guideline-concordant antibiotic management, n (%) | 471 (73) | 3837 (71) | .36 | 49 611 (73) | .98 |
| Streptococcal pharyngitis visits, n | 250 | 2086 | — | 26 347 | — |
| Any antibiotic use, n (%) | 178 (71) | 1403 (67) | .21 | 17 315 (66) | .069 |
| Guideline-concordant antibiotic management, n (%) | 136 (54) | 974 (47) | .021 | 11 424 (43) | <.001 |
| Otitis media visits, n | 369 | 3078 | — | 38 888 | — |
| Any antibiotic use, n (%) | 244 (66) | 1948 (63) | .28 | 23 380 (60) | .019 |
| Guideline-concordant antibiotic management, n (%) | 315 (85) | 2553 (83) | .24 | 32 109 (83) | .16 |
| Visits where antibiotics not indicated, n | 3339 | 27 855 | — | 351 887 | — |
| Guideline-concordant antibiotic management, n (%) | 1790 (54) | 18 424 (66) | <.001 | 283 114 (80) | <.001 |
| Bronchiolitis visits, n | 51 | 263 | — | 4037 | — |
| Guideline-concordant antibiotic management, n (%) | 31 (61) | 182 (69) | .25 | 3197 (79) | .001 |
| Bronchitis visits, n | 363 | 2686 | — | 27 308 | — |
| Guideline-concordant antibiotic management, n (%) | 160 (44) | 1114 (41) | .344 | 11 270 (41) | .28 |
| Visits for other viral URIs, n | 2925 | 24 906 | — | 320 542 | — |
| Guideline-concordant antibiotic management, n (%) | 1599 (55) | 17 128 (69) | <.001 | 268 647 (84) | <.001 |
| Visits where antibiotics prescribed, n | 2381 | 16 198 | — | 152 687 | — |
| Guideline-concordant antibiotic management, n (%) | 561 (24) | 4260 (26) | .005 | 52 776 (35) | <.001 |
—, not applicable.
P value determined from χ2 test for DTC telemedicine versus urgent care visits.
P value determined from χ2 test for DTC telemedicine versus PCP visits.
Visits with guideline-concordant antibiotic management include those categorized as guideline-concordant antibiotic use, no antibiotic use (for acute otitis media and sinusitis visits), and guideline-concordant nonuse of antibiotics.
Streptococcal Testing and Return Visits
In the matched sample, among visits with a diagnosis of streptococcal pharyngitis, streptococcal testing was performed for 4% of DTC telemedicine visits compared with 75% of urgent care and 68% of PCP visits (P < .001 for both; Table 4). Subsequent visits within 2 days occurred for a slightly higher percentage of DTC telemedicine visits (5% vs 2% urgent care; 1% PCP; P < .001 for both).
Streptococcal Testing and Return Visits in Matched Sample, 2015–2016
| Pediatric Visits | |||||
|---|---|---|---|---|---|
| DTC Telemedicine | Urgent Care | Pa | PCP | Pb | |
| Visits with diagnosis of streptococcal pharyngitis, N | 250 | 2086 | — | 26 347 | — |
| Streptococcal testing performed, n (%) | 9 (4) | 1557 (75) | <.001 | 17 818 (68) | <.001 |
| Total ARI visits, N | 4604 | 38 408 | — | 485 201 | — |
| Return ARI visit within 2 d, n (%) | 226 (5) | 869 (2) | <.001 | 5875 (1) | <.001 |
| Return ARI visit within 21 d, n (%) | 525 (11) | 3719 (10) | <.001 | 45 629 (9) | <.001 |
| Pediatric Visits | |||||
|---|---|---|---|---|---|
| DTC Telemedicine | Urgent Care | Pa | PCP | Pb | |
| Visits with diagnosis of streptococcal pharyngitis, N | 250 | 2086 | — | 26 347 | — |
| Streptococcal testing performed, n (%) | 9 (4) | 1557 (75) | <.001 | 17 818 (68) | <.001 |
| Total ARI visits, N | 4604 | 38 408 | — | 485 201 | — |
| Return ARI visit within 2 d, n (%) | 226 (5) | 869 (2) | <.001 | 5875 (1) | <.001 |
| Return ARI visit within 21 d, n (%) | 525 (11) | 3719 (10) | <.001 | 45 629 (9) | <.001 |
—, not applicable.
P value determined from χ2 test for DTC telemedicine versus urgent care visits.
P value determined from χ2 test for DTC telemedicine versus PCP visits.
Stratified Analysis by Patient Characteristics
Compared with urgent care and PCP visits, DTC telemedicine visits had significantly increased antibiotic prescribing and decreased guideline-concordant antibiotic management within each category of age and chronic medical complexity (Fig 1). The 1 exception was that DTC telemedicine and urgent care results were not statistically different for the youngest children (0–1 year old), for which the sample size was small. Interaction terms indicated that the degree of difference between DTC telemedicine and PCP visits varied significantly with age for antibiotics prescribed (interaction term P < .001) and guideline concordance (interaction term P = .004); other interaction terms were not significant.
Antibiotic prescribing and guideline-concordant antibiotic management for DTC telemedicine compared with urgent care (UC) and PCP visits, stratified by child age and chronic medical complexity. A, Antibiotic prescribing associated with DTC telemedicine visits compared with UC and PCP visits, stratified by 4 levels of child age (left) and 3 levels of chronic medical complexity (right). B, Guideline-concordant antibiotic management associated with DTC telemedicine visits compared with UC and PCP visits, stratified by 4 levels of child age (left) and 3 levels of chronic medical complexity (right). Within each age or medical complexity category, the values for UC and PCP visits differed significantly from DTC telemedicine with 1 exception: DTC telemedicine and UC results were not statistically different for the youngest children (0–1 year old). Interaction terms indicated that the degree of difference between DTC telemedicine and PCP visits varied significantly with age for antibiotics prescribed (interaction term P < .001) and guideline concordance (interaction term P = .004); other interaction terms were not significant.
Antibiotic prescribing and guideline-concordant antibiotic management for DTC telemedicine compared with urgent care (UC) and PCP visits, stratified by child age and chronic medical complexity. A, Antibiotic prescribing associated with DTC telemedicine visits compared with UC and PCP visits, stratified by 4 levels of child age (left) and 3 levels of chronic medical complexity (right). B, Guideline-concordant antibiotic management associated with DTC telemedicine visits compared with UC and PCP visits, stratified by 4 levels of child age (left) and 3 levels of chronic medical complexity (right). Within each age or medical complexity category, the values for UC and PCP visits differed significantly from DTC telemedicine with 1 exception: DTC telemedicine and UC results were not statistically different for the youngest children (0–1 year old). Interaction terms indicated that the degree of difference between DTC telemedicine and PCP visits varied significantly with age for antibiotics prescribed (interaction term P < .001) and guideline concordance (interaction term P = .004); other interaction terms were not significant.
Sensitivity Analyses
When no longer classifying “no antibiotic use” as guideline-concordant for otitis media and sinusitis, rates of guideline-concordant antibiotic management remained lower overall at DTC telemedicine (51% vs 59% urgent care; 69% PCP; P < .001 for both; Supplemental Table 9). Analysis excluding index visits with follow-up visits within 2 days did not differ meaningfully from main results.
Discussion
Children treated for ARIs at commercial DTC telemedicine visits were substantially more likely to receive an antibiotic and less likely to receive guideline-concordant antibiotic management than children presenting to urgent care or primary care. These quality differences were observed in metrics agnostic to visit diagnosis (antibiotic prescribing rate) and conditional on diagnosis (guideline-concordant antibiotic management). These differences were primarily driven by increased use of antibiotics for visits receiving viral diagnoses.
These differences in antibiotic prescribing for children contrast with previous studies of DTC telemedicine quality among adult patients in which quality differences have been smaller or nonexistent.9,10 For example, using data from the same health plan, we observed similar antibiotic management for adults seeking care for ARIs in these 3 settings.32 Professional groups such as the American Academy of Pediatrics and the American Telemedicine Association have previously raised concerns about DTC telemedicine care outside of the medical home for pediatric patients.7,8 Our results support these concerns and underscore the importance of pediatric-specific evaluation and guidelines. Targeted improvement initiatives have the potential to substantially improve the quality of care delivered via DTC telemedicine.33 Pediatric-specific guidelines and metrics should be incorporated into such efforts.
Several mechanisms could contribute to increased antibiotic prescribing and decreased guideline concordance in pediatric DTC telemedicine visits. First, the potentially rich information that telemedicine can provide in other models of telemedicine, such as when the patient is at a facility designed for telemedicine consultation, is relatively limited in DTC telemedicine. Specifically, the information transmitted in DTC telemedicine is limited in that personal devices used by parents for DTC telemedicine visits rarely incorporate peripheral attachments designed to enhance remote visits (eg, tele-otoscopy) and also do not incorporate trained telepresenters (ie, nurses or medical assistants with the patient). The limited availability of otoscopy and streptococcal testing in DTC telemedicine may contribute to the different diagnosis mix at DTC telemedicine (increased viral ARIs and sinusitis and decreased otitis media and streptococcal pharyngitis) compared with in-person settings. Additionally, personal devices may vary in the quality of microphones, cameras, and Wi-Fi, and some DTC telemedicine visits are completed by using only telephone. These issues reduce the data available to the treating DTC telemedicine provider, which may be of greater concern in pediatric care because of the more limited ability of children to communicate symptoms. Second, there may be differential expectations for antibiotics among children and parents who use DTC telemedicine versus in-person care. Although we matched on a number of observed variables, parental expectations may still differ across settings in ways that we were unable to address. Third, DTC telemedicine visits outside of the medical home lack 3 types of continuity: informational (lack of medical records), relationship (lack of ongoing relationship between the provider and patient), and clinical management (lack of opportunity to manage the patient’s care over time),34 which has the potential to impact quality of care. However, this is unlikely to be the only explanation given that authors of previous evaluations of retail-based clinics, which are also a model of in-person care with limited continuity, have found comparable quality to physician office visits.15,19 Fourth, the majority of commercial DTC telemedicine visits are for adults, such that DTC telemedicine providers may have variable pediatric training or experience, which could impact providers’ knowledge of and adherence to guideline-based pediatric care.35 Prior studies indicate that only 9.6% of urgent care centers employ any pediatricians,36 and pediatricians may also be underrepresented in DTC telemedicine. Together, these issues may increase clinical uncertainty during pediatric DTC telemedicine visits, prompting physicians to prescribe antibiotics “just to be safe.”37,38
We note additional quality concerns raised by our analysis beyond antibiotic use. First, the American Telemedicine Association has recommended that commercial DTC telemedicine should not be used for children <2 years old.8 We note that 5% of studied DTC telemedicine visits for ARIs were within this age group but that gaps in quality existed for children of all ages. Second, accurate diagnosis of otitis media in children requires visualization of the tympanic membranes,11 and accurate diagnosis of streptococcal pharyngitis requires streptococcal testing.27 Making these diagnoses via telemedicine in the absence of reliable visualization and testing could in itself be considered a quality concern. Third, increased rates of follow-up visits after DTC telemedicine raise questions about the effectiveness and system-level efficiency of these visits.
Quality of care for ARIs may be better in other models of acute telemedicine care. For example, acute telemedicine visits in which peripheral devices such as tele-stethoscopes or tele-otoscopes (such as offered at telemedicine kiosks39) are integrated may allow for enhanced examination compared with DTC telemedicine. Using telemedicine technology to connect patients to their PCP’s office would allow telemedicine use within the context of continuity of information, relationships, and care management.7 School-based or day care–based models of acute telemedicine can be used to integrate peripherals and telepresenters.40,41 Finally, pediatric-specific improvement initiatives may also result in improved quality.33 Examining the variation in quality across different models of acute telemedicine care may inform optimal use of technology for acute pediatric concerns.
Although our focus was DTC telemedicine, the quality of antibiotic prescribing was also lower at urgent care visits compared with PCP visits (59% vs 67% guideline concordant). Urgent care clinics have grown rapidly in the United States,42 and urgent care and DTC telemedicine share some characteristics that might impede quality, such as limited continuity. Across all 3 settings, our findings emphasize the need for improvement in guideline-concordant antibiotic use and ongoing antibiotic stewardship efforts in outpatient settings.43,–48
Our analysis had several limitations. As an analysis of insurance claims, we did not have additional sociodemographic or clinical data and thus could not account for variables such as patient race, severity of illness, time constraints, or family expectations, which might drive differences in prescribing.38,49 Given the data available, we could not independently confirm the diagnosis made. The differences in diagnosis mix in the unmatched data could be due to true differences in case mix (perhaps due to self-selection to specific settings on the part of parents) or biases in diagnosis (perhaps due to differences in information available to clinicians or differences in desire to select a diagnosis that justifies prescription of antibiotics). Insurance claims also do not have information on patient allergies. Although we would not expect substantial differences in rates of allergies across settings, we may be underestimating overall guideline concordance in the absence of allergy data. Also, because some children were enrolled for brief periods, we may not be capturing all children with higher medical complexity. However, we confirmed that identified rates of chronic complex disease in the full population of pediatric health plan beneficiaries (6.4%) were relatively similar to rates identified in other population studies (6% in previous claims analysis29). We used claims data rather than electronic health record or provider-reported data. Although this means we examined antibiotics filled rather than prescribed, in-person visit antibiotic use appeared consistent with previous studies of antibiotic use. Specifically, our rates of antibiotic use among unmatched PCP visits were comparable to previous studies for diagnoses for which antibiotics may be indicated (59% vs 50%–80% in previous literature) and diagnoses for which antibiotics are not indicated (16% vs 20%–21% in previous literature).14,20,50 We did not include retail-based clinics in our analyses. Authors of other studies have found that among adults, retail-based clinics have higher rates of guideline-concordant care relative to PCPs.15,19 Physician specialty was not available for DTC telemedicine visits, so we could not examine whether pediatric-specific physician training explained any quality differences. We note also that our analysis is limited to a specific health plan and visits by its members to their contracted DTC telemedicine vendor, and our analysis also did not include Medicaid beneficiaries. Antibiotic prescribing among other DTC telemedicine companies, models, and populations may differ. Understanding the impact of physician training and telemedicine models on quality of care during DTC telemedicine should be a priority in future work.
Conclusions
Children who receive care via DTC telemedicine visits were significantly more likely to receive antibiotics and less likely to receive guideline-concordant antibiotic management than children who visit either urgent care or PCPs.
Dr Ray designed the study, supervised analysis, interpreted the data, and drafted the manuscript; Ms Shi conducted the analyses and interpreted the data; Dr Poon assisted with analysis and interpreted the data; Drs Gidengil and Uscher-Pines assisted with the study design and interpreted the data; Dr Mehrotra designed the study, acquired the data, supervised analysis, and interpreted the data; and all authors critically revised the manuscript, approved the final manuscript as submitted, and agree to be accountable for the work.
FUNDING: Supported in part by grants from the Eunice Kennedy Shriver National Institute of Child Health and Human Development (K23HD088642; Dr Ray) and gifts from Melvin Hall (Dr. Mehrotra). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication. Funded by the National Institutes of Health (NIH).
COMPANION PAPER: A companion to this article can be found online at www.pediatrics.org/cgi/doi/10.1542/peds.2019-0631.
- ARI
acute respiratory infection
- DTC
direct-to-consumer
- HDHP
high-deductible health plan
- ICD-9-CM
International Classification of Diseases, Ninth Revision, Clinical Modification
- ICD-10-CM
International Classification of Diseases, 10th Revision, Clinical Modification
- PCP
primary care provider
- URI
upper respiratory infection
Comments
Immediate Care, Telehealth and Antibiotic Stewardship
In a recent Pediatrics publication, titled “Antibiotic Prescribing during Pediatric DTC telemedicine”, Ray and colleagues concluded that pediatric telemedicine visits resulted in greater overall antibiotic prescribing rates and less adherence to evidence-based guidelines (2019). Media then publicized that telemedicine was associated with inappropriate prescribing of antibiotics, unnecessary costs and suggested patients were being exposed to needless harm.
We feel this study did not accurately reflect today’s modern ambulatory telemedicine providers, and the media representation was inaccurate. As a result, the media missed the opportunity to show the importance of telemedicine for the millions of Americans who can now gain access to quality healthcare where they might otherwise have not received care at all... Telemedicine is not a treatment, it is simply another care delivery model, and it is the provider who determines the treatment. Media coverage implies that the same doctor would prescribe differently based on setting- office versus telemedicine. There is simply no evidence for this.
This study is not representative of today’s telehealth provider. Challenges include, relying on older claims data (2015-2016) from an unknown source, small sample size comparing <5000 telehealth visits to >400,000 PCP visits, provider maturity and experience. The researchers used self-defined metrics around antibiotic utilization combining appropriate and inappropriate utilization rather than the well-defined HEDIS metrics on antibiotic avoidance. And finally, it is important to recognize in that over the past 3-4 years, established telehealth providers have greatly evolved their quality control processes.
Separate from the Pediatrics study, other research study results may add additional context and help provide a better understanding of telemedicine practice. Regardless of location, telemedicine, emergency department, urgent care or primary care provider, prescribing rates for antibiotics associated with URI can range from 2-75% with high variability in prescribing rates being more indicative of individual provider than delivery location. (Mangione-Smith, 2005).
Another study, from Chan et al., researched determinants of antibiotic prescribing in ED’s despite good primary care access identifying multiple factors impacting prescribing rates of antibiotics for URI, including patient- provider relationship factors and expectations, perceived practice norms, reliance on clinical knowledge, clinical uncertainty and patient education (2019). Similarly, McKay et al.’s systematic review of factors related to prescribing antibiotics for respiratory infections unveiled four domains that impact prescribing rates, including multiple patient, provider and geographical factors (2016). These studies verify that multiple factors impact antibiotic prescribing rates.
Adherence to guidelines, another area identified in Ray’s Pediatrics article, cited telemedicine visits to be less adherent to evidence based practice. Their analysis, based on claims data illuminates the basic assumption of accurate coding. A review of data elements from the history of present illness section, though more difficult to do, can support a more accurate picture. Using chart review methodology, Halpren-Ruder showed that the same providers working in an emergency department, urgent care or on telemedicine were at least as likely, if not more likely to adhere to Choosing Wisely guidelines for sinusitis when using telemedicine. Mangione-Smith also found that support systems, including EHR templates and decision aids, common at most academic medical centers, are missing at small, rural organizations.
At our various organizations, we all offer direct-to-consumer telemedicine and have numerous structures in place to assure high quality care for adult and pediatric patients. A robust infrastructure linking national guidelines to organizational guidelines, structured clinical documentation templates with queryable data elements support data monitoring, guideline concordance and identification of gaps in performance. The resulting antibiotic utilization and appropriateness rates are often the same or better than national standards.
It is important to recognize that it is not telemedicine that is at the root of inappropriate antibiotic prescribing, but the structures and processes promoting antibiotic stewardship that require improvement in any setting. Telemedicine is merely a modality augmenting care delivery.
Matthew Faiman, MD, MBA, FACP
Medical, Director, Express Care Online
Digital Health, ITD
Clinical Assistant Professor of Medicine
Cleveland Clinic
Lyle Berkowitz, MD, FACP, FHIMSS
President / MDLIVE Medical Group
Associate Professor of Internal Medicine, Feinberg School of Medicine at Northwestern University
Judd E. Hollander, MD
Senior Vice President for Healthcare Delivery Innovation, TJU
Associate Dean for Strategic Health Initiatives, Sidney Kimmel Medical College
Vice Chair for Finance and Healthcare Enterprises
Department of Emergency Medicine
Thomas Jefferson University
Joanne Murren-Boezem, MD, MPH
Medical Director Nemours CareConnect
Elizabeth A. Krupinski, PhD, FSIIM, FSPIE, FATA, FAIMBE
Professor & Vice Chair for Research
Department of Radiology & Imaging Sciences
Emory University
1364 Clifton Rd NE D107
Atlanta, GA 30322
Ann Mond Johnson, MHA, MBA
CEO
American Telemedicine Association
RE: Ray KN, Shi Z, Gidengil CA, et al. Antibiotic Prescribing in Pediatric Direct-to-Consumer Telemedicine Visits: Implications. Pediatrics. 2019;143(5):e20182491
Ray and colleagues published compelling and concerning findings regarding antibiotic prescribing associated with commercial, direct-to-consumer telemedicine (DTC telemedicine). In comparisons of matched samples of children with acute respiratory infections (ARIs) seen in urgent care, in primary care and via DTC telemedicine, children seen via DTC telemedicine were substantially more likely to receive antibiotics than those seen in either of the two comparison settings. Also, “guideline-concordant” antibiotic use was substantially lower at DTC telemedicine visits than in other settings (59% vs. 67% vs. 78% with DTC telemedicine, urgent care, and PCP visits, respectively). On this basis, authors raise important concerns about antibiotic overuse with DTC telemedicine.
It is important to recognize, however, that a range of different telemedicine models exist and that these models differ distinctly in the type and quality of information that can be acquired to support medical decision making. These distinct differences hold important implications for quality of care. In their definition of DTC telemedicine, authors include “visits in which patients are connected to physicians outside of the medical home through audio only or audio-video conferencing on their personal devices.” Thus, it is clear that observations critical to determining presence of the most common conditions for which antibiotic use is often (e.g., acute otitis media) or always (e.g., Streptococcal pharyngitis) warranted are not obtainable. To diagnose acute otitis media, one must, of course, obtain a good look at the tympanic membrane. To diagnose Streptococcal pharyngitis, one should at least include capacity to perform a rapid Streptococcal antigen test and, ideally, capacity to confirm negative rapid Strep tests with a throat culture.
Some telemedicine models include capacity to obtain a broad range of diagnostic quality information, including high quality images of tympanic membranes as well as simple laboratory testing.1,2,3 Such models offer both the convenience of DTC telemedicine and quality of care consistent with guidelines endorsed by the American Academy of Pediatrics.
1. McConnochie KM. Potential of telemedicine in pediatric primary care. Pediatrics in Review. September 2006, online edition. American Academy of Pediatrics, Elm Grove, IL.
2. McConnochie KM, Wood NE, Alarie C, Ronis S. Care offered by an information-rich pediatric acute illness connected care model. Telemedicine and e-Health 2016; DOI: 10.1089/ tmj.2015.0161/22(6):1-7.
3. Ronis SD, McConnochie KM, Wang H, Wood NE.. Urban telemedicine enables equity in access to acute illness care. Telemedicine and e-Health. 2017;23:105-112. doi: 10.1089/tmj.2016.0098. Epub 2016 Jul 6.
RE: Pediatric Direct-to-Consumer Telemedicine Visits
We appreciated the recent article on “Antibiotic Prescribing During Pediatric Direct-to-Consumer Telemedicine Visits,”1 and applaud the authors for comparing the frequency of antibiotic prescribing across direct to consumer (DTC) telemedicine, urgent care, and primary care settings. In their article, Ray and colleagues noted that when addressing acute respiratory infections, antibiotics were prescribed more frequently during DTC telemedicine visits than other settings (52% with telemedicine; 42% of urgent care; 31% of primary care provider (PCP) visits). Claims data used in the study were extracted from a single national DTC telemedicine vendor. It is unclear whether providers had training in pediatrics. The authors note, “The majority of commercial DTC telemedicine visits are for adults, such that DTC telemedicine providers may have variable pediatric training or experience, which could impact providers’ knowledge of and adherence to guideline-based pediatric care. Prior studies indicate that only 9.6% of urgent care centers employ any pediatricians, and pediatricians may also be underrepresented in DTC telemedicine,” p.7.1 Whereas we agree that these findings are concerning and worthy of study, it is important to emphasize that the identified issues are not endemic to all DTC platforms.
Ray and colleagues note other models of DTC telemedicine where quality may be better than that evidenced in commercial platforms. Omitted in the author’s list is mention of DTC platforms contained within an integrated pediatric health system. Such platforms are more likely to offer providers with extensive training in pediatrics and in the appropriate use of prescription medications to children through these channels.
For example, Nemours Children’s Health System launched its DTC platform (“Nemours CareConnect”) in 2015, employing only board-certified pediatricians. For this team, adherence of antibiotic stewardship and carefully-developed quality metrics are measured via monthly chart reviews for the platform’s pediatricians. Physicians are trained to counsel parents who request medication that is not medically necessary, and to refer patients for whom a telemedicine encounter is not deemed appropriate. Since the 2015 launch, the oral antibiotic prescription rate for Nemours CareConnect is 31.8%. This rate is below that found in the reported study and is comparable to that found when children are examined in PCP offices.
Integrated health system platforms also may offer patients multiple care options. For instance, Nemours CareConnect offers virtual specialty care visits utilizing board-certified pediatric sub-specialists in addition to their DTC urgent care model. The pediatricians as well as the sub-specialists have full access to the patient’s medical history and maintain the integrity of the medical home by sharing the medical progress note with the patient’s primary care provider.
The integrated model offers a way to responsibly incorporate pediatric telemedicine into the child’s medical neighborhood. Research is underway to examine usage characteristics and quality data to elucidate the model’s benefits and challenges. Additional studies examining a variety of key quality indicators across multiple acute care virtual platforms are needed to more clearly elucidate the sources of quality variation in DTC platforms.
References
1. Ray KN, Shi Z, Gidengil CA, et al. Antibiotic prescribing during pediatric direct-to-consumer telemedicine visits [published online ahead of print April 9, 2019]. Pediatrics. doi:10.1542/peds.2018-2491.
RE: Antibiotic Prescribing During Pediatric Direct-to-Consumer Telemedicine Visits
We appreciate the letters from Joshi and from Hersh and colleagues in response to our study. We are in general agreement with the points raised regarding the momentum behind direct to consumer (DTC) telemedicine. Our prior analysis identified rapid increase in DTC telemedicine use over the last 5 years.1 We anticipate that this trend will continue, and for this reason, we believe that this is a crucial time for identifying and promoting best practices in DTC telemedicine, including antibiotic stewardship.
We agree with the letter writers on the importance of bringing antibiotic stewardship best practices into DTC telemedicine. Many antibiotic stewardship methods used in traditional care settings, such as continuing education, providing individual audit and feedback to clinicians, requiring justification of inappropriate antibiotic use, and strengthening clinician shared decision-making skills, are likely to be as effective within telemedicine models as in traditional practices. Indeed, some DTC telemedicine companies have reported improved antibiotic stewardship through such methods,2 and Hersh and colleagues report a comprehensive antimicrobial stewardship program as a key context of their findings.
In addition to these general antibiotic stewardship strategies, another key component of prudent antibiotic stewardship within telemedicine will be identification and referral of clinical scenarios that require in-person diagnostic evaluation, consistent with the concerns noted in the editorial by Gerber.3 Some of these scenarios may be prevalent enough to warrant advice to seek in-person care before even initiating the telemedicine encounter (for example, ear pain in the absence of tele-otoscopy). Other cases requiring in-person evaluation may become apparent during the course of a telemedicine encounter. Acknowledging and planning for these circumstances will also be crucial for high-quality antibiotic stewardship in a telemedicine program. Practically, this may mean developing strategies to facilitate timely follow-up in-person care and avoid additional fees to families, as Hersh and colleagues note. Also important will be avoiding any explicit or implicit incentives that might discourage clinicians from referring to in-person setting when clinically warranted. Setting appropriate expectations with families may be useful as well, including general guidance about clinical scenarios for which telemedicine may be more or less appropriate, and about the possibility that a telemedicine encounter may need to be followed by an in-person visit if clinically warranted.
Finally, the letters underscore the importance of attention to the variation in key features across DTC telemedicine models which may result in variation in the quality of care. Indeed, when standardized patients sought care through virtual visits, guideline concordant treatment for viral pharyngitis and acute rhinosinusitis scenarios varied significantly across eight companies.4 Factors potentially influencing such company-level variation could include clinician training and experience (including pediatric-specific training); continuity with usual care (including informational, management, and relationship continuity); understanding of local contexts of care and referral options; use of peripheral devices (facilitating tele-otoscopic images or tele-stethoscope sounds); and company commitment to continuous improvement in general and antibiotic stewardship specifically. Through careful antibiotic stewardship efforts and transparent data reporting across a range of models and outcomes, we will continue to refine our understanding of best practices for pediatric telemedicine.
References
1. Ray KN, Shi Z, Poon SJ, Uscher-Pines L, Mehrotra A. Use of Commercial Direct-to-Consumer Telemedicine by Children. Acad Pediatr. 2019 [Epub ahead of print].
2. Gali K, Faiman M, Romm S. Ensuring clinical quality in telemedicine. NEJM Catalyst. 2018; https://catalyst.nejm.org/clinical-quality-telemedicine-online-care/. Accessed October 12, 2018.
3. Gerber JS. Need an Antibiotic? There's an App for That. Pediatrics. 2019;143(5).
4. Schoenfeld AJ, Davies JM, Marafino BJ, et al. Variation in Quality of Urgent Health Care Provided During Commercial Virtual Visits. JAMA Intern Med. 2016;176(5):635-42.
RE: Antibiotic Prescribing During Pediatric Direct-to-Consumer Telemedicine Visits
Dear Editor,
We read with great interest the excellent study by Ray and colleagues and insightful accompanying editorial by Gerber addressing the topic of antibiotic use during telemedicine visits. As noted, this is a rapidly growing sector of healthcare delivery for not only children but for patients of all ages. This was a study of antibiotic prescribing practices for acute respiratory infections (ARIs) in a large population of commercially insured children encompassing telemedicine, urgent care and PCP encounters. The authors found that children were more likely to receive antibiotics and when prescribed, less likely to receive guideline-recommended antibiotics in telemedicine visits compared to the other settings. This suggests that antibiotic overuse may be greater in this setting than in the others, supporting concerns about lower quality of care raised by the AAP.
We share concerns with these authors and the AAP about ensuring that the quality of care delivered outside of more traditional clinical settings remain high, especially when outside of the child’s medical home. Intermountain Healthcare (IH) is a large integrated healthcare delivery system in Utah with 23 acute care hospitals and over 170 clinics. Of the 170 clinics, 32 are InstaCare Clinics, providing urgent care services to all ages, and 6 are KidsCare Clinics, providing urgent care services to children ≤ 18 years of age. In addition, since 2016, IH has operated a direct-to-consumer telemedicine platform providing synchronous urgent care services known as Connect Care staffed by Advanced Practice Pactitioners. The purpose is to provide improved access to care for low acuity conditions (e.g., ARIs) while offloading traditional urgent care sites. At IH, we have a comprehensive antimicrobial stewardship program led by one of the authors of this letter (ES) that encompasses inpatient and outpatient settings. Although baseline prescribing rates in our region of the US are lower than national averages, we nonetheless have identified outpatient stewardship, specifically in urgent care and telemedicine encounters, as an important priority for the development and implementation of novel interventions.
A recent analysis of antibiotic use for ARIs in our Connect Care system suggests that our system may be performing differently than that reported by Ray et al. From Aug 2017 through July 2018, we identified at total of 3,967 telemedicine encounters for children ≤ 18 years of age (25% of all telemedicine encounters), including 1,609 for ARIs. In our system, antibiotic prescribing during telemedicine encounters for ARIs was lower than in traditional urgent care settings. Antibiotics were prescribed for ARIs during 27% of telemedicine encounters, compared to 38% of KidsCare encounters and 48% of InstaCare encounters. The top diagnoses associated with antibiotic prescriptions included sinusitis and pharyngitis. No antibiotics were prescribed for otalgia or any other ear related diagnoses codes including otitis media. Amoxicillin, penicillin or amoxicillin/clavulanate were prescribed in 75% of the encounters that received an antibiotic for a respiratory encounter.
We acknowledge that the appropriateness of these telemedicine prescriptions remains uncertain and that the case mix of patients may differ from those reported in the study by Ray. Additionally, as noted by Gerber, the appropriateness of almost any ARI antibiotic prescription initiated via telemedicine is potentially dubious because most diagnoses require either a physical examination (e.g. pneumonia, otitis media) or a test to be performed (pharyngitis caused by group A Streptococcus [GAS]). In our system, GAS testing can be facilitated by Connect Care by using our network of outpatient laboratories and our community pharmacies to dispense antibiotics if tests are positive. However, it remains possible that for some of these patients, testing may not have been indicated in the first place and positive results may represent carriage rather than true infection. In addition, acute otitis media is not diagnosed or treated via Connect Care. Rather, these patients are referred to traditional outpatient settings without incurring an additional fee. We feel that our findings and experience provide an important complement to the study reported by Ray et al. We believe that pediatric telemedicine is likely here to stay. When used in the context of an integrated care delivery system with a shared EMR and robust attention to the Core Elements of Antibiotic Stewardship, direct-to-consumer telemedicine can provide convenient and high-quality care for children.
Adam L. Hersh, MD, PhD
Edward Stenehjem, MD
Will Daines, MD
RE: Antibiotic Prescribing During Pediatric Direct-to-Consumer Telemedicine Visits
Dear Editor,
We read with great interest the excellent study by Ray and colleagues and insightful accompanying editorial by Gerber addressing the topic of antibiotic use during telemedicine visits. As noted, this is a rapidly growing sector of healthcare delivery for not only children but for patients of all ages. This was a study of antibiotic prescribing practices for acute respiratory infections (ARIs) in a large population of commercially insured children encompassing telemedicine, urgent care and PCP encounters. The authors found that children were more likely to receive antibiotics and when prescribed, less likely to receive guideline-recommended antibiotics in telemedicine visits compared to the other settings. This suggests that antibiotic overuse may be greater in this setting than in the others, supporting concerns about lower quality of care raised by the AAP.
We share concerns with these authors and the AAP about ensuring that the quality of care delivered outside of more traditional clinical settings remain high, especially when outside of the child’s medical home. Intermountain Healthcare (IH) is a large integrated healthcare delivery system in Utah with 23 acute care hospitals and over 170 clinics. Of the 170 clinics, 32 are InstaCare Clinics, providing urgent care services to all ages, and 6 are KidsCare Clinics, providing urgent care services to children ≤ 18 years of age. In addition, since 2016, IH has operated a direct-to-consumer telemedicine platform providing synchronous urgent care services known as Connect Care staffed by Advanced Practice Pactitioners. The purpose is to provide improved access to care for low acuity conditions (e.g., ARIs) while offloading traditional urgent care sites. At IH, we have a comprehensive antimicrobial stewardship program led by one of the authors of this letter (ES) that encompasses inpatient and outpatient settings. Although baseline prescribing rates in our region of the US are lower than national averages, we nonetheless have identified outpatient stewardship, specifically in urgent care and telemedicine encounters, as an important priority for the development and implementation of novel interventions.
A recent analysis of antibiotic use for ARIs in our Connect Care system suggests that our system may be performing differently than that reported by Ray et al. From Aug 2017 through July 2018, we identified at total of 3,967 telemedicine encounters for children ≤ 18 years of age (25% of all telemedicine encounters), including 1,609 for ARIs. In our system, antibiotic prescribing during telemedicine encounters for ARIs was lower than in traditional urgent care settings. Antibiotics were prescribed for ARIs during 27% of telemedicine encounters, compared to 38% of KidsCare encounters and 48% of InstaCare encounters. The top diagnoses associated with antibiotic prescriptions included sinusitis and pharyngitis. No antibiotics were prescribed for otalgia or any other ear related diagnoses codes including otitis media. Amoxicillin, penicillin or amoxicillin/clavulanate were prescribed in 75% of the encounters that received an antibiotic for a respiratory encounter.
We acknowledge that the appropriateness of these telemedicine prescriptions remains uncertain and that the case mix of patients may differ from those reported in the study by Ray. Additionally, as noted by Gerber, the appropriateness of almost any ARI antibiotic prescription initiated via telemedicine is potentially dubious because most diagnoses require either a physical examination (e.g. pneumonia, otitis media) or a test to be performed (pharyngitis caused by group A Streptococcus [GAS]). In our system, GAS testing can be facilitated by Connect Care by using our network of outpatient laboratories and our community pharmacies to dispense antibiotics if tests are positive. However, it remains possible that for some of these patients, testing may not have been indicated in the first place and positive results may represent carriage rather than true infection. In addition, acute otitis media is not diagnosed or treated via Connect Care. Rather, these patients are referred to traditional outpatient settings without incurring an additional fee. We feel that our findings and experience provide an important complement to the study reported by Ray et al. We believe that pediatric telemedicine is likely here to stay. When used in the context of an integrated care delivery system with a shared EMR and robust attention to the Core Elements of Antibiotic Stewardship, direct-to-consumer telemedicine can provide convenient and high-quality care for children.
Adam L. Hersh, MD, PhD
Edward Stenehjem, MD
Will Daines, MD
RE: DTC use discouragement is unrealistic despite real concerns about antibtiotics
The Ray et al study of DTC company antibiotic prescribing compared to urgent care and pediatricians’ offices for ARI brought up a legitimate concern about overprescribing antibiotics over telehealth. There were several limitations to this study, eloquently outlined by Jeffrey S. Gerber, which included looking at filled prescriptions, data from only one company, unknown sociodemographic factors and total telehealth visits being a small percentage of overall visits. Regardless, the authors and the commentator are absolutely right; concern about antibiotic prescribing is legitimate. As we deal with the ever-growing threat of antibiotic resistant strains of bacteria, we, as physicians, have had to backtrack and undo the damage created by overprescribing antibiotics, both scientifically as well as culturally.
The study does, however, bring into question the use of DTC telehealth by parents, which is differentiated by pediatrician office run telehealth offered to existing patients. The American Academy of Pediatrics (AAP) has discouraged use of these types of private companies for pediatrics and the ATA also has concern of using them for children under 2. The concerns are that current telehealth practice patterns almost never warrant antibiotic prescription for ARI due to lack of ear exam, xray, or strep test. Some DTC companies don’t have pediatrician staffing, and that patients are lost to follow-up which is detrimental to long term care.
Some of these concerns have easier solutions than others as telehealth has grown considerably over the last five years; there are more physicians working in the space allowing for the right physician seeing the right age group. There’s a recognition that staffing should be similar to other physician practices that include quality assurance, antibiotic stewardship, compliance and recredentialing. All of that can improve antibiotic prescribing patterns.
However, dictating that parents should be discouraged from using DTC is unrealistic. Most of those who have used the service are likely to use it again. Also, it might be an issue of access, which is not limited to having insurance but includes transportation, parents’ ability to take time off work, amount of copay and access to a clinic. Under these circumstances, the alternative to a telehealth visit may be no medical encounter and no care. Ideally all patients could see their own doctor but that is not always feasible based on financial and social constraints. DTC, however, does need to create steps for better quality care for peds until all patients can get the care they need when they need it.
Local telehealth options by pediatricians are going to be the adjunct to primary care in the future. In the meantime, other means of care provided by DTC companies are rapidly spreading and filling gaps. Simply asking parents not to use it won’t be a long-term solution to a very real concern about the quality of virtual healthcare. We have a real responsibility to work within the confines of our current climate which currently includes telehealth and is unlikely to trek backward.
[EDITOR'S NOTE: The second paragraph above was updated on 5/28/2019. The original paragraph read: "The study does, however, bring into question the use of DTC telehealth by parents, which is differentiated by pediatrician office run telehealth offered to existing patients. The American Academy of Pediatrics (AAP) has discouraged use of these companies for pediatrics and the ATA also has concern of using them for children under 2. The concerns are that current telehealth practice patterns almost never warrant antibiotic prescription for ARI due to lack of ear exam, xray, or strep test. Some DTC companies don’t have pediatrician staffing, and that patients are lost to follow-up which is detrimental to long term care."]