Telemedicine is widely used but has uncertain value. We assessed telemedicine to further improve outcomes and reduce costs of comprehensive care (CC) for medically complex children.
We conducted a single-center randomized clinical trial comparing telemedicine with CC relative to CC alone for medically complex children in reducing care days outside the home (clinic, emergency department, or hospital; primary outcome), rate of children developing serious illnesses (causing death, ICU admission, or hospital stay >7 days), and health system costs. We used intent-to-treat Bayesian analyses with neutral prior assuming no benefit. All participants received CC, which included 24/7 phone access to primary care providers (PCPs), low patient-to-PCP ratio, and hospital consultation from PCPs. The telemedicine group also received remote audiovisual communication with the PCPs.
Between August 22, 2018, and March 23, 2020, we randomly assigned 422 medically complex children (209 to CC with telemedicine and 213 to CC alone) before meeting predefined stopping rules. The probability of a reduction with CC with telemedicine versus CC alone was 99% for care days outside the home (12.94 vs 16.94 per child-year; Bayesian rate ratio, 0.80 [95% credible interval, 0.66–0.98]), 95% for rate of children with a serious illness (0.29 vs 0.62 per child-year; rate ratio, 0.68 [0.43–1.07]) and 91% for mean total health system costs (US$33 718 vs US$41 281 per child-year; Bayesian cost ratio, 0.85 [0.67–1.08]).
The addition of telemedicine to CC likely reduced care days outside the home, serious illnesses, other adverse outcomes, and health care costs for medically complex children.
Few previous studies have rigorously addressed the effectiveness or cost-effectiveness of telemedicine for direct patient‐provider communication as an alternative to face‐to‐face care for children with chronic conditions.
In this randomized trial, the Bayesian probability was 91% to 99% that telemedicine with comprehensive care reduced days of care in a medical setting (primary outcome, 12.94 vs 16.94 days per child-year with comprehensive care alone), children with serious illnesses, and health system costs.
Medically complex children1,2 account for only 0.4% of all children but are responsible for ∼40% of pediatric deaths3 and 53% of all pediatric hospital charges.4 All too often, their care is fragmented, ineffective, and inefficient.5,6
To promote prompt effective care for medically complex children at all hours, we developed an outpatient comprehensive care (CC) program at the University of Texas Health Science Center at Houston (UTH) that now includes a hospital consultation service. As a result, CC has substantially reduced serious illnesses, hospital days, PICU days, and health system costs for medically complex children in our center.6–8 Even so, their need for medical attention and the costs of care remain distressingly high.
Optimizing the outcomes and reducing the costs of medically complex children are likely to require an integrated approach to augment care in the home as well as the clinic and hospital. Use of telemedicine to augment their care in the home might be highly beneficial for families of medically complex children, many of whom face financial, transportation, and health system obstacles, particularly for technology-dependent medically complex children. Moreover, any time spent in a medical setting imposes a risk of acquiring serious, even life-threatening, infections. However, there is a dearth of research assessing the implementation and effects of telemedicine for medically complex children.
For these reasons, we conducted a randomized quality improvement (QI) trial comparing CC with telemedicine (treatment hereafter referred to as telemedicine) versus CC alone. Bayesian analyses9–13 were used to assess the overall probability and magnitude of benefit in reducing the total days of care outside the home (primary outcome), serious illnesses, other adverse outcomes, and health system costs.
Methods
Our institutional review board approved the trial as a QI trial14–17 to verify the benefits of an intervention to increase access to care for medically complex children, promote better informed decisions by primary care providers (PCPs) than would otherwise be made over the telephone, and facilitate prompt recognition and treatment of serious illnesses while avoiding unnecessary clinic or emergency department (ED) visits and needless exposures, expense, inconvenience, school absences, and parental time off work.18–20 As allowed under federal regulations for QI studies judged unlikely to increase patient risk,15,17 our institutional review board waived the consent requirements. As in many QI studies, parental consent was not obtained for the QI intervention (telemedicine) being assessed, although of course every parent was aware of their own child’s care, and no child received less care than would have been provided in the absence of the study.
Study Design
This QI study had the design of a hybrid type 1 effectiveness-implementation trial21 in which we worked to fully implement, augment, and verify telemedicine benefits and cost-effectiveness.
Eligibility Criteria
We included patients treated in the High-Risk Children's Clinic (HRCC) at UTH. All had ≥2 hospitalizations, or ≥1 PICU admission in the year before clinic enrollment and a >50% estimated risk of hospitalization in the absence of CC as judged by the clinic’s medical director (R.M.) on the basis of the patient’s diagnoses and clinical course and prospectively validated.6,7 We excluded medically complex children with specific disorders given primary care outside our clinic (eg, serious unrepaired congenital heart disease) or a do-not-resuscitate order.
Randomization
Partly because the number of routine visits depended on age, patients were stratified by age (<2 years or older) as well as estimated baseline risk (risk level 1 [positive-pressure ventilation], risk level 2 [above the expected median risk but not ventilator-dependent], and risk level 3 [at or below the expected median risk]), as judged by the clinic’s medical director (R.M.). All eligible patients already treated in the HRCC were randomly assigned to either telemedicine or CC at the outset of the trial. All later eligible patients were randomly assigned during their first HRCC visit except for infants receiving continued mechanical ventilation after discharge from the NICU, who were randomly assigned within 24 hours after discharge. A study nurse (M.P.), uninvolved in patient care, randomly assigned all patients using a computer-generated algorithm in the Research Electronic Data Capture with variable block sizes. Enrollment occurred between August 22, 2018, and March 23, 2020 (before coronavirus disease 2019 [COVID-19] was identified in the families of any of our patients).
CC
CC was provided to all trial participants to promote prompt effective care at all hours. As previously detailed,6–8 CC included highly experienced PCPs and subspecialists working in the same clinic, 24/7 access to the clinic emergency cell phone that is rotated between the PCPs, same-day care on weekdays in the clinic for acute illnesses and complications, a low patient-to-PCP ratio (≤100:1), weekly scrutiny of the care received before any ED visit or hospitalization, and a hospital consultation service from the CC providers during hospitalizations in our primary teaching hospital, the Children’s Memorial Hermann Hospital (CMHH).8
CC With Telemedicine
Telemedicine was exclusively available to this group. Telemedicine was provided by Zoom for Healthcare, an application compliant to the Health Insurance Portability and Accountability Act that offered reliable connectivity through cellular data for families without Wi-Fi and could be downloaded to any smart phone at no charge. A bilingual staff member assisted each family in the telemedicine group in downloading the application, creating a Zoom account, and learning to use Zoom for audiovisual communication with our PCPs. Several meetings with the telemedicine parents were often required, particularly when new phones were obtained.
The parents in this treatment group were asked if they would like a telemedicine visit whenever they called the clinic for an acute illness. The PCPs chose whether to schedule return visits or health maintenance visits by telemedicine or to communicate with parents using telemedicine at nights and on weekends. At the study outset, telemedicine had not been used in any pediatric clinic in our center, and the parents often declined to have telemedicine visits. In addition, our PCPs who were all experienced in communicating with patients by telephone at all hours7,8 were initially skeptical that telemedicine would add much benefit to CC. In promoting use of telemedicine, the weekly meetings between the medical director (R.M.) and the PCPs and staff included a discussion of any difficulties encountered and potential ways to improve the use of telemedicine for both scheduled follow-up checks and sick calls. He also provided periodic updates of the data to assure the staff of its safety and reviewed the number of telemedicine visits by each PCP.
Blinding
Although our providers and parents could not be blinded to the treatment provided, our statisticians (C.P. and K.L.) and health care economist (E.A.) were unaware of treatment group.
Process Measures
The staff tabulated all clinic visits (including vaccine-only appointments) and telemedicine visits for all study patients. The type of visit, reason for the telemedicine visit, and the intervention recommended after a telemedicine encounter were abstracted from the clinic electronic medical records (EMR).
Outcome Measures
These included the total number of days of care outside of the home (in the clinic, ED, or hospital) (primary outcome), office visits (excluding well-child checks), well-child checks, hospital admissions, readmissions (within 30 days of discharge), PICU admissions, and serious illnesses (causing death, PICU admission, or hospital stay >7 days). To identify clinic and telemedicine visits, we assessed the clinic EMR. Hospital days, ED visits, and hospitalizations were identified by assessing the CMHH EMR and claims data from the 14 hospitals in the Memorial Hermann Hospital System (MHHS). In addition, parents were also asked at each visit about any medical care since previous visit.
A health care economist (E.A.) conducted detailed evaluations of hospital and clinic costs from the health system perspective. For each ED visit and hospitalization in the MHHS, total hospital-specific costs were obtained from the cost-accounting system for these hospitals. Costs for physician services provided at CMHH were estimated by using UTH claims data and relative value units (RVUs) from the 2020 Medicare Fee Schedule.22 The costs at other hospitals outside MHHS (7% of total admissions and 6% of all ED visits) were estimated on the basis of the mean costs of these encounters at MHHS.
Clinic costs for CC were estimated by using the HRCC total expenditures to account for the low patient-to-PCP ratio and the provision of multiple nonbillable services not captured by conventional claims-based methods. The mean personnel time spent providing an office visit, a telemedicine visit, and an inpatient consultation was assessed by surveying the PCPs. The total time spent for each service was then multiplied by the observed number of such encounters and the unit time costs based on the PCPs’ salary, fringes, and overhead costs. The costs for telemedicine included the monthly fees for Zoom for Healthcare and the staff time assisting patients with Zoom installation and use. To capture the full costs of providing CC, the above tabulated costs were subtracted from the total HRCC expenditures and the otherwise unaccounted for CC costs were allocated to patients on the basis of their length of follow-up. Costs for outpatient services occurring at our institution but outside our clinic (eg, diagnostic imaging) were calculated by using the standard RVU-based methodology.22
All costs were inflated to 2020 on the basis of the Consumer Price Index for medical services.23
Statistical Analyses
As in previous trials,8,24,25 we prespecified use of Bayesian analyses to avoid the limitations and the pervasive misinterpretation of P values24,25 and to address the issue of central importance to patients and providers: the probability that an intervention is effective based on previous knowledge or assumptions and the results of the trial.26–28 Although telemedicine was considered likely to be beneficial, we conservatively assumed a neutral prior probability29 centered at a relative risk of 1.0 (no effect on days of care outside home) and a 95% credible interval of 0.3–3.3, encompassing the largest likely effect size for major outcomes in randomized trials. This neutral prior probability was also used for the secondary outcomes.
Intent-to-treat analyses were performed. Negative binomial models were used to assess total number of outcomes. Final models with the lowest k-fold information criterion were chosen after performing k-fold cross validation with k = 10.30 Binary outcomes were analyzed with logistic models and estimates of relative risk derived from the models’ predicted probabilities of outcomes.31 Costs were assessed by using generalized linear mixed models with log-link and γ distribution. To account for the patients who did not incur any hospital-related costs, a zero-inflated negative binomial distribution was used to analyze hospital costs. All models included age and baseline risk factor (stratifying variables) as covariates and length of follow-up as an offset variable. The models for total and clinic costs also included a random intercept for siblings (to account for within-family correlation); we note that for clinical outcomes and hospital costs there was not enough variability to estimate this extra parameter.
REDCap software was used to collect and manage the study data. All the analyses were performed by using R software version 3.6.2. (R Foundation for Statistical Computing) with the brms package.30
Stopping Guideline
To assess the largest sample size feasible with the available resources, we originally planned to enroll almost all eligible patients for 2 years with a goal of accruing ∼400 patients. To avoid unduly prolonging this QI trial, our predefined stopping guideline32 was to discontinue enrollment if an interim Bayesian analyses at the end of the trial’s first year (using a neutral prior) identified a ≥75% posterior probability that telemedicine reduced total care days outside the home.
Results
Enrollment occurred from August 22, 2018, to March 23, 2020, when the first interim analysis identified a 95% probability that telemedicine decreased total days of care outside the home. (Because telemedicine was implemented more slowly than expected [see below], the preplanned interim analysis was deferred from 12 to 19 months after the trial was started. The analysis at that point was prompted by concerns that we should make telemedicine available to all medically complex children during the impending COVID-19 pandemic.).
Excluding 63 ineligible children in our clinic, we randomly assigned 422 medically complex children, 209 to telemedicine and 213 to CC alone (Fig 1). Two medically complex children randomly assigned to telemedicine were unable to install the Zoom application because of lack of Internet access or a smart phone but were included in our intent-to-treat analysis. The 2 groups were similar at baseline (Table 1). A total of 259 child-years of telemedicine and 253 child-years of CC alone were provided. The mean length of follow-up per child was 1.24 years (95% confidence interval [CI], 1.17–1.30) for telemedicine and 1.19 years (95% CI, 1.12–1.26) for CC alone.
Telemedicine Process Measures
Of the 313 telemedicine visits, 51% were follow-up visits (for chronic conditions or after an acute illness, ED visit or hospitalization) and 49% were sick visits. Half the encounters (51%) did not result in any change in care. Of the remaining encounters, 56% involved a pharmacologic change, 20% a nonpharmacologic change, and 24% involved both.
With increasing acceptance of telemedicine by our PCPs and parents, total telemedicine visits increased gradually from 10 in the trial’s first month to 55 in its last month, with a mean of 16 telemedicine visits per month during the study period.
Clinical Outcomes
Table 2 presents clinical outcomes. Despite limited initial use of telemedicine, the probability of an overall reduction with telemedicine versus CC alone was 99% for care days outside the home (12.94 vs 16.94 days per child-year; Bayesian rate ratio [RR], 0.80 [95% credible interval, 0.66–0.98) (Fig 2) and 95% for rate of children with a serious illness (0.29 vs 0.62 per child-year; RR, 0.68 [0.43–1.07]). Except for in-person well-child visits, the findings for all other clinical outcomes consistently favored the telemedicine group with probabilities of a reduction ranging from 66% to 98%, including total hospital admissions (90%) and PICU admissions (97%).
Economic Analyses
Table 3 summarizes the study’s economic analyses. The probability that telemedicine reduced mean total health system costs per child-year was 91% ($33 718 with telemedicine vs $41 281 with CC alone; Bayesian cost ratio, 0.85 [0.67–1.08]), owing mainly to reduced costs for hospital and ED services ($26 759 with telemedicine vs $34 419 with CC alone; cost ratio, 0.77 [0.54–1.10]).
The estimated total costs of telemedicine during the study period were $79 766 (or $308 per child-year), accounting for 4.4% of the total outpatient costs in the telemedicine group; 63% ($50 101) of the telemedicine costs was for the extensive personnel time needed to assist patients with Zoom (25% full-time equivalent [FTE] of a staff member, 11% FTE of a registered nurse, and 2.5% FTE of a research nurse), 33% ($26 498) was for the PCPs’ time providing telemedicine consultations (mean length of telemedicine consultations was 41 minutes), and 4% ($3167) was for expenditures on the Zoom for Healthcare account for 10 hosts.
Discussion
As recently emphasized by Menon and Belcher,33 telehealth has major potential benefits, particularly for disadvantaged children for whom health care may be least accessible. However, the value of telemedicine has been variable and disputed across a broad range of conditions and age groups,34,35 and few randomized trials have assessed the effectiveness or cost-effectiveness of telemedicine to improve the outcomes of children with chronic conditions.
As in an increasing number of clinical trials,9,12,24,29,36 we used Bayesian analyses to estimate the (posterior) probability of benefit or harm from the intervention and to avoid the limitations of the dichotomous frequentist approach that requires either rejecting or failing to reject the null hypothesis based on an arbitrary significance level.9–14 ,26–28 ,37 Recent publications,38 including a commentary in Nature with >800 signatories,27 have called for avoiding dichotomous thinking and abandoning the term statistical significance in the medical literature. We also used a hybrid effectiveness-implementation trial design that allowed us to identify and address the barriers to delivering telemedicine, in particular the initial hesitation among PCPs and the intense clinic staff support needed to ensure family comfort and familiarity with Zoom, while gradually augmenting it and verifying its overall benefits and cost-effectiveness.
Our Bayesian analyses identified a 99% probability of reduced days of care in the clinic, ED, or hospital, a 95% probability of a reduced rate of children with serious illness, and a 91% probability of reduced health system costs. Almost all other outcomes favored the telemedicine group as well. Our adjusted rates and RRs can be considered conservative because the RR values are “shrunk” toward the null and closer to a 1.0 than in frequentist analyses39 because we used a prior Bayesian probability that assumed no treatment benefit (50/50 likelihood of benefit).
Our findings indicate that telemedicine can be safely used with CC and is likely to be beneficial for low income, high-risk medically complex children like ours while providing more convenient and readily accessible care and reducing their exposures in medical settings. Reducing exposures is especially important during the current COVID-19 pandemic33 or other outbreaks of contagious and seasonal illnesses. Our previous trials of CC involving both outpatient and inpatient measures to promote prompt effective care at all times demonstrated reduced adverse outcomes and decreased health system costs among medically complex children.6–8 The current trial indicates that adding telemedicine to augment care in the home for medically complex children is likely to further increase these benefits and could have led to savings of $7563 per child-year for a total of $1.9 million.
The observed benefits from telemedicine might be due to multiple factors, including an enhanced ability of the PCPs to assess medically complex children in their home, a closer PCP-family relationship, increased treatment adherence, reduced delays of the family in seeking medical attention when needed, avoidance of unnecessary exposures, and a greater awareness of the benefits of seeking care from the PCPs rather than ED physicians unfamiliar with their child and more likely to hospitalize medically complex children.
Limitations of this trial include the impossibility to blind the families and PCPs to the intervention. Our trial involved a single center, few clinicians, methods of care, and a population that may differ substantially from those in other centers. The benefits and cost savings with telemedicine can be expected to be less for populations at lower risk than in this trial. Our sample size, although larger than in most telemedicine trials, was limited, and we were unable to adjust for within-family correlation for the clinical outcomes. Finally, a Hawthorne effect40 during the trial may have resulted in greater benefits than if telemedicine was part of routine clinical care for all medically complex children. However, such a Hawthorne effect may have been partially avoided by not specifically informing the families about the trial.
Conclusions
The addition of telemedicine to augment CC for medically complex children in their home was likely to reduce days of treatment outside the home, serious illnesses, other outcomes, and health system costs while increasing access to PCPs. Other studies of telemedicine to augment outpatient care for broader populations of medically complex children are needed.
FUNDING: Supported in part from grant 5KL2TR000370 from the Center for Clinical and Translational Sciences, grant 5 UL1TR00371 from the National Center for Advancing Translational Sciences, a grant from the Texas Medical Center Health Policy Institute, a Learning Healthcare Scholar Award from the University of Texas Health Science Center at Houston, and support from the Network Access Improvement Program. The funding sources had no role in the design, conduct, analyses, interpretation, or the decision to submit results of this study. Partially funded by the National Institutes of Health (NIH).
COMPANION PAPER: A companion to this article can be found online at www.pediatrics.org/cgi/doi/10.1542/peds.2021-051429.
Dr Mosquera takes responsibility for the integrity of the data and accuracy of the data analysis, conceptualization and design of the study, acquisition, analysis, and interpretation of data, drafting of the manuscript, critical revision of the manuscript for important intellectual content, obtaining funding, administrative, technical, and material support, and study supervision; Dr Avritscher takes responsibility for the integrity of the data and accuracy of the data analysis, conceptualization and design of the study, acquisition, analysis, and interpretation of data, drafting of the manuscript, critical revision of the manuscript for important intellectual content, economic analysis, obtaining funding, administrative, technical, and material support, and study supervision; Dr Pedroza takes responsibility for the conceptualization and design of the study, acquisition, analysis, and interpretation of data, drafting of the manuscript, and critical revision of the manuscript for important intellectual content, and statistical and economic analyses; Dr Lee takes responsibility for the acquisition, analysis, and interpretation of data, critical revision of the manuscript for important intellectual content, and statistical analyses; Drs Ramanathan, Harris, Eapen, Yadav, Caldas, Poe, Harting, Ottosen, Gonzalez, and Ms Martinez Castillo take responsibility for the acquisition and interpretation of data, administrative, technical, and material support, and critical revision of the manuscript for important intellectual content; Dr Tyson takes responsibility for conceptualization and design of the study, acquisition, analysis, and interpretation of data, drafting of the manuscript, critical revision of the manuscript for important intellectual content, obtaining funding, administrative, technical, and material support, and study supervision; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
This trial has been registered at ClinicalTrials.gov (identifier NCT03590509).
Deidentified individual participant data will not be made available.
Preliminary results of this trial were presented in the Pediatric Academic Society Conference, March 2020; virtual and were also presented in the AcademyHealth Annual Research Meeting, May 10, 2021; virtual.
- CC
comprehensive care
- CI
confidence interval
- CMHH
Children’s Memorial Hermann Hospital
- COVID-19
coronavirus disease 2019
- ED
emergency department
- EMR
electronic medical record
- FTE
full-time equivalent
- HRCC
High-Risk Children’s Clinic
- MHHS
Memorial Hermann Hospital System
- PCP
primary care provider
- QI
quality improvement
- RR
rate ratio
- UTH
University of Texas Health Science Center at Houston
References
Competing Interests
POTENTIAL CONFLICT OF INTEREST: The authors have no conflict of interest to disclose.
FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.