Telemedicine for Children With Medical Complexity: A Randomized Clinical Trial

Our institutional review board approved the trial as a QI trial^14–17  to verify the benefits of an intervention to increase access to care for medically complex children, promote better informed decisions by primary care providers (PCPs) than would otherwise be made over the telephone, and facilitate prompt recognition and treatment of serious illnesses while avoiding unnecessary clinic or emergency department (ED) visits and needless exposures, expense, inconvenience, school absences, and parental time off work.^18–20  As allowed under federal regulations for QI studies judged unlikely to increase patient risk,^15,17  our institutional review board waived the consent requirements. As in many QI studies, parental consent was not obtained for the QI intervention (telemedicine) being assessed, although of course every parent was aware of their own child’s care, and no child received less care than would have been provided in the absence of the study.

This QI study had the design of a hybrid type 1 effectiveness-implementation trial²¹  in which we worked to fully implement, augment, and verify telemedicine benefits and cost-effectiveness.

We included patients treated in the High-Risk Children's Clinic (HRCC) at UTH. All had ≥2 hospitalizations, or ≥1 PICU admission in the year before clinic enrollment and a >50% estimated risk of hospitalization in the absence of CC as judged by the clinic’s medical director (R.M.) on the basis of the patient’s diagnoses and clinical course and prospectively validated.^6,7  We excluded medically complex children with specific disorders given primary care outside our clinic (eg, serious unrepaired congenital heart disease) or a do-not-resuscitate order.

Partly because the number of routine visits depended on age, patients were stratified by age (<2 years or older) as well as estimated baseline risk (risk level 1 [positive-pressure ventilation], risk level 2 [above the expected median risk but not ventilator-dependent], and risk level 3 [at or below the expected median risk]), as judged by the clinic’s medical director (R.M.). All eligible patients already treated in the HRCC were randomly assigned to either telemedicine or CC at the outset of the trial. All later eligible patients were randomly assigned during their first HRCC visit except for infants receiving continued mechanical ventilation after discharge from the NICU, who were randomly assigned within 24 hours after discharge. A study nurse (M.P.), uninvolved in patient care, randomly assigned all patients using a computer-generated algorithm in the Research Electronic Data Capture with variable block sizes. Enrollment occurred between August 22, 2018, and March 23, 2020 (before coronavirus disease 2019 [COVID-19] was identified in the families of any of our patients).

CC was provided to all trial participants to promote prompt effective care at all hours. As previously detailed,^6–8  CC included highly experienced PCPs and subspecialists working in the same clinic, 24/7 access to the clinic emergency cell phone that is rotated between the PCPs, same-day care on weekdays in the clinic for acute illnesses and complications, a low patient-to-PCP ratio (≤100:1), weekly scrutiny of the care received before any ED visit or hospitalization, and a hospital consultation service from the CC providers during hospitalizations in our primary teaching hospital, the Children’s Memorial Hermann Hospital (CMHH).⁸ 

Telemedicine was exclusively available to this group. Telemedicine was provided by Zoom for Healthcare, an application compliant to the Health Insurance Portability and Accountability Act that offered reliable connectivity through cellular data for families without Wi-Fi and could be downloaded to any smart phone at no charge. A bilingual staff member assisted each family in the telemedicine group in downloading the application, creating a Zoom account, and learning to use Zoom for audiovisual communication with our PCPs. Several meetings with the telemedicine parents were often required, particularly when new phones were obtained.

The parents in this treatment group were asked if they would like a telemedicine visit whenever they called the clinic for an acute illness. The PCPs chose whether to schedule return visits or health maintenance visits by telemedicine or to communicate with parents using telemedicine at nights and on weekends. At the study outset, telemedicine had not been used in any pediatric clinic in our center, and the parents often declined to have telemedicine visits. In addition, our PCPs who were all experienced in communicating with patients by telephone at all hours^7,8  were initially skeptical that telemedicine would add much benefit to CC. In promoting use of telemedicine, the weekly meetings between the medical director (R.M.) and the PCPs and staff included a discussion of any difficulties encountered and potential ways to improve the use of telemedicine for both scheduled follow-up checks and sick calls. He also provided periodic updates of the data to assure the staff of its safety and reviewed the number of telemedicine visits by each PCP.

Although our providers and parents could not be blinded to the treatment provided, our statisticians (C.P. and K.L.) and health care economist (E.A.) were unaware of treatment group.

The staff tabulated all clinic visits (including vaccine-only appointments) and telemedicine visits for all study patients. The type of visit, reason for the telemedicine visit, and the intervention recommended after a telemedicine encounter were abstracted from the clinic electronic medical records (EMR).

These included the total number of days of care outside of the home (in the clinic, ED, or hospital) (primary outcome), office visits (excluding well-child checks), well-child checks, hospital admissions, readmissions (within 30 days of discharge), PICU admissions, and serious illnesses (causing death, PICU admission, or hospital stay >7 days). To identify clinic and telemedicine visits, we assessed the clinic EMR. Hospital days, ED visits, and hospitalizations were identified by assessing the CMHH EMR and claims data from the 14 hospitals in the Memorial Hermann Hospital System (MHHS). In addition, parents were also asked at each visit about any medical care since previous visit.

A health care economist (E.A.) conducted detailed evaluations of hospital and clinic costs from the health system perspective. For each ED visit and hospitalization in the MHHS, total hospital-specific costs were obtained from the cost-accounting system for these hospitals. Costs for physician services provided at CMHH were estimated by using UTH claims data and relative value units (RVUs) from the 2020 Medicare Fee Schedule.²²  The costs at other hospitals outside MHHS (7% of total admissions and 6% of all ED visits) were estimated on the basis of the mean costs of these encounters at MHHS.

Clinic costs for CC were estimated by using the HRCC total expenditures to account for the low patient-to-PCP ratio and the provision of multiple nonbillable services not captured by conventional claims-based methods. The mean personnel time spent providing an office visit, a telemedicine visit, and an inpatient consultation was assessed by surveying the PCPs. The total time spent for each service was then multiplied by the observed number of such encounters and the unit time costs based on the PCPs’ salary, fringes, and overhead costs. The costs for telemedicine included the monthly fees for Zoom for Healthcare and the staff time assisting patients with Zoom installation and use. To capture the full costs of providing CC, the above tabulated costs were subtracted from the total HRCC expenditures and the otherwise unaccounted for CC costs were allocated to patients on the basis of their length of follow-up. Costs for outpatient services occurring at our institution but outside our clinic (eg, diagnostic imaging) were calculated by using the standard RVU-based methodology.²² 

All costs were inflated to 2020 on the basis of the Consumer Price Index for medical services.²³ 

As in previous trials,^8,24,25  we prespecified use of Bayesian analyses to avoid the limitations and the pervasive misinterpretation of P values^24,25  and to address the issue of central importance to patients and providers: the probability that an intervention is effective based on previous knowledge or assumptions and the results of the trial.^26–28  Although telemedicine was considered likely to be beneficial, we conservatively assumed a neutral prior probability²⁹  centered at a relative risk of 1.0 (no effect on days of care outside home) and a 95% credible interval of 0.3–3.3, encompassing the largest likely effect size for major outcomes in randomized trials. This neutral prior probability was also used for the secondary outcomes.

Intent-to-treat analyses were performed. Negative binomial models were used to assess total number of outcomes. Final models with the lowest k-fold information criterion were chosen after performing k-fold cross validation with k = 10.³⁰  Binary outcomes were analyzed with logistic models and estimates of relative risk derived from the models’ predicted probabilities of outcomes.³¹  Costs were assessed by using generalized linear mixed models with log-link and γ distribution. To account for the patients who did not incur any hospital-related costs, a zero-inflated negative binomial distribution was used to analyze hospital costs. All models included age and baseline risk factor (stratifying variables) as covariates and length of follow-up as an offset variable. The models for total and clinic costs also included a random intercept for siblings (to account for within-family correlation); we note that for clinical outcomes and hospital costs there was not enough variability to estimate this extra parameter.

REDCap software was used to collect and manage the study data. All the analyses were performed by using R software version 3.6.2. (R Foundation for Statistical Computing) with the brms package.³⁰ 

To assess the largest sample size feasible with the available resources, we originally planned to enroll almost all eligible patients for 2 years with a goal of accruing ∼400 patients. To avoid unduly prolonging this QI trial, our predefined stopping guideline³²  was to discontinue enrollment if an interim Bayesian analyses at the end of the trial’s first year (using a neutral prior) identified a ≥75% posterior probability that telemedicine reduced total care days outside the home.

Enrollment occurred from August 22, 2018, to March 23, 2020, when the first interim analysis identified a 95% probability that telemedicine decreased total days of care outside the home. (Because telemedicine was implemented more slowly than expected [see below], the preplanned interim analysis was deferred from 12 to 19 months after the trial was started. The analysis at that point was prompted by concerns that we should make telemedicine available to all medically complex children during the impending COVID-19 pandemic.).

Excluding 63 ineligible children in our clinic, we randomly assigned 422 medically complex children, 209 to telemedicine and 213 to CC alone (Fig 1). Two medically complex children randomly assigned to telemedicine were unable to install the Zoom application because of lack of Internet access or a smart phone but were included in our intent-to-treat analysis. The 2 groups were similar at baseline (Table 1). A total of 259 child-years of telemedicine and 253 child-years of CC alone were provided. The mean length of follow-up per child was 1.24 years (95% confidence interval [CI], 1.17–1.30) for telemedicine and 1.19 years (95% CI, 1.12–1.26) for CC alone.

Of the 313 telemedicine visits, 51% were follow-up visits (for chronic conditions or after an acute illness, ED visit or hospitalization) and 49% were sick visits. Half the encounters (51%) did not result in any change in care. Of the remaining encounters, 56% involved a pharmacologic change, 20% a nonpharmacologic change, and 24% involved both.

With increasing acceptance of telemedicine by our PCPs and parents, total telemedicine visits increased gradually from 10 in the trial’s first month to 55 in its last month, with a mean of 16 telemedicine visits per month during the study period.

Table 2 presents clinical outcomes. Despite limited initial use of telemedicine, the probability of an overall reduction with telemedicine versus CC alone was 99% for care days outside the home (12.94 vs 16.94 days per child-year; Bayesian rate ratio [RR], 0.80 [95% credible interval, 0.66–0.98) (Fig 2) and 95% for rate of children with a serious illness (0.29 vs 0.62 per child-year; RR, 0.68 [0.43–1.07]). Except for in-person well-child visits, the findings for all other clinical outcomes consistently favored the telemedicine group with probabilities of a reduction ranging from 66% to 98%, including total hospital admissions (90%) and PICU admissions (97%).

Table 3 summarizes the study’s economic analyses. The probability that telemedicine reduced mean total health system costs per child-year was 91% ($33 718 with telemedicine vs $41 281 with CC alone; Bayesian cost ratio, 0.85 [0.67–1.08]), owing mainly to reduced costs for hospital and ED services ($26 759 with telemedicine vs $34 419 with CC alone; cost ratio, 0.77 [0.54–1.10]).

The estimated total costs of telemedicine during the study period were $79 766 (or $308 per child-year), accounting for 4.4% of the total outpatient costs in the telemedicine group; 63% ($50 101) of the telemedicine costs was for the extensive personnel time needed to assist patients with Zoom (25% full-time equivalent [FTE] of a staff member, 11% FTE of a registered nurse, and 2.5% FTE of a research nurse), 33% ($26 498) was for the PCPs’ time providing telemedicine consultations (mean length of telemedicine consultations was 41 minutes), and 4% ($3167) was for expenditures on the Zoom for Healthcare account for 10 hosts.

As recently emphasized by Menon and Belcher,³³  telehealth has major potential benefits, particularly for disadvantaged children for whom health care may be least accessible. However, the value of telemedicine has been variable and disputed across a broad range of conditions and age groups,^34,35  and few randomized trials have assessed the effectiveness or cost-effectiveness of telemedicine to improve the outcomes of children with chronic conditions.

As in an increasing number of clinical trials,^{9,12,24,29,36}  we used Bayesian analyses to estimate the (posterior) probability of benefit or harm from the intervention and to avoid the limitations of the dichotomous frequentist approach that requires either rejecting or failing to reject the null hypothesis based on an arbitrary significance level.^{9–14 ,26–28 ,37}  Recent publications,³⁸  including a commentary in Nature with >800 signatories,²⁷  have called for avoiding dichotomous thinking and abandoning the term statistical significance in the medical literature. We also used a hybrid effectiveness-implementation trial design that allowed us to identify and address the barriers to delivering telemedicine, in particular the initial hesitation among PCPs and the intense clinic staff support needed to ensure family comfort and familiarity with Zoom, while gradually augmenting it and verifying its overall benefits and cost-effectiveness.

Our Bayesian analyses identified a 99% probability of reduced days of care in the clinic, ED, or hospital, a 95% probability of a reduced rate of children with serious illness, and a 91% probability of reduced health system costs. Almost all other outcomes favored the telemedicine group as well. Our adjusted rates and RRs can be considered conservative because the RR values are “shrunk” toward the null and closer to a 1.0 than in frequentist analyses³⁹  because we used a prior Bayesian probability that assumed no treatment benefit (50/50 likelihood of benefit).

Our findings indicate that telemedicine can be safely used with CC and is likely to be beneficial for low income, high-risk medically complex children like ours while providing more convenient and readily accessible care and reducing their exposures in medical settings. Reducing exposures is especially important during the current COVID-19 pandemic³³  or other outbreaks of contagious and seasonal illnesses. Our previous trials of CC involving both outpatient and inpatient measures to promote prompt effective care at all times demonstrated reduced adverse outcomes and decreased health system costs among medically complex children.^6–8  The current trial indicates that adding telemedicine to augment care in the home for medically complex children is likely to further increase these benefits and could have led to savings of $7563 per child-year for a total of $1.9 million.

The observed benefits from telemedicine might be due to multiple factors, including an enhanced ability of the PCPs to assess medically complex children in their home, a closer PCP-family relationship, increased treatment adherence, reduced delays of the family in seeking medical attention when needed, avoidance of unnecessary exposures, and a greater awareness of the benefits of seeking care from the PCPs rather than ED physicians unfamiliar with their child and more likely to hospitalize medically complex children.

Limitations of this trial include the impossibility to blind the families and PCPs to the intervention. Our trial involved a single center, few clinicians, methods of care, and a population that may differ substantially from those in other centers. The benefits and cost savings with telemedicine can be expected to be less for populations at lower risk than in this trial. Our sample size, although larger than in most telemedicine trials, was limited, and we were unable to adjust for within-family correlation for the clinical outcomes. Finally, a Hawthorne effect⁴⁰  during the trial may have resulted in greater benefits than if telemedicine was part of routine clinical care for all medically complex children. However, such a Hawthorne effect may have been partially avoided by not specifically informing the families about the trial.

The addition of telemedicine to augment CC for medically complex children in their home was likely to reduce days of treatment outside the home, serious illnesses, other outcomes, and health system costs while increasing access to PCPs. Other studies of telemedicine to augment outpatient care for broader populations of medically complex children are needed.

CC

comprehensive care

CI

confidence interval

CMHH

Children’s Memorial Hermann Hospital

COVID-19

coronavirus disease 2019

ED

emergency department

EMR

electronic medical record

FTE

full-time equivalent

HRCC

High-Risk Children’s Clinic

MHHS

Memorial Hermann Hospital System

PCP

primary care provider

QI

quality improvement

RR

rate ratio

UTH

University of Texas Health Science Center at Houston