Clinical Effectiveness of Telemedicine-Based Pediatric Genetics Care

The institutional review board at Children’s Hospital of Philadelphia (CHOP) determined that this study met exemption criteria per 45 CFR 46.104(d) 4(iii). A waiver of Health Information Portability and Accountability Act authorization per 45 CFR 164.512(i)(2)(ii) was granted for accessing identifiable information from the medical records.

This study was performed by the Division of Human Genetics at CHOP, which comprises the sections of Clinical Genetics (including the Individualized Medical Genetics Center and the 22q and You Center) and Metabolism (also known as Biochemical Genetics and including the Mitochondrial Medicine Program). The division has ∼5500 outpatient encounters annually. The periods analyzed for this study were April 1 through October 1, 2019, and April 1 through October 1, 2020. In 2019, the division included 21 attending physicians, 10 fellows, 19 genetic counselors, and 3 advanced practice providers. In 2020, the division included 20 attending physicians, 12 fellows, 24 genetic counselors, and 5 advanced practice providers.

Data extracted from the electronic health record (EHR; Epic Systems) included visit type; patient demographic characteristics (age, sex, race and ethnicity, primary language, zip code, payer); International Classification of Diseases, Tenth Revision (ICD-10), diagnosis codes; and amounts billed and reimbursed for each encounter. Median income by zip code was determined from 2019 US census data. Distance to CHOP was calculated “as the crow flies” from the global positioning system coordinates of the patient’s home address to that of the main hospital. A random amount of deviation between −0.05° and 0.05° was added to figures to protect privacy. Press Ganey score and free-text comments were compiled to assess patient satisfaction.

To assess the diagnostic process in the CHOP Clinical Genetics section, we manually reviewed 2240 new patient encounters during our study periods. For each new patient encounter, we recorded the date and type of test (single gene, single-nucleotide polymorphism microarray, gene panel, and exome) recommended, the date the sample was collected, the date of the test results, and the date results were disclosed to the patient. A test was recommended if a clinician from the division documented their intention to perform it. The date a sample was collected or test results returned was determined from the EHR or from the test report form if results came from an external reference laboratory. The disclosure date was obtained from clinical notes. Finally, we recorded whether the recommended test findings led to a new molecular diagnosis for the patient.

We calculated a metric of test breadth recommended at each initial evaluation as a proxy for the clinician’s confidence in their diagnostic assessment. We asked the clinically trained coauthors to assign each class of test an integer value from most targeted (1) and to most broad (5). We determined the mean value for each test and rounded to the nearest integer. Based on this, fragile X and single-gene testing were assigned 1, panels were assigned 3, microarray was assigned 4, and exome was assigned 5. The breadth of the recommended tests for a given patient was equal to the integer value of the most broad test recommended.

The R statistical language and software environment was used to visualize and analyze data. Sentiment analysis of free-text comments was performed using the “tm” and “syuzhet” packages. Equivalence and noninferiority testing were performed using the “TOSTER” package. Equivalence was claimed if both bounds of the 95% confidence interval (CI) of the proportion of patients receiving a molecular diagnosis was within a predetermined margin of equivalence (−2% to 2%). Noninferiority, which we refer to as comparable throughout the article, was claimed if the lower bound of the 95% CI was within the margin but the upper bound exceeded it.

In April 2020, we rapidly transitioned to telemedicine-based care delivery, with 99% of encounters that month conducted virtually (n = 430 of 433 total; Fig 1A). Overall, 78% of visits in the 2020 study period were conducted virtually compared with 2% in 2019. We found that genetic counselor–only visits in our 2020 study period (n = 206) were nearly quadruple that of the same period in 2019 (n = 53), demonstrating the important role of genetic counselors in staffing telegenetics appointments.

As 2020 progressed, there was variable return to in-person appointments across the division. Although the Clinical Genetics section increased in-person encounters to 53% of overall encounters by September 2020, Metabolism section appointments remained largely virtual (Fig 1B). Of note, >95% of all visits performed exclusively by a genetic counselor remained virtual, regardless of section.

We compared demographic data for patients who were seen virtually, in person, or both during our study periods (Table 1). We found significant differences in patient age, race and ethnicity, preferred language, median income by zip code, and payer type based on encounter type. Interestingly, we did not find a significant difference in distance from the patient’s home to the hospital based on encounter type, although patients evaluated only in person lived an average of 15 km closer to the hospital (Supplemental Fig 5). There was no significant difference in patient sex.

We also explored the distribution of the most common ICD-10 diagnosis categories seen during our study periods and found no evidence for a significant relationship between evaluation method and the 10 most common ICD-10 diagnosis categories (Supplemental Fig 6).

To assess for differences in patient satisfaction levels, we analyzed survey responses for encounters within our study periods (Supplemental Fig 7). We found similar rates of overall satisfaction, with 89.2% of 2019 respondents and 88.4% of 2020 respondents selecting the highest possible score (P = .76, Wilcoxon test). Importantly, 2020 respondents expressed significantly increased satisfaction with access, with 74.3% selecting the top score compared with 60.8% in 2019 (P < .001, Wilcoxon test). Interestingly, we found that patient satisfaction with their care provider was decreased in 2020, with 87.7% rating the top score compared with 91.8% in 2019 (P = .01, Wilcoxon test). We also analyzed the sentiment (negative, neutral, or positive) of respondents’ free-text comments and found no significant difference between the time periods (P = .62, χ² test).

Anecdotally, providers felt that videoconferencing introduced challenges to the dysmorphology physical examination and anthropometric measurement. We wondered how diagnostic uncertainty caused by these limitations might affect recommendations for genetic testing and whether there was a difference in the proportion of new patient evaluations resulting in a molecular diagnosis. Metabolism section encounters were excluded to minimize the influence of newborn screening results and pre-evaluation biochemical testing.

We found that providers who evaluated patients initially by video recommended at least 1 genetic test for 8.6% more patients (79.5% for virtual vs. 70.9% for in-person evaluation, P < .001, Fisher’s exact test; Fig 2A). We found that clinicians completing the evaluation in person were significantly more likely to recommend only the most targeted (single-gene) tests as the initial step of genetic diagnosis (P < .01, Fisher’s exact test with Bonferroni correction; Supplemental Fig 8). However, averaged over all recommended genetic tests, there was no significant difference in the breadth of testing for patients seen in person compared with those seen virtually (Wilcoxon test; Supplemental Fig 8).

Although more total tests were recommended for patients evaluated by video, we found no significant difference in ultimate genetic test completion rate between the 2 evaluation methodologies (51.2% for virtual vs. 55.1% for in-person evaluation; P = .09, Fisher’s exact test; Fig 2A). Similarly, we found no difference in breadth of completed testing (Wilcoxon test, Supplemental Fig 8). However, patients evaluated in person completed testing in a shorter amount of time (P < .001, Kolmogorov-Smirnov test; Fig 2B). Analysis of the steps leading to results disclosure revealed sample collection as the bottleneck, with a DNA sample drawn a median of 40 days sooner for patients seen in person (Fig 2C). For a considerable proportion of these patients, a DNA sample was collected on the day of their visit, whereas genomic studies for telegenetics patients required distribution and return of a saliva collection kit or subsequent presentation to a laboratory.

Given similar test use regardless of encounter method, we wanted to understand potential effects of telehealth on the ultimate molecular diagnosis rate. Strikingly, we found that our overall molecular diagnosis rate for patients seen virtually was comparable to that for patients seen in person (13.8% vs 12.4%, respectively; P = .40, Fisher’s exact test; Supplemental Fig 9).

Given the delay in sample collection for patients undergoing telemedicine evaluations within the Clinical Genetics section, we wondered whether similar delays affected care for established patients with inborn errors of metabolism. We evaluated time to sample collection in patients with maple syrup urine disease (MSUD) and phenylketonuria (PKU), as amino acid levels are used to guide management decisions in both conditions. In-person evaluation permits same-day sample collection, and next-day results are available through our in-house metabolic laboratory. When patients with MSUD were evaluated virtually, plasma samples were collected later, and results were available a median of 13.5 days later (P < .001, Wilcoxon test; Fig 3). In contrast, monitoring for patients with PKU can be achieved through dried blood spots collected on filter paper and mailed to the laboratory. We also identified a significant, but smaller delay in monitoring laboratory results in this patient population (median, 6.5 days; P < .001, Wilcoxon test; Fig 3).

Finally, we asked whether charges and reimbursement amounts were different for in-person versus virtual care encounters (Fig 4). For new patient visits, the median amount charged was $203 higher for in-person encounters, but the median amount reimbursed was only $53 higher. Across the 1235 new video evaluations, this amount represented $65 455 in potentially lost reimbursement. For follow-up appointments, the amounts charged and reimbursed were similar between appointment modalities.

In this retrospective cohort study, we analyzed 1 academic medical center’s experience in the delivery of in-person and virtual clinical care before and after the onset of the COVID-19 pandemic. Telegenetics has long been considered an attractive option to increase patient access to subspecialty care but has remained relatively underexplored in pediatric genetics before 2020.

We found significant differences in patient race and ethnicity, preferred language, median income by zip code, and payer type based on encounter type. Patients evaluated by telehealth were more likely to report non-Hispanic White background, English language preference, living in areas with high median income, and having commercial insurance. These findings are consistent with some previous studies revealing disparities in telemedicine use, particularly during the COVID-19 pandemic.^11,20–22  In historic al clinical contexts, the use of telemedicine has improved access to rural communities and those with lower annual household incomes.^7,12,23–25  Therefore, our findings should be taken in the context of 1 academic medical center during a global pandemic.

In 2020, use of telemedicine allowed us to maintain a consistent patient volume despite the limitations imposed by the beginning of the COVID-19 pandemic. Analysis of the most common ICD-10 diagnosis categories showed no evidence that patients with a particular diagnosis category were being systematically triaged to in-person or telemedicine evaluation. Interestingly, the Clinical Genetics section had increasingly more in-person encounters, representing at least 50% of appointments by October 2020, whereas the Metabolism section continued to deliver care mostly through telemedicine. Most in-person Clinical Genetics section encounters were for new patient appointments; clinicians believed that in-person evaluations generally provided a better opportunity for phenotyping than could be achieved virtually. In contrast, most video appointments for patients with metabolic and mitochondrial disorders were follow-up encounters. Telemedicine was an attractive option for families, as it eliminated challenges of coming to the hospital for care and potential infectious disease exposure that could lead to metabolic decompensation.

Although telemedicine will continue to play an essential role in patient care, it is important to consider the impact of virtual appointments on the acquisition of monitoring laboratories. We found that the time between evaluation and amino acid laboratory monitoring for patients with MSUD and PKU was significantly longer for those seen virtually compared with in-person evaluation. This delay may represent a trend among monitoring laboratories for many metabolic conditions. The lag for patients with PKU was shorter than for those with MSUD. An attractive hypothesis is that the families of patients with PKU had longstanding use of mail-based monitoring through state newborn screening laboratory data, which may have primed them for continued remote monitoring.

Because video evaluation does not allow for a comprehensive dysmorphology physical examination and anthropometric assessment, previous opinion pieces express the community’s hesitation with this medium.⁵  Indeed, the results of modern genetic testing, such as those from exome sequencing, appear to be influenced by the amount and quality of phenotype information submitted with testing requisitions.²⁶  Surprisingly, we found that the molecular diagnosis rate for patients seen virtually was comparable to that achieved for patients seen in person.

Our analysis did not address all potential impediments to sample collection, including steps in prior authorization for genetic testing, notification of testing authorization, out-of-pocket cost for genetic testing, access to diagnostic laboratories for blood draws, or staffing difficulties affecting this process. However, patients had the opportunity to submit saliva samples by mail from the earliest days of the COVID-19 pandemic. Additional studies are needed to better understand impediments to sample collection for diagnostic evaluation imposed by the pandemic.

Other potential limitations of our diagnostic efficacy analysis exist. There was a slight variation in clinicians delivering care, and through our clinical efficacy results, we could only consider diagnoses with a Mendelian genetic etiology and exclude other diagnoses within our purview, including teratogenicity and malformation associations such as vertebral defects, anal atresia, cardiac defects, tracheoesophageal fistula, renal anomalies, and limb abnormalities (VACTERL) or omphalocele, extrophy, imperforate anus, and spinal defects (OEIS).

Historically, one source of hesitation in implementing telemedicine has been limited reimbursement of telehealth services.^27,28  We found only minor differences in reimbursements for in-person and virtual encounters, which likely reflects an increase in insurance coverage of telemedicine during the COVID-19 pandemic. Additionally, we found that providers charged a similar amount or less for video encounters than for in-person care, whereas the diagnostic rate was similar. Together, these data suggest that telemedicine is a clinically and cost-effective mode of care and lend support to continued insurance coverage of telegenetics beyond the current global health crisis.

Unexpectedly, considering the presumed importance of the dysmorphology physical examination, we have found the clinical efficacy of pediatric telegenetics evaluation to be comparable to that of in-person evaluation; however, delays in sample collection may affect timely diagnosis and management of existing conditions. In addition, we have found high levels of patient satisfaction with telehealth and similar levels of reimbursement. Overall, telemedicine appears to be an appropriate care delivery platform for genetics. Our findings may be applicable to other pediatric subspecialties in which physical examination is presumed to be highly important, but diagnostic testing can be broad and accurate, such as endocrinology or rheumatology.

CHOP

Children’s Hospital of Philadelphia

CI

confidence interval

EHR

electronic health record

ICD-10

International Classification of Diseases, Tenth Revision

MSUD

maple syrup urine disease

PKU

phenylketonuria