OBJECTIVES

The coronavirus pandemic created significant, abrupt challenges to the delivery of ambulatory health care. Because tertiary medical centers limited elective in-person services, telehealth was rapidly enacted in settings with minimal previous experience to allow continued access to care. With this quality improvement (QI) initiative, we aimed to achieve a virtual visit volume of at least 75% of our prepandemic volume. We also describe patient and provider experience with telehealth services.

METHODS

Our QI team identified the primary drivers contributing to low telehealth volume and developed a telehealth scheduling protocol and data tracking system using QI-based strategies. Patients and providers were surveyed on their telehealth experience.

RESULTS

At the onset of the pandemic, weekly visit volume dropped by 65% (99 weekly visits; historical average of 281). Over the subsequent 3 weeks, using rapid Plan-Do-Study-Act cycles, we achieved our goal volume. In surveys, it was indicated that most participants had never before used telehealth (71% of patients; 82% of providers) yet reported high satisfaction (90% of patients; 81% of providers). Both groups expressed concern over the lack of in-person assessments. Most respondents were interested in future use of telehealth.

CONCLUSIONS

With a QI-based approach, we successfully maintained access to care via telehealth services for pediatric pulmonary patients during the coronavirus pandemic and found high rates of satisfaction among patients and providers. Telehealth will likely continue to be a part of our health care delivery platform, expanding the reach of our services. Further work is needed to understand the effects on clinical outcomes.

With the onset of the coronavirus disease 2019 (COVID-19) pandemic, the need for health care environments to rapidly adopt strategies to reduce disease spread became clear. The Massachusetts governor announced a state of emergency on March 10, 2020,1  and enacted stay-at-home orders effective March 24, 2020.2  To align with this, Boston Children’s Hospital (BCH) cancelled in-person elective ambulatory care. To maintain access to care, insurance reimbursement for telehealth services was mandated.1  Previously, telehealth had been employed in specialized settings, such as pediatric neurology3,4  and cardiology5  practices serving remote areas as well as in pediatric surgery for perioperative visits6 ; however, it had not yet been widely adopted in US pediatric specialty clinics at the time local stay-at-home orders went into effect. To continue uninterrupted routine care, efficient implementation of telehealth became necessary.

Pediatric pulmonary clinics serve a diverse patient population, composed of patients with both common and rare chronic conditions, including asthma, cystic fibrosis (CF), bronchopulmonary dysplasia (BPD), airway malformations, and interstitial lung disease, as well as those with acute conditions, including complicated pneumonia and pneumothorax. Our patients require frequent follow-up to support ongoing evaluation, assess response to treatment, and provide education around diagnosis and management. For example, routine quarterly care is recommended for all children with CF7  as it is for those with moderate to severe asthma.8 

Telehealth provides the opportunity to assess and provide longitudinal medical management, although it has its limitations. Although in early reports during the COVID-19 pandemic, researchers highlighted that conversion from in-person to telehealth appointments was possible,9  optimal operational systems to support efficient scheduling and effective delivery of telehealth care have not been described. Best practices on the basis of recent experience have been offered by national organizations, including the American Academy of Pediatrics,10,11  although researchers use quality improvement (QI) methodologies to optimize procedures to meet these new demands in few reports.12  QI initiatives are essential to study and improve the rapidly changing health care delivery paradigm.

Patient and provider experiences with this new model of telehealth-based care during COVID-19 have been a focus of recent descriptive studies. In multiple studies, mostly performed in the adult population, researchers describe overall positive experiences by both patients and providers in telehealth clinics.1215  There are no reports, to our knowledge, of patient and provider perceptions of telehealth as experienced in a pediatric pulmonary subspecialty clinic.

When our hospital restricted elective in-person assessments, there was no structured approach to scheduling, organizing, and executing telehealth visits to continue routine care for our patients. Our specific, measurable, applicable, realistic, and timely (SMART) aim starting in mid-March was to increase the number of virtual visits completed in the division of pulmonary medicine to 75% (210 visits) of the historical weekly average ambulatory clinic volume (281 visits) within 6 weeks after the initial shutdown (the end of April 2020). To accomplish this, we used QI principles to inform our approach, recognizing that the call for immediate action somewhat limited our ability to follow more rigorous QI methodologies, a reality supported by others working in this space.16,17  In this article, we present the strategies we adopted, our initial evaluation of provider and patient experiences with telehealth, and lessons learned.

This QI project was driven by changes in social interaction mandated during the COVID-19 pandemic. Before the pandemic, the BCH pediatric pulmonary clinic, staffed by 25 attending physicians, 12 fellows, and 4 nurse practitioners, managed an average weekly clinic volume of 281 provider and 158 pulmonary laboratory visits. The most common visit diagnoses in 2019 were CF (18%), asthma (14%), chronic cough (9%), sleep apnea and/or snoring (5%), and BPD (4%). Because of the need to social distance and reserve the use of personal protective equipment to urgent medical care, BCH limited elective medical care in mid-March 2020. To continue care virtually, the institution supported 2 technology platforms: first SBR Health, which was integrated with our scheduling platform, and, later, Zoom, which facilitated multidisciplinary visits and required a manual scheduling processes. To ensure those without access to the Internet or a device could continue to see their providers, families could conduct their visits by telephone. Because there was no virtual visit program in our division before the pandemic, our division developed new processes to incorporate these resources into a model of virtual care delivery.

At the start of our shift toward telehealth in mid-March 2020, the clinic’s QI team (composed of physicians, registered nurses, and QI consultants) increased their meeting frequency from monthly to weekly. To specifically address the optimization of our new telehealth program, a workgroup was created, with representation from all key stakeholders: patient-experience representatives (PERs) (responsible for communicating with and scheduling visits for families), pulmonary function test (PFT) technicians, clinical assistants, nurses, and physicians. To track progress toward reaching our SMART aim, the team defined a process for measuring virtual visit volume. Because of the rapid shift in care delivery the pandemic necessitated, subsequent Plan-Do-Study-Act (PDSA) cycles (outlined in Table 1) were implemented quickly and with overlap. The goals of these rapid PDSA cycles were broad, addressing concerns related to administrative processes, communication with patients and families, provider training and support, and patient and provider experience. Throughout this period, the QI team met weekly and, also, coordinated outreach to relevant stakeholders.

TABLE 1

PDSA Cycles

Time FramePDSA 1: MarchPDSA 2: AprilPDSA 3: April to MayPDSA 4: April to June
PLAN Formed core QI team dedicated to telehealth
Discussed the need for virtual visit process to be rapidly implemented 
Asked team members to contribute to KDD and fishbone diagram to understand barriers and where we may find points to intervene Met with PERs to understand the barriers they had been facing Gathered feedback from patients and providers on the telehealth experience 
 Outlined needs for defining new process, facilitating provider training, tracking process, and understanding patient perceptions Considered strategies for incorporating fellows and multidisciplinary providers into virtual visit model
Planned wider discussion outside of the QI group 
  
DO Established weekly meeting frequency for the core QI team
Defined the SMART aim
Created a process flowsheet and provider tips-and-tricks sheet
Implemented tracking queries from medical records: virtual visits, in-person visits, and no shows 
Created fishbone and KDDQI meeting with fellows to elicit perspective and discuss participation
Created living document with QI processes accessible to the division
Huddled daily to review data and address scheduling issues
Gave the first division-wide update 
QI team drafted letters to family to discuss the transition and created scripts for schedulers to use Created and distributed patient and provider surveys via Research Electronic Data Capture to assess the experience of telehealth
Publicized the experience with telehealth through a division-wide webinar and subspeciality specific webinars 
STUDY Weekly meetings were well attended and generated many ideas Daily huddle allowed for real- time discussion of successes and challenges Communication around best practices for telehealth visits was crucial Routine analysis of data through the collection period to address any time-sensitive issues and suggestions 
 Tracking of virtual visits revealed we were below goal Major barriers identified included perceptions of patients and providers re the following: the utility of visits, uncertainty in scheduling processes, and need for different visit platforms for multiprovider fellow or interdisciplinary visits Learned PERs were understaffed relative to communication volume  
 Uncertainty in the reliability of no-show data on the basis of how it were collected    
 Tips-and-tricks document was deemed helpful by provider feedback    
ACT Continued tracking data of virtual visit and in-person visits and continued to troubleshoot algorithm for defining no-show rate
Brainstormed key drivers to low volume numbers
Continued to meet as a QI group 
Obtained approval for alternative platform, Zoom, for multiprovider care
Brainstormed ways to improve communication with patients
Continued tracking virtual visit volume
Continued to meet with the QI group, with added fellow representation, and with the division 
Redeployed clinical personnel with less in-person responsibility to communicate with families
Continued tracking virtual visit volume
Continued to meet with the QI group, PERs, and division 
Considered how to improve telehealth and overcome identified challenges, including plans for home spirometry program 
Time FramePDSA 1: MarchPDSA 2: AprilPDSA 3: April to MayPDSA 4: April to June
PLAN Formed core QI team dedicated to telehealth
Discussed the need for virtual visit process to be rapidly implemented 
Asked team members to contribute to KDD and fishbone diagram to understand barriers and where we may find points to intervene Met with PERs to understand the barriers they had been facing Gathered feedback from patients and providers on the telehealth experience 
 Outlined needs for defining new process, facilitating provider training, tracking process, and understanding patient perceptions Considered strategies for incorporating fellows and multidisciplinary providers into virtual visit model
Planned wider discussion outside of the QI group 
  
DO Established weekly meeting frequency for the core QI team
Defined the SMART aim
Created a process flowsheet and provider tips-and-tricks sheet
Implemented tracking queries from medical records: virtual visits, in-person visits, and no shows 
Created fishbone and KDDQI meeting with fellows to elicit perspective and discuss participation
Created living document with QI processes accessible to the division
Huddled daily to review data and address scheduling issues
Gave the first division-wide update 
QI team drafted letters to family to discuss the transition and created scripts for schedulers to use Created and distributed patient and provider surveys via Research Electronic Data Capture to assess the experience of telehealth
Publicized the experience with telehealth through a division-wide webinar and subspeciality specific webinars 
STUDY Weekly meetings were well attended and generated many ideas Daily huddle allowed for real- time discussion of successes and challenges Communication around best practices for telehealth visits was crucial Routine analysis of data through the collection period to address any time-sensitive issues and suggestions 
 Tracking of virtual visits revealed we were below goal Major barriers identified included perceptions of patients and providers re the following: the utility of visits, uncertainty in scheduling processes, and need for different visit platforms for multiprovider fellow or interdisciplinary visits Learned PERs were understaffed relative to communication volume  
 Uncertainty in the reliability of no-show data on the basis of how it were collected    
 Tips-and-tricks document was deemed helpful by provider feedback    
ACT Continued tracking data of virtual visit and in-person visits and continued to troubleshoot algorithm for defining no-show rate
Brainstormed key drivers to low volume numbers
Continued to meet as a QI group 
Obtained approval for alternative platform, Zoom, for multiprovider care
Brainstormed ways to improve communication with patients
Continued tracking virtual visit volume
Continued to meet with the QI group, with added fellow representation, and with the division 
Redeployed clinical personnel with less in-person responsibility to communicate with families
Continued tracking virtual visit volume
Continued to meet with the QI group, PERs, and division 
Considered how to improve telehealth and overcome identified challenges, including plans for home spirometry program 

—, not applicable.

The primary measure tracked during this period was weekly virtual visit volume, which was compared with 2019 average weekly volume. On the basis of 2019 data, the average weekly clinic volume was 281 visits, making our virtual visit goal 210 weekly visits (75% of 2019 volume). Balancing measures that were considered included rates of no show and technical difficulty; however, our systems at the time of this transition did not allow us to reliably obtain these data.

To assess patients’ and providers’ acceptability of telehealth visits and identify challenges associated with its use, surveys were developed, with questions designed to capture previous telehealth experience, ease of use, convenience, satisfaction, and concerns regarding lack of in-person assessments. Likert scales were used to assess satisfaction, convenience, and level of concern regarding lack of in-person assessments (1 = least or none; 5 = most or high). For ratings of concern, scores of 1 to 2 were classified as low concern, and 3 to 5 were classified as moderate to high concern.

A subset of patients seen for a virtual visit between March 2020 and June 2020 were e-mailed a link for the anonymized survey starting in April. They received 2 subsequent e-mail reminders to complete the survey. Patients were not solicited multiple times if they had repeat visits during this time frame, and, in cases in which siblings were seen, each family was sent only 1 survey. Patients with CF were evaluated with a similar, parallel survey, with additional questions geared toward their unique care needs. Their results were analyzed separately. Pediatric pulmonary providers who participated in telehealth during this time were sent an anonymized survey link by hospital e-mail. The scope of this work did not fall under our institution’s definition of human subjects research and did not meet criteria for institutional review board review. Surveys are provided in the Supplemental Information.

Virtual as well as in-person visit volumes were tracked and reviewed with the QI team weekly. Progress was followed with visual analysis of a time-series graph. Visit diagnoses were analyzed at the end of the study period and compared with those from 2019. Given the inability to collect no-show and technical-difficulty rates, we relied on regular qualitative verbal feedback from clinicians and stakeholders through frequent workgroup and division-wide meetings to understand the unintended consequences of our interventions and inform our PDSA cycles. To evaluate patient and provider perceptions, descriptive survey data were reviewed monthly to address any time-sensitive issues and formally analyzed at the end of the survey period.

With the first PDSA cycle (Table 1), we aimed to assemble the telehealth-focused QI team, establish goals, define measures, and plan for immediate process. Our SMART aim was collectively determined, and we established an EMR-based algorithm to track visit volume. Our division superuser created and distributed best practice guidelines as well as tips-and-tricks references to navigate the new software.

The next PDSA cycle was focused on understanding and addressing the key drivers and contributors to low virtual visit volume. A fishbone diagram (Fig 1) and key driver diagram (KDD) (not shown) were constructed. To increase discussion outside our QI group, we organized huddles among the clinic administrative staff, increased communication at division-wide meetings, and met with trainees. In these discussions, we learned of perceptions held by both families and providers of the low utility of virtual visits as well as uncertainty in the new scheduling processes. We also identified technology limitations in incorporating multiprovider care into our model because the SBR Health platform supported only 1 provider. We obtained approval for a second platform, Zoom, to allow multiple providers to log on with a patient, necessary for trainee precepting and interdisciplinary visits.

FIGURE 1

Fishbone diagram created in early April, revealing the identified contributors to low virtual visit volume and highlighting areas of focus.

FIGURE 1

Fishbone diagram created in early April, revealing the identified contributors to low virtual visit volume and highlighting areas of focus.

Close modal

With the third PDSA, we aimed to improve administrative processes related to scheduling virtual visits because uncertainty in these processes had been expressed during huddles set up in the previous PDSA. PDSA 3 began with a meeting exclusively with the PERs, whose responsibilities include communicating with and scheduling patients. We drafted and sent communications to families describing the necessary transition to telehealth, emphasizing provider support for new process. We also addressed concerns surrounding inadequate staffing to manage call burden by redeploying clinical personnel to help with scheduling. We constructed scripts to be used by all staff communicating with patients to schedule visits. During this time, we continued meeting with all stakeholders in various formats.

Finally, we assessed patient and provider experience with telehealth. Responses to patient and provider surveys were informally reviewed during the collection period, and there were no widespread concerns conveyed about technical difficulties hampering the ability for the visits to be completed. Results were formally analyzed at the end of survey distribution in June.

Weekly visits from March 2020 through June 2020 are presented in a time-series graph (Fig 2), stratified by virtual and in person. The first week of March 2020 was reflective of prepandemic volume, with 210 in-person visits and 0 virtual visits. At the onset of the pandemic, in-person visits dropped significantly to 38, with 61 virtual visits, yielding a total weekly visit volume of 99 (a 65% reduction from the historical average). Over the next 3 weeks, we achieved our goal of 210 virtual visits by the beginning of April and maintained this goal for many of the subsequent 10 weeks studied.

FIGURE 2

Time-series graph representing telehealth and in-clinic visits per week from March to June 2020. aIndicates 4-day week (holiday).

FIGURE 2

Time-series graph representing telehealth and in-clinic visits per week from March to June 2020. aIndicates 4-day week (holiday).

Close modal

The state began a stepwise re-entry plan in mid-June, and BCH, accordingly, resumed some in-person services. Starred dates in the time-series graph (also marked with red circles on the data points) represent 4-day weeks, during which clinic volume was expectedly less than during nonholiday weeks. We presume our unmet virtual visit volume goals in mid to late June reflect shifted efforts toward resuming in-person visits, as infection control and safety measures allowed. The chief complaints leading to telehealth visits from March to June were similar to common chief complaints prepandemic: asthma (19%), CF (15%), obstructive sleep apnea and/or snoring (9%), chronic cough (6%), and BPD (6%).

A total of 52 families responded to a survey sent to 392 unique families seen in our clinic by telehealth during March 2020 to June 2020 (13% response rate). There was representation of patients across the pediatric age span (27% infant or toddler, 52% school aged, 10% teenaged, and 11% ≥18 years). Most patients (71%) had no previous experience with telehealth (Table 2). Most families (77%) did not have difficulty logging in to the visit (Table 2). A majority of respondents were able to see all the providers they desired to see (96%), had adequate time (98%), and felt their questions were answered (98%) during the visit. Most patients rated their visits highly satisfactory (90%) and convenient (88%). A notable proportion of patients reported at least moderate concern about the lack of physical examination (53%) and testing (42%; Table 2). Looking to future telehealth use, 58% of patients reported a desire to have “some” of their future visits virtually, 24% preferred “most,” and 8% wished to no longer participate in telehealth visits (Table 2). Patients with CF at BCH polled as part of a parallel survey (n = 20) reported similar rates of satisfaction (90%) and convenience (95%). A total of 60% were at least moderately concerned about the lack of testing, and most (85%) desired “some” of future visits virtually.

TABLE 2

Survey Responses

Patient Responses, %Provider Responses, %
Telehealth use before March 2020   
 Yes 29 18 
 No 79 82 
Technical difficulties   
 Yes 23 73 
 No 77 17 
Device used   
 Computer or laptop with video 39 86 
 Tablet or iPad with video 15 
 Cellphone with video 42 
 Cellphone with audio only 
Concern regarding missing assessments   
 Low 39a; 56b; 42c 19a; 32b; 10c 
 Moderate to high 42a; 36b; 53c 81a; 68b; 90c 
 Not applicable 19a; 8b; 6c — 
Preferred frequency of future telehealth visits   
 None 
 Some 58 76 
 Most 34 24 
 All 
Providers’ perceived impact on the patient and provider relationship   
 Negative — 
 Neutral — 41 
 Positive — 50 
Patient Responses, %Provider Responses, %
Telehealth use before March 2020   
 Yes 29 18 
 No 79 82 
Technical difficulties   
 Yes 23 73 
 No 77 17 
Device used   
 Computer or laptop with video 39 86 
 Tablet or iPad with video 15 
 Cellphone with video 42 
 Cellphone with audio only 
Concern regarding missing assessments   
 Low 39a; 56b; 42c 19a; 32b; 10c 
 Moderate to high 42a; 36b; 53c 81a; 68b; 90c 
 Not applicable 19a; 8b; 6c — 
Preferred frequency of future telehealth visits   
 None 
 Some 58 76 
 Most 34 24 
 All 
Providers’ perceived impact on the patient and provider relationship   
 Negative — 
 Neutral — 41 
 Positive — 50 

—, not applicable.

a

Lack of testing and/or PFTs, %.

b

Lack of vital signs, %.

c

Lack of examination, %.

A total of 35 pediatric pulmonary clinicians participating in telehealth were contacted by e-mail, and 22 clinicians (63%) completed the anonymized survey. Most respondents (82%) had never used telehealth before, citing a lack of institutional support (63%), technology limitations (45%), scheduling logistics (41%), and out-of-state regulatory concerns (41%). A significant proportion (73%) reported technical difficulties (Table 2), with 31% describing issues that were quickly resolved, 55% endorsing issues that took some time but were resolvable, and a minority of respondents (14%) describing visits that were terminated as a result of these technical difficulties. A total of 81% of providers were satisfied with the telehealth experience, and 50% felt telehealth had a positive impact on the provider-patient relationship. As with surveyed patients, providers conveyed concerns regarding lack of in-person assessments with at least moderate concern for lack of physical examination (90%), lack of PFTs (81%), and lack of vital signs (68%; Table 2). Overall, provider concern regarding lack of in-person assessments was greater than that expressed by patients. Most providers (76%) felt “some” visits should be conducted by telehealth in the future, with 24% endorsing using telehealth for “most” future visits (Table 2).

We achieved of our specific aim to increase virtual visit volume during the COVID-19 pandemic, using QI processes and principles to drive change. This goal was compelled by the need to limit in-person interaction, and conversion to telehealth allowed continued access to pediatric pulmonary care. Additionally, through surveys, we demonstrated high ratings of satisfaction and convenience with telehealth, and, despite technical difficulties and missing in-person testing, patients and providers expressed interest in using telehealth going forward.

Our success was driven by the collective efforts across our rapid PDSA cycles, many of which occurred in parallel. Because the pandemic necessitated immediate action and a drastic overhaul in our usual processes to maintain access to care, we were unable to follow a more typical QI model of performing distinct iterative tests of change, a model that would allow better evaluation of each intervention’s impact. In Foster and Stack’s16  recent commentary, they suggest that QI work in situations requiring immediate change in practice may have several phases (acute, subacute, and chronic), with the acute phase focused on protocol development and frequent communication on the rapidly evolving practices. Our described work falls into the acute phase of pandemic response, and we attribute much of our success to incorporating core QI principles into our project design and implementation, such as generating a clear definition of our goal, establishing meaningful and reliable measures, prioritizing a patient-centered care experience, supporting frequent communication with stakeholders, and using a QI framework with PDSA cycles to guide our work.

Beyond our QI efforts, we recognize other barriers to telehealth’s implementation were alleviated through institutional and regulatory policies, including technology support services and clear state guidelines mandating full insurance coverage.1  By encouraging institutions and governing bodies to continue these new supports, services, and regulations postpandemic, centers across the country can maximize the great opportunity telehealth lends to our health care system.

Patient engagement is another critical element to the success of telehealth’s sustainability.18  Through patient surveys, we found respondents reported positive experiences, with high ratings of satisfaction and convenience and interest to incorporate telehealth into future care. Although respondents did not convey high rates of technical difficulty, this result could be biased by those who responded, and we were unable to reliably track rates of visit no show nor technical difficulties as balancing measures during the acute phase of this project. As we moved to the subacute phase after the scope of this project, we were able to establish more consistent practices around tracking no show as a meaningful balancing measure as well as visit fill rate.

Clinical appropriateness for continued telehealth postpandemic should be considered. One drawback of telehealth is the lack of in-person assessments, which was highlighted by surveyed patients and providers. Some of these missing assessments may be supplanted, at least in part, by novel home monitoring devices, including pulse oximeters, virtual stethoscopes, and, recently, Food and Drug Administration–approved home spirometry devices, which may significantly add to the assessment we are able to make virtually.19,20  Our division has initiated a home spirometry monitoring program in an attempt to partially fill this gap in virtual assessment, with plans to formally study the program’s outcomes. Each specialty clinic will, ultimately, need to define the standards of care regarding the integration of telehealth into their clinical care model.

There are several limitations to mention. In the current study, we examine the feasibility of implementing telehealth in a pediatric pulmonary clinic during the COVID-19 pandemic, and the uniqueness of the context may limit the generalizability of these findings. Additionally, the processes implemented and technology used relied on robust administrative and information technology support available to us in a large tertiary care center. With different resources, changes to clinic process may be significantly more challenging to achieve. We recognize the low patient response rate as a limitation because it is possible those unwilling or unable to engage with technology may have been excluded, leading to a nonresponse bias toward positivity. Ongoing assessment of patient experience will be important. Finally, we were unable to reliably track desired balancing measures with our existing programs, and our group is actively working on creating these measurement tools.

Telehealth will likely remain a part of our health care system going forward, and it is essential to continue QI-guided work to optimize this form of care delivery. We advocate working with patients and families through a model of coproduction to best understand how this incredible tool can be leveraged most effectively. Additionally, clinical outcomes, including exacerbations of underlying disease, missed or delayed diagnoses, health care use, and costs, should be evaluated, as should the interplay between telehealth, access to care, and social determinants of health. Continued QI-based and outcomes-driven work will be integral in adapting an evolving virtual care model to patient and population needs and will help us understand how to maximize the quality, equity, and value of care we can deliver.

Dr Davis conceptualized and designed the study, designed the data collection instruments, collected data, conducted the final analyses, drafted the initial manuscript, and reviewed and revised the manuscript; Ms Gordon designed the data collection instruments, collected data, conducted the initial analyses, and reviewed and revised the manuscript; Mr Hammond designed the data collection instruments, collected data, conducted the initial analyses, and reviewed the manuscript; Drs Perkins and Flanagan were involved in conceptualizing and designing the study, designed data collection instruments, collected data, and reviewed and revised the manuscript; Drs Simoneau and Rabinowitz conceptualized and designed the study and critically reviewed the manuscript for important intellectual content; Dr Sawicki conceptualized and designed the study, supervised the design of data collection instruments, data collection, and analysis, and reviewed and revised the manuscript for important intellectual content; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

FUNDING: No external funding.

BCH

Boston Children’s Hospital

BPD

bronchopulmonary dysplasia

CF

cystic fibrosis

COVID-19

coronavirus disease 2019

KDD

key driver diagram

PDSA

Plan-Do-Study-Act

PER

patient-experience representative

PFT

pulmonary function test

QI

quality improvement

SMART

specific, measurable, applicable, realistic, and timely

1
Mass.gov
.
COVID-19 state of emergency
.
2
Mass.gov
.
Governor Charlie Baker orders all non-essential businesses to cease in person operation, directs the Department of Public Health to issue stay at home advisory for two weeks
.
3
Dayal
P
,
Chang
CH
,
Benko
WS
, et al
.
Hospital utilization among rural children served by pediatric neurology telemedicine clinics
.
JAMA Netw Open
.
2019
;
2
(
8
):
e199364
4
Lo
MD
,
Gospe
SM
Jr
.
Telemedicine and child neurology
.
J Child Neurol
.
2019
;
34
(
1
):
22
26
5
Maia
MR
,
Castela
E
,
Pires
A
,
Lapão
LV
.
How to develop a sustainable telemedicine service? A pediatric telecardiology service 20 years on-an exploratory study
.
BMC Health Serv Res
.
2019
;
19
:
681
6
Harting
MT
,
Wheeler
A
,
Ponsky
T
, et al;
APSA Informatics and Telemedicine Committee
.
Telemedicine in pediatric surgery
.
J Pediatr Surg
.
2019
;
54
(
3
):
587
594
7
Borowitz
D
,
Parad
RB
,
Sharp
JK
, et al;
Cystic Fibrosis Foundation
.
Cystic Fibrosis Foundation practice guidelines for the management of infants with cystic fibrosis transmembrane conductance regulator-related metabolic syndrome during the first two years of life and beyond
.
J Pediatr
.
2009
;
155
(
suppl 6
):
S106
S116
8
National Asthma Education and Prevention Program
.
Expert Panel Report 3 (EPR-3): guidelines for the diagnosis and management of asthma-summary report 2007. [published correction appears in J Allergy Clin Immunol. 2008;121(6):1330]
.
J Allergy Clin Immunol
.
2007
;
120
(
suppl 5
):
S94
S138
9
Compton
M
,
Soper
M
,
Reilly
B
, et al
.
A feasibility study of urgent implementation of cystic fibrosis multidisciplinary telemedicine clinic in the face of COVID-19 pandemic: single-center experience
.
Telemed J E Health
.
2020
;
26
(
8
):
978
984
10
Brent
A
,
Chuo
J
,
Freeman
B
, et al
.
Telehealth support: compendium
.
11
Berg
EA
,
Picoraro
JA
,
Miller
SD
, et al
.
COVID-19-a guide to rapid implementation of telehealth services: a playbook for the pediatric gastroenterologist
.
J Pediatr Gastroenterol Nutr
.
2020
;
70
(
6
):
734
740
12
Gilbert
AW
,
Billany
JCT
,
Adam
R
, et al
.
Rapid implementation of virtual clinics due to COVID-19: report and early evaluation of a quality improvement initiative
.
BMJ Open Qual
.
2020
;
9
(
2
):
e000985
13
Yang
L
,
Brown-Johnson
CG
,
Miller-Kuhlmann
R
, et al
.
Accelerated launch of video visits in ambulatory neurology during COVID-19: key lessons from the Stanford experience
.
Neurology
.
2020
;
95
(
7
):
305
311
14
Itamura
K
,
Rimell
FL
,
Illing
EA
, et al
.
Assessment of patient experiences in otolaryngology virtual visits during the COVID-19 pandemic
.
OTO Open
.
2020
;
4
(
2
):
2473974X2093357
15
Triantafillou
V
,
Layfield
E
,
Prasad
A
, et al
.
Patient perceptions of head and neck ambulatory telemedicine visits: a qualitative study
.
Otolaryngol Head Neck Surg
.
2021
;
164
(
5
):
923
931
16
Foster
AA
,
Stack
AM
.
Quality improvement in a pandemic
.
Pediatr Qual Saf
.
2020
;
5
(
4
):
e321
17
Fitzsimons
J
.
Quality & safety in the time of Coronavirus-design better, learn faster [published online ahead of print May 13, 2020]
.
Int J Qual Heal Care
.
doi:10.1093/intqhc/mzaa051
18
Srinivasan
M
,
Phadke
AJ
,
Zulman
D
, et al
.
Enhancing patient engagement during virtual care: a conceptual model and rapid implementation at an academic medical center
.
NEJM Catal Innov Care Deliv
.
2020
;
09
:
1
10
19
Logie
K
,
Welsh
L
,
Ranganathan
SC
.
Telehealth spirometry for children with cystic fibrosis
.
Arch Dis Child
.
2020
;
105
(
12
):
1203
1205
20
Kruizinga
MD
,
Essers
E
,
Stuurman
FE
, et al
.
Technical validity and usability of a novel smartphone-connected spirometry device for pediatric patients with asthma and cystic fibrosis
.
Pediatr Pulmonol
.
2020
;
55
(
9
):
2463
2470

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

Supplementary data