OBJECTIVES:

Health care providers (HCPs) require ongoing support to meet the evolving care needs of children with medical complexity (CMC). Project Extension for Community Healthcare Outcomes (ECHO) is a model for delivering technology-enabled medical education and cultivating a community of practice. In this study, we focused on developing, implementing, and evaluating the first ECHO program dedicated to the care of CMC. Specific objectives were to evaluate the program feasibility (participation and acceptability) and impact on perceived HCP knowledge, self-efficacy, and clinical practice after 6 months.

METHODS:

A needs assessment was conducted to inform an interprofessional CMC curriculum. This curriculum was delivered through monthly virtual TeleECHO clinics (didactic and case-based learning) from January 2018 to 2020. The program was available at no cost to HCPs throughout Ontario. Surveys were distributed at baseline and 6 months to assess program acceptability, knowledge, self-efficacy, and practice impact by using 7-point Likert scales. Descriptive and inferential data analyses were conducted.

RESULTS:

Twenty-four clinics were completed with a mean of 19 ± 6 attendees. Acceptability scores (n = 27) ranged from 5.0 ± 1.1 to 6.4 ± 0.6. Participants reported an improvement in their knowledge and self-efficacy across all probed topics and skills (P values ranged from <.001 to .006). These knowledge and self-efficacy scores related to “complex care support,” “feeding support,” and “respiratory support.” The majority of participants reported positive or very positive practice impacts, including enhanced ability to provide quality care to CMC.

CONCLUSIONS:

Project ECHO is a feasible and acceptable model for virtual education of interprofessional HCPs in managing CMC. This program has the potential to increase system capacity to provide quality care to CMC close to home.

Children with medical complexity (CMC) are characterized by medical fragility, technology dependency, and intensive care needs. They often require the expertise of many health care providers (HCPs) spanning different disciplines, institutions, community agencies, and settings of care. Although CMC represent <1% of all children, their numbers are growing because of advances in medical and surgical care that have created a new generation of long-term survivors of childhood-onset diseases.1  Despite their small numbers, the impact of these children on the health care system is already substantial. In the United States, addressing the medical needs of CMC accounts for 33% of pediatric health care spending.2  In Canada, CMC account for a similar proportion of provincial health care spending.3 

There are limited training programs focused on the care of CMC, yet HCPs require ongoing education to meet the complicated and evolving health care needs of this group.4  Given their significant fragility, without proper care, CMC are at high risk for prolonged hospitalization, medical error, fragmented care, readmission, and critical illness.5,6 

Dedicated hospital-based programs with interprofessional teams designed to care for CMC have been developed over the last decade.7  Within complex care, workforce development with a focus on interprofessional learning opportunities is recognized as a priority for HCPs.810  The management of CMC is a necessary skill of pediatricians, spanning inpatient and outpatient settings.11,12  Researchers have identified educational gaps and lack of comfort in caring for CMC among HCPs.1214  Pediatric residents and recent residency graduates have reported inadequate training to support CMC and communicate with families about medical decisions.12,14  Education requires greater attention to the psychosocial needs of CMC and their families as well as dedicated training in care coordination and technological management. National health organizations have emphasized the provision of continuing education from tertiary centers as a necessary step in building system capacity to care for CMC.15  Overall, formal training opportunities are limited, creating a need for accessible programs to support interprofessional HCPs in managing this growing patient group.

Project Extension for Community Healthcare Outcomes (ECHO) is a model for technology-enabled interprofessional education.16,17  It is designed to expand access and capacity to deliver evidence-informed specialized care. The ECHO model provides HCPs with training, mentorship, and support to locally manage their patients with specialized health needs. ECHO uses a “hub-and-spoke” structure, wherein the hub (a specialty interprofessional team) regularly connects via videoconference with multiple spokes (community-based HCPs) to learn together with the shared goal of enhancing local patient care.

The ECHO model has been successfully applied across various clinical conditions.16,18,19  Within the pediatric complex care setting, ECHO has the potential to improve access to specialist knowledge, particularly among the family physicians, community-based consultant pediatricians, nurse practitioners, and allied health providers who manage CMC in hospital and community settings. However, although ∼10% of all global ECHO programs concentrate on pediatric health care, none are focused on CMC.18 

In this article, we describe the process of developing, implementing, and evaluating Pediatric Project ECHO for Complex Care, the first ECHO program focused on CMC. The primary study objective was to determine if the program could be feasibly implemented on the basis of participation and acceptability. The secondary objective was to characterize the perceived impact of Pediatric Project ECHO for Complex Care on participant knowledge, self-efficacy, and clinical practice after 6 months.

A needs assessment was conducted to inform the development of a complex care-specific Project ECHO curriculum.20  An online survey (38 items; 15 minutes) was distributed via targeted emails to professional networks, associations, and allied health organizations related to complex care throughout Ontario. Administered by using Research Electronic Data Capture (REDCap), the survey was designed to assess interest in specific topics by using individual Likert scales ranging from 1 (not interested) to 5 (very interested). The topic list was developed through clinician consensus (including authors M.H., J.O., and E.C.). Survey respondents could also suggest additional topics through open-text fields. See the Supplemental Information for survey items. The survey was live between May and August 2017.

For initial program implementation, resources were focused on leveraging an existing network of complex care clinics called Complex Care for Kids Ontario. This provincial organization provides quality standards and guidance to support clinical care for CMC. Marketing materials were disseminated through Complex Care for Kids Ontario with the aim of recruiting community HCPs to the program. Additionally, materials were disseminated broadly through pediatric-focused conferences and organizations. The Pediatric Project ECHO for Complex Care program was launched in January 2018. Any HCP in Ontario could register as a program participant at no cost and earn continuing professional development credits. HCPs could join at any time (ie, rolling recruitment). Registrants received regular e-mail notifications of upcoming ECHO clinics.

The Pediatric Project ECHO for Complex Care program is based at a major pediatric teaching hospital in Canada. The program is funded by the Ontario Ministry of Health, initially as a demonstration project (2017–2019) before transitioning into annual operational funding. Day-to-day program logistics are overseen by an operations team (authors A.J., S.S., and A.G.), whereas program delivery is provided by clinicians from the hospital’s complex care program (including authors C.D., M.H., E.C., and J.O.). The specialist team at the Project ECHO academic hub includes pediatricians, nurse practitioners, registered dietitians, and pharmacists (see Fig 1). Program evaluation is overseen by an independent research team (C.L., J.O.T., A.S.H., and J.S.).

FIGURE 1

Structure of Pediatric Project ECHO for Complex Care, a technology-enabled virtual interprofessional education program to support interprofessional management of CMC.

FIGURE 1

Structure of Pediatric Project ECHO for Complex Care, a technology-enabled virtual interprofessional education program to support interprofessional management of CMC.

Close modal

Pediatric Project ECHO for Complex Care offers monthly virtual interactive seminars called TeleECHO clinics (60 minutes). Each TeleECHO clinic includes a didactic presentation (∼15 minutes), a deidentified case presentation by a community HCP (∼20 minutes), and facilitated discussion to generate best practice recommendations for managing the presented case (∼25 minutes). TeleECHO clinics are conducted by using Zoom videoconferencing technology.

Each program registrant was invited to take part in an embedded research study (hospital ethics approval 1000057321). Program registrants who provided informed consent were enrolled into this study as research participants. Attendance levels for each TeleECHO clinic were centrally tracked, and electronic surveys were distributed at the time of registration (baseline). Research participants who attended at least 1 ECHO clinic in the 6 months after registration were eligible to complete a follow-up survey. The surveys were designed to assess program acceptability as well as perceived knowledge and self-efficacy for topics and skills from the program curriculum. Survey items were developed to accommodate the interprofessional nature of participants because no existing validated knowledge or self-efficacy questionnaires related to the management of CMC were suitable for all HCPs. Consequently, surveys were purposefully designed such that respondents could align their self-assessments of knowledge and self-efficacy with best judgement of their own scope of practice. Knowledge and self-efficacy items were framed with the stem, “for my scope of practice, I currently have an appropriate level of knowledge about [topic]” and “for my scope of practice, I am confident in my ability to [skill]”. Likert scales were used to assess item-level agreement, in which 1 indicated “strongly disagree” and 7 indicated “strongly agree.” “Not applicable” options were provided when relevant. Anecdotal evidence from other Project ECHO programs suggests that participants may not accurately estimate baseline self-efficacy. To correct for this possibility, at the 6-month time point, participants were given the opportunity to review and adjust baseline self-efficacy responses. For each self-efficacy item, participants had the option to (1) accept their baseline rating, (2) lower their baseline rating (ie “didn’t know what they didn’t know”), or (3) increase their baseline rating (ie “didn’t realize how much they already knew”). Adjusted baseline responses were used in subsequent analyses. Survey items used to assess perceived clinical practice impact were derived from ECHO literature and the expertise of the program team. Participant reflections on the program were also collected through open-text responses. Surveys were administered by using REDCap.

Quantitative data were exported from REDCap and analyzed by using Stata, version 15.1, (Stata Corp, College Station, TX) by author A.S.H. Qualitative data from open-text fields were exported verbatim from REDCap and organized by author J.O.T. All collected participant quotes are included within the results. Curriculum preferences from the needs assessment survey were sorted descriptively. The a priori target number of attendees at each TeleECHO clinic was ≥6, on the basis of previous pilot ECHO programs. The target threshold for acceptability was a mean score of ≥5 (7-point Likert scale) across all items. No thresholds were specified for perceived knowledge, self-efficacy, or practice impact. Knowledge and self-efficacy items were collapsed into categories based on clinical area by authors C.D. and M.H. Scores for each category were calculated by averaging scores of relevant survey items. Two-tailed paired sample t tests were used to examine changes in knowledge and self-efficacy items from baseline to 6 months among those participants who completed both surveys.

Respondents were interprofessional, with a total of 49 respondents from 27 unique health care institutions.20  The topics of interest spanned all domains of clinical care provided to CMC, including biological (eg, pain and irritability of unknown origin), psychological (eg, challenging behavior), and social (eg, supports available for parents and siblings). Service delivery was also highlighted by participants (eg, transition to adult care and advanced care planning). These educational preferences were used to inform the curriculum for Pediatric Project ECHO for Complex Care. For instance, the identified need for education on “feeding and nutrition” was addressed through TeleECHO clinics focused on enteral tube issues, gastroesophageal reflux disease, and blenderized diets for CMC. Given that the quantity of suggested topics exceeded the number of TeleECHO clinics, topics were ranked on the basis of respondent interest and arranged on the basis of content complexity. The top 12 ranked topics were integrated into the initial curriculum on the basis of respondent interest and speaker availability. Forty-nine percent of respondents were “very interested” in “nutrition in complex patients,” whereas there was less interest in “central venous access” (9%) and “fever of unknown origin” (18%). Respondents also provided additional topics, such as “mental health comorbidities,” “assigning complexity scores for chronic pain patients,” and “assessing respite care needs.” The curriculum was accredited through the University of Toronto such that participants could earn continuing professional development credits.

Between January 2018 and January 2020, 24 TeleECHO clinics were held. Each clinic included a mean of 19 attendees (SD: ±6). Attendance varied each month, from a minimum of 8 to a maximum of 37 participants. Overall, 14 participants (29%) attended 1 clinic, 13 participants (27%) attended 2 clinics, and 12 participants (25%) attended ≥5 clinics.

In total, 367 HCPs registered for Pediatric Project ECHO for Complex Care; however, not all registrants consented to this research study or attended a TeleECHO clinic. A total of 205 HCPs provided informed consent for this research study at the time of program registration. Of those consented participants, 73 (36%) completed the baseline survey, and, within this group, 49 participants (67%) attended ≥1 sessions and were, thus, eligible to complete a 6-month survey. Twenty-seven (55%) of the eligible 49 participants completed a 6-month survey. The characteristics of research participants, broken down by clinic attendance status, are described in Table 1. Participant perceptions of program acceptability and satisfaction after 6 months are presented in Table 2. In addition, 4 participants provided written feedback on program acceptability via open-text fields on the 6-month survey. This feedback supported the generally high ratings of program acceptability. For instance, a registered nurse stated, “It is a supportive community that allowed me to interact, ask questions in a very comfortable environment. It was refreshing to know that others had similar issues, and I was pleasantly surprised at the range of knowledge and diversity of those participating.” A physiotherapist participant shared, “[the program] encourages reflection on how my team has managed similar clients, great to get new ideas.” An occupational therapist stated that “interdisciplinary learning” was a benefit of the program. Lastly, a physician stated, “I would suggest to have [TeleECHO] clinics more often.”

TABLE 1

Characteristics of Pediatric Project ECHO for Complex Care Research Participants

CharacteristicResearch Participants, n = 73Completed 6-mo Survey (n = 27)
Attended No Sessions (n = 24)Attended ≥1 Session (n = 49)
Sex, n (%)    
 Female 18 (82) 46 (96) 25 (96) 
 Male 2 (9) 2 (4) 1 (4) 
 Prefer not to respond 2 (9) — — 
Age group, n (%)    
 ≤29 y 7 (31) 5 (10) 2 (8) 
 30–49 y 11 (50) 32 (67) 18 (69) 
 50–69 y 4 (19) 11 (23) 6 (23) 
Profession, n (%)    
 Dietitian 2 (8) 6 (12) 5 (19) 
 Kinesiologist and exercise counselor — 1 (2) 1 (4) 
 Nurse    
  Nurse practitioner 1 (4) 7 (14) 4 (15) 
  Registered nurse 10 (42) 9 (18) 4 (15) 
Pharmacist 1 (4) 1 (2) — 
Physician assistant — 1 (2) — 
 Physician    
  Pediatrician (general) 2 (8) 11 (22) 7 (26) 
  Pediatrician (specialist) 1 (4) 1 (2) 1 (4) 
  Primary care 1 (4) — — 
 Psychologist — 1 (2) — 
 Rehabilitation therapist 1 (4) 3 (6) 2 (7) 
 Social worker 1 (4) 2 (4) — 
 Othera 4 (17) 6 (12) 3 (11) 
Primary practice setting, n (%)    
 Academic hospital 11 (48) 18 (38) 10 (39) 
 Family health team — 1 (2) — 
 Community health center — 6 (13) 1 (4) 
 Nonacademic hospital 3 (13) 4 (8) 3 (12) 
 Hospice facility 4 (17) 4 (8) 1 (4) 
 Home and community care 3 (13) 7 (15) 5 (19) 
 Private practice 1 (4) 5 (10) 3 (12) 
 Otherb 1 (4) 3 (6) 3 (12) 
 Years in practice (after completing training), n (%)    
 <1 y 1 (4) 6 (12) 3 (11) 
 1–4 y 8 (35) 12 (25) 8 (30) 
 5–10 y 5 (22) 5 (10) 2 (7) 
 >10 y 7 (30) 25 (51) 13 (48) 
 Not applicable 2 (9) 1 (2) 1 (4) 
CharacteristicResearch Participants, n = 73Completed 6-mo Survey (n = 27)
Attended No Sessions (n = 24)Attended ≥1 Session (n = 49)
Sex, n (%)    
 Female 18 (82) 46 (96) 25 (96) 
 Male 2 (9) 2 (4) 1 (4) 
 Prefer not to respond 2 (9) — — 
Age group, n (%)    
 ≤29 y 7 (31) 5 (10) 2 (8) 
 30–49 y 11 (50) 32 (67) 18 (69) 
 50–69 y 4 (19) 11 (23) 6 (23) 
Profession, n (%)    
 Dietitian 2 (8) 6 (12) 5 (19) 
 Kinesiologist and exercise counselor — 1 (2) 1 (4) 
 Nurse    
  Nurse practitioner 1 (4) 7 (14) 4 (15) 
  Registered nurse 10 (42) 9 (18) 4 (15) 
Pharmacist 1 (4) 1 (2) — 
Physician assistant — 1 (2) — 
 Physician    
  Pediatrician (general) 2 (8) 11 (22) 7 (26) 
  Pediatrician (specialist) 1 (4) 1 (2) 1 (4) 
  Primary care 1 (4) — — 
 Psychologist — 1 (2) — 
 Rehabilitation therapist 1 (4) 3 (6) 2 (7) 
 Social worker 1 (4) 2 (4) — 
 Othera 4 (17) 6 (12) 3 (11) 
Primary practice setting, n (%)    
 Academic hospital 11 (48) 18 (38) 10 (39) 
 Family health team — 1 (2) — 
 Community health center — 6 (13) 1 (4) 
 Nonacademic hospital 3 (13) 4 (8) 3 (12) 
 Hospice facility 4 (17) 4 (8) 1 (4) 
 Home and community care 3 (13) 7 (15) 5 (19) 
 Private practice 1 (4) 5 (10) 3 (12) 
 Otherb 1 (4) 3 (6) 3 (12) 
 Years in practice (after completing training), n (%)    
 <1 y 1 (4) 6 (12) 3 (11) 
 1–4 y 8 (35) 12 (25) 8 (30) 
 5–10 y 5 (22) 5 (10) 2 (7) 
 >10 y 7 (30) 25 (51) 13 (48) 
 Not applicable 2 (9) 1 (2) 1 (4) 

—, no respondents.

a

Other professions were clinical nurse specialist, clinical nurse educator, developmental services worker, registered massage therapist, palliative care volunteer, health professional student, and parent educator and advocate.

b

Other primary practice settings were the children’s treatment center and rehabilitation center.

TABLE 2

Program Acceptability and Satisfaction After 6 Months, n = 27

ItemMeanSDMinimum; Maximum
I would recommend the program to my colleagues. 6.4 0.6 5; 7 
I respect the knowledge of the facilitators. 6.4 0.6 5; 7 
Involvement in the program is a worthwhile experience for me. 6.2 0.7 5; 7 
I have learned new information. 6.1 0.6 5; 7 
The program is an effective way for me to learn. 5.8 0.6 5; 7 
I have learned best practice care. 5.8 0.8 4; 7 
The program has created a supportive community of practice. 5.7 0.9 4; 7 
The program has connected me with peers and diminished my professional isolation. 5.0 1.1 3; 7 
ItemMeanSDMinimum; Maximum
I would recommend the program to my colleagues. 6.4 0.6 5; 7 
I respect the knowledge of the facilitators. 6.4 0.6 5; 7 
Involvement in the program is a worthwhile experience for me. 6.2 0.7 5; 7 
I have learned new information. 6.1 0.6 5; 7 
The program is an effective way for me to learn. 5.8 0.6 5; 7 
I have learned best practice care. 5.8 0.8 4; 7 
The program has created a supportive community of practice. 5.7 0.9 4; 7 
The program has connected me with peers and diminished my professional isolation. 5.0 1.1 3; 7 

Possible item scores ranged from 1 to 7.

Reported changes in participant knowledge and self-efficacy from baseline to 6 months are presented in Table 3. For self-efficacy, 13 study participants (48%) opted to adjust at least 1 baseline response at 6 months. Of 31 changed items, 4 (13%) scores were lowered (“didn’t know what they didn’t know”), and 27 (87%) scores were increased (“didn’t realize how much they already knew”).

TABLE 3

Knowledge and Self-Efficacy Scores Among Participants in Pediatric Project ECHO for Complex Care

Baseline (n = 27), Mean (SD)6 mo (n = 27), Mean (SD)Mean Difference (SD)Paired t Test
P95% CI
Knowledge      
 Complex care support 5.4 (0.9) 5.8 (0.5) 0.4 (0.8) .006 0.14–0.74 
 Feeding support 4.8 (1.1) 5.5 (0.8) 0.7 (1.0) .002 0.26–1.05 
 Respiratory support 4.0 (1.0) 4.8 (1.1) 0.8 (0.9) <.001 0.46–1.17 
Self-efficacy      
 Complex care support 5.7 (0.9) 6.2 (0.4) 0.5 (0.7) .001 0.23–0.80 
 Feeding support 5.0 (1.0) 5.7 (0.6) 0.7 (0.8) <.001 0.35–0.98 
 Respiratory support 4.4 (1.3) 5.6 (1.3) 1.2 (1.2) <.001 0.68–1.75 
Baseline (n = 27), Mean (SD)6 mo (n = 27), Mean (SD)Mean Difference (SD)Paired t Test
P95% CI
Knowledge      
 Complex care support 5.4 (0.9) 5.8 (0.5) 0.4 (0.8) .006 0.14–0.74 
 Feeding support 4.8 (1.1) 5.5 (0.8) 0.7 (1.0) .002 0.26–1.05 
 Respiratory support 4.0 (1.0) 4.8 (1.1) 0.8 (0.9) <.001 0.46–1.17 
Self-efficacy      
 Complex care support 5.7 (0.9) 6.2 (0.4) 0.5 (0.7) .001 0.23–0.80 
 Feeding support 5.0 (1.0) 5.7 (0.6) 0.7 (0.8) <.001 0.35–0.98 
 Respiratory support 4.4 (1.3) 5.6 (1.3) 1.2 (1.2) <.001 0.68–1.75 

Possible item scores ranged from 1 to 7.

The reported impact of program participation on various aspects of clinical practice is presented in Fig 2. Some participants reported that, although the nature of their practice was not conducive to the immediate application of ECHO knowledge, the program had altered their perspective. As summarized by a physiotherapist participant, “in my work setting, I have my clients long-term and [there is] not a lot of change-over, and so it may be awhile before clients will come up where I can apply my new knowledge, but my mind is opening and I am learning to see clients through multiple lenses now.” Twelve participants (44%) indicated that program discussions changed the care plan for their patients, and 8 respondents provided additional information about these changes. For those participants who did have opportunities to implement new skills, they shared examples such as, “changes how I complete some physical assessments; changed how I order routine tests,” “completely changed plan for child with seizure disorder,” “helped to identify different management strategies related to topic areas,” “common GT/GJ issues,” and “targeted assessment.” The emphasis on best practice recommendations was also well received. A pediatrician participant stated, “understanding best practices also helps me understand how to counsel my patients better.” Another pediatrician participant stated that they gained information on the “trial of Atrovent for sialorrhea, less aggressive investigation for someone with neuro-irritability.”

FIGURE 2

Perceived practice impact of Pediatric Project ECHO for Complex Care after 6 months of participation (n = 27; because of rounding percentages, they may not total to 100%). a 1 missing response. N/A, not applicable.

FIGURE 2

Perceived practice impact of Pediatric Project ECHO for Complex Care after 6 months of participation (n = 27; because of rounding percentages, they may not total to 100%). a 1 missing response. N/A, not applicable.

Close modal

With this study, we sought to develop the first ECHO program focused on CMC and evaluate its feasibility and preliminary impact. Our results indicate that the program is feasible to implement, with participation levels averaging 19 HCPs per session. Scores of acceptability also exceeded a priori thresholds for success. On average, participants reported improved knowledge and enhanced self-efficacy after 6 months. Most participants indicated that the program had a positive impact on various aspects of their practice, including their ability to provide quality care to CMC.

When professional education is inadequate, the care received by CMC can be fragmented and of poor quality.19  Project ECHO facilitated the delivery of continuing education to HCPs from hospital and community settings that were focused on common clinical issues. These sessions are important for building system capacity and facilitating conversations between care providers from different centers. These types of relationships are particularly helpful in navigating the clinical care of CMC, which is often provided without sound evidence.2,10,19,21,22 

It is recognized that there is a deficit in CMC curriculum development within residency education.9,10,20  The brief needs assessment completed within this study represents the first attempt at designing an interprofessional curriculum for established professionals delivering care to CMC. Our needs assessment identified knowledge gaps and provided a scaffold for the curriculum design. These gaps highlight the need for continuing education within complex care, particularly considering its varied inclusion in postgraduate curricula.22  Future in-depth and targeted needs assessments are required to address ongoing education requirements related to CMC.

By virtually connecting professionals, Project ECHO overcomes barriers of geographical disparity and accessibility.13  Our program was able to engage HCPs who were located nearly 900 miles from the hub. Participant characteristics were similar to those of other ECHO programs piloted in Canada and the United States.19,2325  Participants were largely female and <49 years of age and represented a variety of disciplines.14,19,24  Although the marketing of Pediatric Project ECHO for Complex Care was primarily aimed by the program funder to target primary care physicians, we found that a broader group of HCPs were engaged. The composition of participants, predominately registered nurses, nurse practitioners, and physicians, is reflective of the interdisciplinary nature of complex care practice. This professional diversity was purposefully cultivated among the hub team (see Fig 1) and, then, naturally occurred among spoke participants. Thus, interprofessional education evolved, as both the hub and spoke participants mirrored the interprofessional care provided to CMC. On average, Pediatric Project ECHO for Complex Care participants had more years of clinical experience than those from comparator programs,19,23  which may reflect initial program marketing through established complex care networks.

In this study, we captured evaluation metrics across the first 2 years of program implementation. A 6-month observation period was satisfactory to assess trends in feasibility and perceived impacts. Other study strengths were the knowledge and self-efficacy assessments, which were developed in conjunction with curriculum development and designed to account for differing scopes of practice among participants.

Survey response rates limit the generalizability of findings. Although surveys were distributed electronically with standardized reminders, resource limitations prevented follow-up by telephone. In addition, a large number of registered participants did not attend a TeleECHO clinic; in previous work, it was found that the timing of clinics was a major barrier to attendance.26  Outcome measures relied on self-report and, thus, reflect perceived changes in knowledge, self-efficacy, and clinical practice impact. The lack of a control group limits the inferences that can be drawn from changes in outcomes because such changes could stem from involvement in other non-ECHO education programs. However, on the basis of the literature and clinical knowledge of our team, other robust educational opportunities related to the care of CMC are limited.

The ECHO model is designed to create a virtual community of practice with all participants (hub and spokes) learning from each other. It is envisioned that community HCPs who consistently participate in Pediatric Project ECHO will eventually support the local management of CMC.17  These providers may then serve as local resources for their colleagues to consult on their complex cases. By facilitating the transfer of skills from specialists to community HCPs, ECHO has the potential to increase system capacity to provide care to CMC in their own communities. In future work, researchers will examine the extent to which these impacts occur (eg, changes in referral patterns) as the program matures. The next steps for research will include longitudinal assessment of outcomes after 12 months in the program. In future work, researchers will also seek to assess the health outcomes of children treated by providers who are part of the Pediatric Project ECHO for Complex Care community of practice.

In this study, we demonstrate that Pediatric Project ECHO for Complex Care is feasible to implement and associated with improvements in perceived knowledge and self-efficacy among interdisciplinary HCPs who care for CMC. Furthermore, HCPs report positive practice impacts due to program participation. These results suggest that ECHO is a worthwhile educational model to support the growing population of children with significant medical fragility.

We thank Dr Garry Salisbury, Ms Annette Guillemette, and Dr Julia Alleyne (Ontario Ministry of Health) for championing this project. Dr Andrea Furlan, Ms Jane Zhao (Project ECHO at University Health Network), and Dr Eva Serhal (Project ECHO Ontario Mental Health at Centre for Addiction and Mental Health) have provided valued guidance since program inception. Ms Pam Hubley (The Hospital for Sick Children) is the executive sponsor for this project. Thank you to all members of the Pediatric Project ECHO advisory committee (The Hospital for Sick Children). Members of the ECHO Hub team include Kathy Netten (social worker), Ashley Edwards (social worker), Laura Vresk (registered dietitian), Jordan Beaulieu (registered dietitian), Erika Gibson (registered dietitian), and Jas Otal (pharmacist). Ms Erinn McCarthy (The Hospital for Sick Children) was responsible for data collection, as the research study coordinator. Mr Stevanus Soegiaman and Mr Ramon Cespedes (The Hospital for Sick Children) provide telemedicine support for all ECHO clinics. Cynthia Nguyen (The Hospital for Sick Children) supported the funding proposal to the Ontario Ministry of Health. Operational oversight for Pediatric Project ECHO for Complex Care is provided by the SickKids Learning Institute, overseen by Kelly McMillen and Kelly Warmington. Linda Nguyen (The Hospital for Sick Children) is the Interprofessional Education Specialist for Pediatric Project ECHO. Lastly, we thank the ECHO Institute (University of New Mexico) for ongoing training and support.

Dr Lalloo conceptualized the study, oversaw data acquisition, analysis, and interpretation, drafted the initial manuscript, and approved the final manuscript as submitted; Dr Diskin contributed to delivery of Pediatric Project Extension for Community Healthcare Outcomes (ECHO) for Complex Care as a member of the academic hub, contributed to data interpretation, and drafted portions of the manuscript; Ms Ho, Drs Orkin, and Cohen are the medical leads responsible for delivering the Pediatric Project ECHO for Complex Care program, contributed to data interpretation, and reviewed and revised the manuscript; Ms Osei-Twum conducted initial data analyses and reviewed and revised the manuscript; Mr Hundert conducted statistical analyses, contributed to data interpretation, and reviewed and revised the manuscript; Ms Jiwan, Ms Sivarajah, and Ms Gumapac are the operational team responsible for delivering the Pediatric Project ECHO for Complex Care program, contributed to data acquisition, and reviewed and revised the manuscript; Dr Stinson oversaw all aspects of the research study as principal investigator of the Pediatric Project ECHO for Complex Care program, contributed to data interpretation, reviewed and revised the manuscript, and approved the final manuscript as submitted; and all authors approved the final manuscript as submitted.

FUNDING: Pediatric Project ECHO for Complex Care is fully funded by the Ontario Ministry of Health in Canada.

1
Cohen
E
,
Patel
H
.
Responding to the rising number of children living with complex chronic conditions
.
CMAJ
.
2014
;
186
(
16
):
1199
1200
2
Bell
LM
.
Estimates of the unrealized costs of caring for children with medical complexity
.
NEJM J Watch
.
2019
. Available at: https://www.jwatch.org/na48272/2019/01/22/estimates-unrealized-costs-caring-children-with-medical
3
Cohen
E
,
Berry
JG
,
Camacho
X
,
Anderson
G
,
Wodchis
W
,
Guttmann
A
.
Patterns and costs of health care use of children with medical complexity
.
Pediatrics
.
2012
;
130
(
6
).
4
Cohen
E
,
Kuo
DZ
,
Agrawal
R
, et al
.
Children with medical complexity: an emerging population for clinical and research initiatives
.
Pediatrics
.
2011
;
127
(
3
):
529
538
5
Lach
LM
,
Kohen
DE
,
Garner
RE
, et al
.
The health and psychosocial functioning of caregivers of children with neurodevelopmental disorders
.
Disabil Rehabil
.
2009
;
31
(
9
):
741
752
6
Broyles
RS
,
Tyson
JE
,
Heyne
ET
, et al
.
Comprehensive follow-up care and life-threatening illnesses among high-risk infants: a randomized controlled trial
.
JAMA
.
2000
;
284
(
16
):
2070
2076
7
Coller
RJ
,
Ehlenbach
ML
.
Making time to coordinate care for children with medical complexity
.
Pediatrics
.
2019
;
143
(
1
):
e20182958
8
Berry
JG
,
Poduri
A
,
Bonkowsky
JL
, et al
.
Trends in resource utilization by children with neurological impairment in the United States inpatient health care system: a repeat cross-sectional study
.
PLoS Med
.
2012
;
9
(
1
):
e1001158
9
Agrawal
R
.
Complex care in pediatrics: great progress, greater challenges
.
J Pediatr Rehabil Med
.
2015
;
8
(
2
):
71
74
10
Kuo
DZ
,
Houtrow
AJ
;
Council on Children with Disabilities
.
Recognition and management of medical complexity
.
Pediatrics
.
2016
;
138
(
6
):
e20163021
11
Simon
TD
,
Mahant
S
,
Cohen
E
.
Pediatric hospital medicine and children with medical complexity: past, present, and future
.
Curr Probl Pediatr Adolesc Health Care
.
2012
;
42
(
5
):
113
119
12
Bogetz
JF
,
Bogetz
AL
,
Rassbach
CE
,
Gabhart
JM
,
Blankenburg
RL
.
Caring for children with medical complexity: challenges and educational opportunities identified by pediatric residents
.
Acad Pediatr
.
2015
;
15
(
6
):
621
625
13
Hewitt-Taylor
J
.
Caring for children with complex needs: staff education and training
.
J Child Health Care
.
2005
;
9
(
1
):
72
86
14
Bogetz
JF
,
Bogetz
AL
,
Bergman
D
,
Turner
T
,
Blankenburg
R
,
Ballantine
A
.
Challenges and potential solutions to educating learners about pediatric complex care
.
Acad Pediatr
.
2014
;
14
(
6
):
603
609
15
Canadian Association of Paediatric Health Centres
.
CAPHC guideline for the management of medically complex children and youth through the continuum of care. Available at: https://www.macpeds.com/documents/CAPHC%20National%20Complex%20Care%20Guideline%202018_final.pdf. Accessed May 28, 2020
16
Arora
S
,
Thornton
K
,
Murata
G
, et al
.
Outcomes of treatment for hepatitis C virus infection by primary care providers
.
N Engl J Med
.
2011
;
364
(
23
):
2199
2207
17
Arora
S
,
Kalishman
S
,
Thornton
K
, et al
.
Project ECHO (Project Extension for Community Healthcare Outcomes): a national and global model for continuing professional development
.
J Contin Educ Health Prof
.
2016
;
36
(
suppl 1
):
S48
S49
18
University of New Mexico
.
ECHO hubs & programs: global
.
Available at: https://echo.unm.edu/locations/global. Accessed February 20, 2020
19
Furlan
AD
,
Zhao
J
,
Voth
J
, et al
.
Evaluation of an innovative tele-education intervention in chronic pain management for primary care clinicians practicing in underserved areas
.
J Telemed Telecare
.
2019
;
25
(
8
):
484
492
20
Marcinko
J
,
Orkin
J
,
Cohen
E
, et al
.
Paediatric Project ECHO: exploring the educational needs of Ontario healthcare providers with interest in caring for children with medical complexity
.
Paediatr Child Health
.
2018
;
23
(
suppl 1
):
e5
e6
21
Allshouse
C
,
Comeau
M
,
Rodgers
R
,
Wells
N
.
Families of children with medical complexity: a view from the front lines
.
Pediatrics
.
2018
;
141
(
suppl 3
):
S195
S201
22
Huth
K
,
Newman
L
,
Glader
L
.
Core curricular priorities in the care of children with medical complexity: a North American modified Delphi study
.
Acad Pediatr
.
2020
;
20
(
4
):
558
564
23
Sockalingam
S
,
Arena
A
,
Serhal
E
,
Mohri
L
,
Alloo
J
,
Crawford
A
.
Building provincial mental health capacity in primary care: an evaluation of a project ECHO mental health program
.
Acad Psychiatry
.
2018
;
42
(
4
):
451
457
24
Tantillo
M
,
Starr
T
,
Kreipe
R
.
The recruitment and acceptability of a project ECHO® eating disorders clinic: a pilot study of telementoring for primary medical and behavioral health care practitioners
.
Eat Disord
.
2020
;
28
(
3
):
230
255
25
Tiyyagura
G
,
Asnes
AG
,
Leventhal
JM
, et al
.
Impact of project ECHO on community ED providers’ perceptions of child abuse knowledge and access to subspecialists for child abuse and neglect
.
Acad Pediatr
.
2019
;
19
(
8
):
985
987
26
Morano
J
,
McKillop
A
,
Jiwan
A
,
Lalloo
C
,
Stinson
J
.
Characterizing reasons for low attendance in the Paediatric Project ECHO Program. In: MetaECHO 2019; March 13–16, 2019; Albuquerque, NM

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

Supplementary data