To evaluate usability of and clinician satisfaction with the electronic health record (EHR) in the context of caring for children with medical complexity (CMC) at a large academic pediatric hospital and to identify key areas for targeted improvements.
Cross-sectional study of pediatric faculty and advanced practice providers across several pediatric specialties using an online Research Electronic Data Capture survey. EHR usability was measured with 6 validated questions from the National Usability-Focused Health Information System Scale, and satisfaction with common EHR functionalities was measured with 6 original Likert-scale questions and 3 free-text questions. Results were analyzed with bivariate testing.
More than half of providers completed the survey (n = 81, response rate 56%). When asked 6 positively-phrased, validated usability questions, a majority of respondents either agreed or were neutral. Respondents were most dissatisfied with information quality of the summary view and most satisfied with physician communication. Older respondents were less satisfied with EHR usability (P < .01). Focusing on functionalities needed for CMC, the majority of respondents were dissatisfied with the chart review process. More respondents were dissatisfied with order entry (P = .002) and documentation (P = .017) when caring for CMC compared with caring for other patients. The most cited challenges were locating recent patient data, performing an accurate medication reconciliation, and lack of specialized documentation templates.
Clinicians are less satisfied with common EHR functionalities when caring for CMC compared with caring for all other patients. Targeted interventions to improve usability and common EHR functionalities for CMC are necessary to improve the user experience.
“Children with medical complexity” (CMC) is a “person-first” term to describe medically fragile pediatric patients with intensive health care needs.1 The increase in CMC cases can be attributed to advances in managing severe, chronic illness and resulting improved survival rates.1,2 In a study of pediatric hospitals spanning 2004 to 2009, the number of hospitalized patients categorized as chronically ill increased annually by 3.6%.2 In 2009, 56.2% of patients in 28 children’s hospitals studied were classified as having a chronic illness and accounted for 81.7% of hospital days and 86.1% of hospital charges.2
CMC have diverging disease processes, but share increased resource utilization secondary to medical technology demands, polypharmacy, complex care coordination, and frequent and prolonged hospitalizations.1 Their complex needs put CMC at risk for adverse events during hospitalization, including suboptimal management from unfamiliar providers, medication errors, and mistakes during transition to the outpatient setting.3–5
Although electronic health records (EHRs) are widely used in pediatric hospitals, they lack tools optimized for pediatric health care and pediatricians report poor functionality and EHR burden as contributors to reduced career satisfaction.6–10 Studies in adult centers demonstrated that well-designed EHR tools can reduce cognitive load and errors, improve task prioritization, reduce documentation time, and improve user satisfaction.11–13
CMC are cared for by multiple clinicians across specialties and locations, making synthesis of the patient’s pertinent EHR data challenging. In the hospital setting, where clinicians must address the patient’s multiple complex needs throughout an encounter, this challenge becomes acute. Few data exist on clinicians’ satisfaction with the EHR when caring for CMC. EHR improvements may alleviate the challenges associated with inpatient CMC care, and further evaluation of clinicians’ needs is warranted. In this study, we evaluated usability of and clinician satisfaction with currently available EHR functionalities in the context of caring for CMC. Findings of this study will highlight areas for intervention in future development of EHR tools targeted at caring for CMC.
Methods
Survey participants included attending physicians and advanced practice providers from the divisions of Pediatric Critical Care, Pediatric Emergency Medicine, Pediatric Hospital Medicine, and Pediatric Pulmonology and Sleep Medicine at a freestanding tertiary care children’s hospital in the southern United States. All participating divisions provide care as a primary service to prehospitalized or hospitalized CMC and all use the EPIC (Verona, Wisconsin) EHR. The division of Pediatric Pulmonology and Sleep Medicine, which operates a robust inpatient service with substantial overlap with general pediatric hospital medicine services, was the only specialty division included in this study. Our cross-sectional study invited participants by e-mail, which included the study information sheet, to complete a 5- to 10-minute Research Electronic Data Capture survey once.14,15 We designed the survey (Supplemental Fig 4) to evaluate EHR usability and satisfaction across 3 domains: Chart review, documentation, and order entry in the context of caring for CMC. Usability questions were adapted from 6 validated questions from the National Usability-Focused Health Information System Scale.16 Satisfaction questions were developed by our study team and included Likert-scale and free-text response questions. We distributed weekly reminders in November and December of 2022 and division chiefs verbally reminded their division members of the opportunity to participate in the survey. Participation was voluntary and no incentive was provided to participants. Survey responses were not linked to individual participants. The institutional review board approved the study with a review exemption.
Data analysis was performed using the Python version 3.9.1 software (Python Software Foundation, Wilmington, Delaware). Demographic data were analyzed using descriptive statistics. Categorical variables representing agreement, neutrality, and disagreement were converted to numerical values for analysis. Values of 1 and 2 were assigned for “agree” and “strongly agree” responses, 0 for neutral responses, and −1 and −2 for “disagree” and “strongly disagree” responses. These values were added to create composite usability and satisfaction scores. We used the χ2 test to explore potential associations between these categorical variables. Composite satisfaction scores were calculated for each division.
Results
We invited 142 participants and received 81 unique survey responses (56% response rate). Respondents were primarily attending physicians (85%), with 11% reporting that they solely work with residents or advanced practice providers, and the remainder reporting that they practice in a combination of independent and supervisory capacities. Respondents primarily identified as female (76%) and 56% were <40 years of age, 37% were between 40 and 55 years of age, and 6% were 55 years or older. The division of Pediatric Hospital Medicine provided the largest number of responses (36, 44%). Respondents were predominantly early career, with 40% reporting <5 years of experience and 30% reporting 6 to 10 years of experience. When asked about experience with the EPIC EHR, 85% of respondents reported >5 years of experience. Just 4% of respondents perceived themselves as having EPIC beginner proficiency, with 49% perceiving intermediate proficiency, 42% advanced, and 5% expert.
When asked 6 validated, positively phrased usability questions, participants overall agreed or were neutral (Fig 1). Participants reported low usability of the summary view because of low information quality, with 58% of participants indicating that they disagreed or strongly disagreed with the statement, “The EHR system generates a summary view that helps to develop an overall picture of the patient's health status.” Participants were most satisfied with statements regarding EHR usability on communicating between physicians and patients or among physicians, with 49% and 73% agreeing or strongly agreeing, respectively.
EHR usability. Respondents' feedback to 6 validated questions about EHR usability, across 3 dimensions including information quality, ease of use, and internal collaboration. The lowest usability rating was for the information quality of the summary view.
EHR usability. Respondents' feedback to 6 validated questions about EHR usability, across 3 dimensions including information quality, ease of use, and internal collaboration. The lowest usability rating was for the information quality of the summary view.
When evaluating usability as a composite score, Pediatric Emergency Medicine clinicians were significantly less satisfied than Pediatric Hospital Medicine clinicians (−0.15 vs 0.37, P = .01) (Fig 2). Participants >55 years of age were significantly less satisfied with usability than younger participants. There were no significant differences in satisfaction with EHR usability across levels of EHR experience or perceived EHR proficiency.
Composite satisfaction scores. Pediatric Emergency Medicine was significantly less satisfied with the EHR when caring for CMC in comparison to Pediatric Hospital Medicine (−0.15 vs 0.37, P = .01). Asterisks indicate divisions with statistically significant differences in satisfaction.
Composite satisfaction scores. Pediatric Emergency Medicine was significantly less satisfied with the EHR when caring for CMC in comparison to Pediatric Hospital Medicine (−0.15 vs 0.37, P = .01). Asterisks indicate divisions with statistically significant differences in satisfaction.
When focusing on 3 domains of EHR functionality (chart review, order entry, and documentation) for all patients and for CMC, clinicians were dissatisfied with chart review for both categories of patients (Fig 3). Clinicians were less satisfied with order entry and documentation for CMC in comparison with other patients (P = .002, P = .017).
Satisfaction with common EHR functionalities. Respondents were asked to compare the common EHR functionalities of chart review, order entry, and documentation for CMC to all other patients. The majority of respondents were dissatisfied with chart review for CMC and significantly fewer respondents were satisfied with order entry and documentation when caring for CMC than other patients.
Satisfaction with common EHR functionalities. Respondents were asked to compare the common EHR functionalities of chart review, order entry, and documentation for CMC to all other patients. The majority of respondents were dissatisfied with chart review for CMC and significantly fewer respondents were satisfied with order entry and documentation when caring for CMC than other patients.
When given the opportunity to provide feedback on the challenges associated with chart review, order entry, and documentation for CMC, there were recurrent themes in responses. Frequently cited challenges included difficulty locating recent and up-to-date patient data, performing an accurate medication reconciliation, and the lack of specialized documentation templates.
Discussion
Our survey results support that clinicians who provide direct inpatient care to CMC are dissatisfied with EHR functionality when caring for this unique patient population. More specifically, usability and satisfaction questions revealed that clinicians are dissatisfied with the process of chart review for all patient populations and are notably more dissatisfied with order entry and documentation functionality when caring for CMC. There was no relationship between satisfaction and experience or self-perceived proficiency with the EPIC EHR.
Our findings support that challenges associated with using the EHR while caring for CMC are not easily remedied with end-user education. Rather, worthwhile interventions to improve usability and satisfaction require development of innovative EHR tools and workflows.
Clinicians from the Division of Pediatric Emergency Medicine had the lowest composite usability scores. This subset of clinicians is required to provide care with limited time and previous knowledge, and their low scores may indicate that retrieving relevant data pertaining to CMC is too difficult or time consuming. This further supports the importance of developing specialized EHR tools and workflows targeting this patient population.
Our study was limited to members of 4 academic divisions in a single pediatric hospital utilizing a single EHR. This limits the generalizability of this study. Future studies should be multicenter and include centers utilizing a variety of EHR products. Additionally, a subset of survey questions was original to this study and not validated in the literature. Although the use of some validated survey questions is a strength of this study, it would be ideal to validate questions focused on EHR tools when caring for special populations in the future.
Conclusions
Clinicians are less satisfied with common EHR functionalities when caring for CMC compared with other patients. Targeted interventions to improve usability and functionalities for CMC are necessary to improve the user experience. To that end, we have begun work on developing a specialized EHR summary view for CMC to aid in efficient and accurate chart review for this unique population, and plan to measure its effect.
Acknowledgments
We thank the University of Texas Southwestern Clinical Informatics Center for the assistance in preparation of this manuscript.
Dr Wilson conceptualized and designed the study and data collection tool, recruited participants, led data collection, analysis, and interpretation, drafted the initial manuscript, and reviewed and revised the manuscript; Mr Diaz contributed to study design, conducted statistical analysis and interpretation of the data, and critically reviewed and revised the manuscript; Dr Lehmann contributed to study design and interpretation of the data, and critically reviewed and revised the manuscript; Dr Maddox supervised and contributed to the conceptualization and design of the study and development of the data collection tool, participated in data collection, analysis, and interpretation, and critically reviewed and revised the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.
FUNDING: Research reported in this publication was supported by the National Center for Advancing Translational Sciences of the National Institutes of Health under award UL1 TR003163. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The funder had no role in the design or conduct of this study.
CONFLICT OF INTEREST DISCLOSURES: The authors have indicated they have no conflicts of interest relevant to this article to disclose.
Comments