The Quality and Management of Penicillin Allergy Labels in Pediatric Primary Care

A subset of 500 of 18 015 children with penicillin allergy labels (including allergy to penicillin, amoxicillin, amoxicillin-clavulanate) born from January 1, 2010 to June 30, 2020 were randomly selected from a retrospective, dual-center birth cohort study.²⁸  Children’s Hospital of Philadelphia (CHOP) and Texas Children’s Hospital are both associated with primary care clinics (CHOP Care Network and Texas Children’s Pediatrics [TCP]), in addition to specialty clinics (including allergy and immunology), urgent care clinics, emergency care centers, and hospitals. Both centers and associated clinics have used the EpicCare EMR exclusively since 2010. The Institutional Review Boards at Baylor College of Medicine and Children’s Hospital of Philadelphia approved this study.

Basic demographics, healthcare encounters, medications, and allergy tab documentation (including reaction, reaction type, severity, comments, dates of modification, and credentials of individuals who made modifications) were extracted from the EMR on all patients in the birth cohort from birth until a patient’s censor date (date when children no longer met criteria for birth cohort as described previously)²⁸  or date of data extraction (November 12, 2020 and December 13, 2020 for TCP and CHOP, respectively). Additional healthcare notes (including telephone, primary care, allergy clinic [in network and out-of-network if available via the media tab], and other encounters) of the 500 randomly selected children were reviewed to determine (1) the nature of the inciting adverse drug event, (2) outcomes of formal penicillin allergy evaluations, and (3) consequences of penicillin re-exposure in children who had their label removed (ie, “delabeled”).

Allergy risk classification was determined based on the reaction characteristics and timing listed in (1) the allergy tab (including comments) and (2) remaining healthcare notes and encounters (Fig 1). Risk classifications were adopted from the “Clinical Pathway for the Assessment of Children with a Penicillin Drug Allergy” from CHOP,²⁹  with 1 modification after consulting with an allergy specialist (Table 1).

Categorical and continuous demographic variables were reported as percentages or median (interquartile range). A Cohen’s κ statistic (ĸ) was used to calculate the measure of agreement in allergy risk classification between the allergy tab and healthcare documentation. Interpretation of ĸ values were made according to Landis and Koch, where ĸ < 0.20 is consistent with poor agreement, ĸ = 0.21–0.40 fair agreement, ĸ = 0.41–0.60 moderate agreement, ĸ = 0.61–0.80 substantial agreement, and ĸ > 0.81 almost perfect agreement.³⁰  The χ² test was used to compare categorical variables. Multivariable logistic regression analyses were used to determine which a priori variables were associated with penicillin allergy referrals and delabeling. The variables that were selected a priori based on prior clinical knowledge included: sex, race and ethnicity, payer status, history of chronic condition, site (A or B representing CHOP or TCP, in no particular order), and penicillin allergy risk classification as determined by healthcare notes. Race and ethnicity, payer status, and chronic condition were determined as previously described.²⁸  Presence of additional antibiotic and nonantibiotic allergy labels, credentials of the labeling provider, and whether a patient was seen by a primary care physician (PCP) within 7 days of the allergic reaction were additionally included in the multivariable analysis for penicillin allergy referrals. History of a completed appointment at an allergy and immunology clinic was included in the multivariable analysis for penicillin allergy label removal.

Of 334 465 children who met criteria for the birth cohort, 18 015 (5.4%) were labeled as penicillin allergic. Demographics of the 500 children with penicillin allergy labels who were randomly selected for retrospective chart review were similar to all labeled children from the birth cohort (Table 2).

Most penicillin allergy labels were placed by physicians (303 of 500, 62.7%), followed by medical assistants (67 of 500, 13.4%), nurses (57 of 500, 11.4%), and advanced practice providers (41 of 500, 8.2%). The most common penicillin class antibiotic allergies were amoxicillin (368 of 500, 73.6%), amoxicillin-clavulanate (67 of 500, 13.4%), and penicillin (65 of 500, 13%). Additional nonpenicillin antibiotic allergies were noted in 46 of 500 (9.2%) children, most commonly third generation cephalosporins (32 of 46, 69.6%), macrolides (7 of 46, 15.2%), and clindamycin (5 of 46, 10.9%).

The most common adverse drug reaction reported in the allergy tab was rash (223 of 500, 44.6%), followed by hives (161 of 500, 32.2%), and no reaction documented (81 of 500,16.2%). Allergy severity and reaction type were listed in 55 of 500 (11.0%) and 72 of 500 (14.4%) of labels, respectively, and comments were used in 165 of 500 (33.0%) of labels. After evaluating the allergy tab, penicillin allergy label risk classification could only be determined in half of cases (Fig 1) and “moderate or high risk” was the most common (188 of 250, 75.2%).

Based on the allergy tab, there was no difference in proportion of penicillin allergy labels that could be risk classified (ie, “low risk,” “moderate or high risk,” “severe cutaneous reaction,” or “not allergy”) compared with labels that could not be risk classified (ie, “unable to classify) if the label was placed by a physician (155 of 303, 51.2%) compared to other types of healthcare personnel (93 of 197, 47%, P = 0.31). Use of the comments box within the allergy tab improved the ability to risk classify over penicillin allergy labels without comments (106 of 165 [64%] vs 144 of 335 [43.0%], P < .001).

The inciting adverse reaction to a penicillin antibiotic was further described in the EMR healthcare notes (including telephone and encounters) of 422 of 500 children (84.6%, Fig 1). The median number of days into an antibiotic course that a reaction was reported was 6 (interquartile range [IQR] 3–8) (Supplemental Fig 2). Most (329 of 422, 78.0%) children were evaluated at a primary care clinic within 7 days of their inciting adverse reaction, whereas 62 of 422 (14.7%) children were evaluated by an emergency care center or urgent care center within 24 hours of their reaction, and 46 of 422 (10.9%) communicated their reaction only via telephone.

The most common penicillin allergy label risk classifications based on review of the healthcare document ation were “moderate or high risk” (185 of 500, 37.0%) and “low risk” (157 of 500, 31.4%); 48 of 500 (9.6%) penicillin allergy labels were classified as “not allergy” (Fig 1). In 197 children with documentation of outcomes of penicillin allergy symptoms, 82 (42%) had persistent symptoms beyond 24 hours after stopping the antibiotic.

The measure of agreement of allergy risk classification between the allergy tab and other EMR documentation was fair (ĸ = 0.35 [SD 0.02], Table 3). Of the 250 labels “unable to classify” based on the allergy tab, 141 (56.4%) were either “low risk” or “not allergy” based on the healthcare notes (Table 3). Only half (24 of 48) of penicillin allergy labels classified as “not allergy” by the notes were reported as “not allergy” in the allergy tab.

Of children with penicillin allergy labels, 84 of 500 (16.8%) were referred to an allergist for penicillin allergy evaluation, but only 54 of 500 (10.8%) were seen by an allergist during the study period, and penicillin skin testing was performed in 25 of 500 (5.0%). No child who underwent penicillin skin testing had a positive skin test, and 23 of 24 (96%) subsequently tolerated oral amoxicillin challenge in a follow-up appointment. Presence of at least 1 additional antibiotic allergy (adjusted odds ratio [aOR] 6.92 [95% confidence interval (CI) 3.25–14.72]), a non-antibiotic allergy (aOR 1.92 [95% CI 1.05–3.52]), site B (aOR 5.43 [95% CI 2.93–10.06]), and Hispanic (aOR 2.40 [95% CI 1.12–5.14]) or non-Hispanic Asian (aOR 3.36 [1.08–10.52]) ethnicity and race were associated with increased odds of allergy referral on a multivariable analysis (Supplemental Table 5). The rate of allergy referral for penicillin allergy evaluation was higher among children with two additional non-penicillin antibiotic allergies (4 of 5, 80%) compared with 1 non-penicillin antibiotic allergy (14 of 42, 33%, P = .042).

Children were labeled at a median age of 1.4 years (IQR 0.9–2.2). Removal of penicillin allergy labels was uncommon (69 of 500, 13.8%) and occurred a median of 0.7 years (IQR 0.3–2.4) after labeling. Penicillin allergy label removal occurred more often following an allergy appointment (26 of 54, 48%) compared with children who were not seen by an allergist (43 of 446, 9.6%, P < .001). History of a completed appointment with an allergist (aOR 14.92 [95% CI 6.96–31.94]) and having a penicillin allergy label classified as “not allergy” (aOR 23.00 [95% CI 3.69–143.03]) were associated with increased rates of delabeling on a multivariable regression analysis (Supplemental Table 6). Non-Hispanic Black race and ethnicity was associated with a decreased rate of label removal compared with non-Hispanic white (aOR 0.11 [95% CI 0.02–0.56], Supplemental Table 2).

PCPs were most likely to remove penicillin allergy labels classified as “low-risk” or “not allergy,” whereas allergists delabeled children with a variety of allergy risk classifications (Supplemental Table 7). The most common reasons children were delabeled in the community were that a physician reviewed the medical history and concluded that symptoms were inconsistent with an immunoglobulin E (IgE)-mediated reaction (13 of 43, 30%), the family reported the child had outgrown their allergy or did not believe the symptoms were related to drug allergy (11 of 43, 26%), or the label had been erroneously placed in the setting of family history or diarrhea (ie, “not allergy,” 7 of 43, 16%, Table 4). Most children evaluated in allergy clinics were delabeled after a negative penicillin skin test or graded oral challenge (23 of 26, 89%).

Of the children who had penicillin allergy labels removed from the EMR, 48 of 69 (69.6%) subsequently received a penicillin derivative. No child who was delabeled suffered an anaphylactic or other severe allergic reaction following penicillin re-exposure, and 45 of 48 (94%) tolerated penicillin re-exposure without relabeling or any minor adverse reaction. Children delabeled in the primary care setting were just as likely to tolerate penicillin after being delabeled compared with those who were delabeled by an allergist (94% vs 93%, P = .87). Only 2 of 69 (3%) children were relabeled during the study period, which occurred 3 and 5 years after the initial delabeling, despite these children tolerating multiple courses of amoxicillin after they were delabeled. One child experienced a minor adverse drug event during amoxicillin re-exposure but was not relabeled as penicillin allergic.

Although our prior birth cohort study²⁸  identified factors associated with having a penicillin allergy label, this in-depth substudy scrutinized the quality of those penicillin allergy labels and determined how those labels were managed in the primary care setting. Across 2 large pediatric primary care networks, penicillin allergy documentation in the allergy tab of the EMR was uninformative and did not provide accurate allergy risk classification. Children with penicillin allergy labels were infrequently referred to an allergy specialist from the community. Although the rate of appropriate label removal was higher among children evaluated at an allergy and immunology clinic, PCPs successfully delabeled children with low-risk PALs or PALs that were inconsistent with allergy. Among children who were delabeled by allergists and pediatricians alike, no anaphylactic or other severe adverse reactions occurred following re-exposure to penicillin derivatives.

The quality of penicillin allergy label documentation within EMR allergy tabs has been previously scrutinized in adults. Studies of adult penicillin allergy labels have shown that 20% to 36% of allergy tabs lack documentation of allergy reaction characteristics,^31,32  and surveyed family medicine physicians and pharmacists have reported skepticism about penicillin allergy documentation within EMRs.³³  In our study, pediatric documentation within the allergy tab was similarly poor. The use of the comments box improved allergy risk classification but was only used in one-third of cases. After review of healthcare notes, we found that most children did not meet criteria for higher risk IgE-mediated hypersensitivity reactions.³⁴  Determining the likelihood of IgE-mediated hypersensitivity reactions (“immediate” reactions) versus benign, delayed maculopapular rashes retrospectively (ie, when evaluated in primary care well child visits) relies on accurate history and/or documentation of prior adverse drug reaction (1) characteristics and (2) timing. In this study, most penicillin allergy tabs incompletely captured these important pieces of the reaction, likely facilitated by a general lack of standardized requirement for allergy documentation. The consequence of poor documentation, however, is that unconfirmed penicillin allergy labels can persist unchallenged in the EMR for years, and at future visits, families may not recall details of prior adverse reactions. A framework on how to improve documentation within the EMR has been previously published by Blumenthal, et al in 2016.³⁵  Recently, the American Academy of Allergy, Asthma, and Immunology published recommendations on improving allergy documentation within electronic health records, including use of comments and free text to accurately document the type and timing of rashes.³⁶  Future studies should explore creative ways to leverage the pediatric EMR allergy tab to facilitate accurate documentation at the time of an adverse drug event and subsequently determine if improving documentation results in more appropriate referrals and formal evaluations.

The American Academy of Allergy, Asthma, and Immunology recommends penicillin allergy testing be routinely performed in patients with self-reported penicillin allergy.³⁷  Many pediatric penicillin allergy evaluations require at least 2 appointments with an allergy specialist: 1 to confirm the allergy history and the second for penicillin skin testing and/or supervised oral amoxicillin challenge. Penicillin skin testing is not advised for all children with penicillin allergies; instead, confirmatory testing via a graded oral amoxicillin challenge has been advised among children with low risk symptoms (eg, benign rashes).³⁸  In our study, we found that few children with a history of penicillin allergy were referred to an allergy and immunology clinic, even fewer were seen at least once in an allergy and immunology clinic, and less than 5% completed the recommended penicillin allergy evaluation. In prior surveys, physicians reported feeling unclear of indications for referring allergic patients or reporting feeling unaware of available allergy services.^39,40  However, a 2021 quality improvement study found that best practice alerts in an academic general pediatric primary care clinic setting increased the number of referrals for penicillin allergy evaluations from 1.9% to 27.8% over 1 year.⁴¹  Additionally, a 2022 quality improvement initiative in an outpatient pediatric clinic increased referral rates to an allergy clinic from 1.9% to 20.4% after implementation of electronic health record alert, combined with education and allergy feedback.⁴²  Therefore, quality improvement efforts to engage with PCPs and/or increase the number of pediatric referrals to allergy and immunology services via targeted education of PCPs, should be supported. Additionally, exploring ways to improve show-rates for the first and second Allergy and immunology appointments for patients who have been referred to an allergist should be pursued. In a survey of 31 adults with penicillin allergy labels, patients were unaware of the benefits of penicillin skin testing.³⁹  Family education and use of telemedicine appointments, primarily for the first appointment for history-taking, could be considered in future studies to improve these show-rates among children with unconfirmed penicillin allergy labels.

In this study, the rates of penicillin allergy label removal were higher among children seen by an allergy and immunology specialist than those who were not. However, pediatric PCPs, who are trusted by families and see children most frequently during periods of wellness (ie, well-child visits), were able to remove “not allergy” and “low risk” labels. None of the children who were successfully delabeled by a PCP were subsequently found to have a severe allergic reaction on re-exposure to penicillin. Therefore, PCPs should be recruited to aid in initial evaluations, clarify the allergy history, and importantly, remove inaccurate labels among children with a family history of penicillin allergy, who have tolerated penicillin antibiotics since being labeled, or whose only symptoms were diarrhea or a yeast infection. For children with low-risk penicillin allergy labels, a graded oral amoxicillin challenge is recommended in a supervised setting, either in the primary care setting or in an allergy and immunology clinic setting.²²  Future work should evaluate other populations of children who can safely have penicillin allergy labels removed by PCPs.

We observed racial and ethnic differences on a multivariable analysis of factors associated with allergy referrals and penicillin allergy delabeling in the community. Compared with non-Hispanic white children, non-Hispanic Asian, and Hispanic children were more likely to be referred to an allergist but just as likely to be delabeled, whereas non-Hispanic black children were just as likely to be referred but less likely to be delabeled. The process of allergy referral, specialist scheduling, and allergy delabeling, whether at an allergist or primary care pediatrician’s office, relies on consistent access to care, and reliable communication with the healthcare provider. Therefore, these data may support a growing body of literature suggesting systematic barriers to care.^28,43–45  Future quality improvement studies that aim to improve access to allergy services for penicillin evaluations should ensure that interventions are equitable and address disparities in care.

A strength of this study was the use of detailed chart review, allowing allergy risk classification at the level of the allergy tab and encounter notes. Detailed chart review also allowed us to study the outcomes of penicillin skin testing and penicillin re-exposure in the community. Additionally, unlike prior studies that have examined the accuracy of penicillin allergy labels among patients referred to allergy clinics^19,21,46  or evaluated in hospitals or emergency centers,^32,47–49  we studied the accuracy of penicillin allergy labels among children in the community, including children who were never seen by an allergist or hospitalized. Use of a birth cohort also allowed us to better characterize the care of children with penicillin allergy labels who have ongoing relationships with their pediatricians and are likely to seek specialty and emergency care in these healthcare networks. However, because of the complexity of chart review, we only studied a small proportion of the 18 015 children with penicillin allergy labels in the birth cohort, which may have precluded identifying additional factors associated with allergy referrals or delabeling in the community. Reassuringly, demographics of the 500 randomly selected children from the 2 centers were similar to those of children labeled as penicillin allergic in the larger birth cohort. Another limitation in this retrospective chart review study is that risk classifications were based on physician documentation. Finally, although chosen from a large birth cohort including more than 90 practices and 334 465 patients, these data may not be generalizable to the larger pediatric primary care population.

The quality of penicillin allergy label documentation across 2 large pediatric primary care networks was poor. Despite guideline recommendations, children with penicillin allergy labels were rarely referred to an allergy specialist for formal evaluation. Allergists more commonly removed inaccurate labels from the EMR, but PCPs were able to remove low-risk penicillin allergy labels or those not consistent with an allergy safely in the community. PCPs should be empowered to address unconfirmed penicillin allergy labels in the community, and future studies to improve equitable access to allergy and immunology services should be pursued.

We thank George Glenn, Adam Witas, Gee Mathen, Deepthi Gunturi, and Hareesh Gunturi for their assistance in initial data collection for the retrospective birth cohort study.

CHOP

Children’s Hospital of Philadelphia

EMR

electronic medical record

PCP

primary care physician

TCP

Texas Children’s Pediatrics