Identifying Acute Neuropsychiatric Events in Children and Adolescents Free

This was a cross-sectional study of children 5 to 18 years of age hospitalized at two US children’s hospitals in the Midwest and South between April 1, 2016 and March 30, 2020 with a discharge International Classification of Diseases, 10^th Revision (ICD-10)-coded NPE diagnosis (Supplemental Table 3).^1,14,15  The selection of NPE diagnosis codes was informed by previous studies using administrative data to capture NPE^1,14  and pediatric mental health disorders¹⁵  as well as the review of ICD-10 code books and discussion with content experts (pediatric psychiatrists, pediatricians, hospitalists, pharmacoepidemiologists, and coding experts) to evaluate coding practices (Supplemental Table 3).

Children 5 to 18 years of age admitted to one of two academic children’s hospitals were included in the study. Children < 5 years of age were excluded from the study because neuropsychiatric diagnoses in this age group are not well defined. The study was restricted to hospitalized patients to better identify clinically significant NPEs similar to other Food and Drug Administration studies in children.¹³  We used a proportional sampling strategy stratified by diagnosis code position (primary or any secondary) to randomly select 1098 encounters with a qualifying NPE diagnosis out of a total of 33 658 hospitalizations with an NPE from the two study sites between April 1, 2016 and March 30, 2020. Electronic medical records were manually reviewed for each encounter and data extracted for analysis included demographic data, race, and ethnicity designation (as reported in the electronic health record) and concurrent diagnoses. Demographic data were included for descriptive purposes only. Complex chronic conditions were classified by using the pediatric complex chronic conditions classification system, version 2.¹⁶ 

NPE codes were categorized by clinical significance (eg, critical, important, or unclear) and subtype (eg, neurologic or psychiatric) on the basis of expert opinion (Supplemental Table 3). The critical category includes homicidal, suicidal, or self-harm ideation or attempts; the important category includes mood disorders (including anxiety and stress), psychosis, hallucination, altered mental status, ataxia or movement disorders, encephalitis or encephalopathy, and seizures; and the unclear significance category consisted of dizziness, headache, sleeping disorders, and vision changes.

The primary outcome was a physician-confirmed NPE that was present on admission and directly related to hospitalization. This gold standard outcome definition of a confirmed case was used to evaluate each qualifying NPE encounter. To be considered a confirmed case, the description of the event (as listed in history of present illness, physical examination, or assessment and plan), was required to match the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition definition for psychiatric events or be consistent with a description of a prespecified neurologic event. To determine if the NPE was related to the hospitalization, a pediatrician reviewed each case and determined whether the NPE was not related to hospitalization, an active hospital problem present on admission and related to hospitalization, not present on admission but active hospital problem, or a problem present on admission but not related to hospitalization. Reviewers were blinded to the NPE diagnosis and algorithm information. A true positive was defined as an active hospital problem present on admission and related to hospitalization. Independent reviews of a random sample of 50 records by two pediatricians at each site demonstrated excellent interrate reliability with 92% concordance in NPE determinations (κ = 0.80).

We evaluated 3 NPE algorithms using an iterative approach (Supplemental Table 4): (1) NPE diagnosis in either primary or any secondary diagnosis fields, (2) identical to algorithm 1 but with the requirement that the NPE was present on admission and the primary diagnosis was not malignancy- or surgery-related, and (3) identical to algorithm 2 but without requiring the NPE be present on admission. For the algorithms, presence on admission was determined by the presence of a “present on admission” flag assigned to the diagnosis. The present on admission flag is assigned for hospitalized patients and is a data element available in administrative databases.

NPE diagnoses were categorized on the basis of clinical significance on the basis of expert opinion (Supplemental Table 3). We determined the PPV of the proposed identification algorithms, overall and stratified by diagnosis order (primary or any secondary), clinical significance (critical, important, unclear, critical, or important), and NPE subtype (neurologic or psychiatric). Details of the primary and secondary algorithms identified can be found in Supplemental Table 4. The institutional review board at our institution determined that this study of deidentified data did not constitute human subjects research (IRB 201677).

There were 1098 NPE hospitalizations included in the study (Table 1). A total of 857 confirmed NPEs were identified by algorithm 1, yielding a PPV of 0.78 (95% confidence interval [CI] 0.76–0.80). Algorithm 2 examined 846 confirmed NPE hospitalizations and had an overall PPV of 0.89 (95% CI 0.87–0.91). For algorithm 3 (938 confirmed NPE hospitalization), the overall PPV was 0.86 (95% CI 0.83–0.88). PPVs varied by diagnosis order, NPE clinical significance, and subtype. The PPV for critical clinical significance was 0.99 (0.97–0.99) for all 3 algorithms (Table 2).

We report 3 algorithms for the identification of incident pediatric NPE with PPVs between 0.78 and 0.89. Algorithm 2, which required both NPE present on admission status and the exclusion of surgical and malignancy primary diagnoses, had the highest PPV while retaining more than 88% of true positive cases. Cases of critical or important clinical significance comprised the most cases and were most accurately identified with algorithm 2. This algorithm specifically identifies acute NPEs related to hospitalization rather than NPEs that occurred during hospitalization but unrelated to admission (eg, ICU delirium) and those present on admission but unrelated to hospitalization (eg, chronic conditions).

Although algorithm 1 has been used for the identification of NPEs in previous studies,^1,14,17  we found that this approach introduces misclassification of NPEs. The enhanced algorithms presented in this study more accurately identify NPEs and may reduce outcome misclassification on relative measures of association.

Pediatric NPEs are commonly reported as a result of underlying chronic conditions, infections, medications, and other exposures.^2,11,18,19  Evaluating new onset NPEs in observational data is challenging because of the low overall frequency of the events. Studies that examine the association between medications and NPEs are usually focused on identifying new or incident NPEs. To enable proper assessments of medication effects, accurate identification of NPEs is necessary. In addition, it is important to identify the timing of the NPEs relative to the study medication use (eg, whether NPEs were already present), as well as other potential explanations that may introduce misclassification of outcomes (eg, NPEs that started during the immediate postoperative period or chemotherapy) and distort study findings. Acknowledging that there are multiple ways to identify NPEs, our study evaluated the performance of different approaches to accurately identify incident NPEs to inform the selection of outcome measurements for future studies.

This study used data from twochildren’s hospitals. We did not assess NPEs in the ambulatory setting or NPEs that did not result in health care encounters, limiting applicability to milder NPEs and other settings. We did not identify true or false negative cases of NPE and, therefore, could not determine the sensitivity, specificity, or negative predictive value of the algorithms. However, we consider that our determination of PPVs is the most informative element for studies that plan to evaluate the association between medication use and NPEs.²⁰ 

The enhanced algorithms presented in this study accurately identify NPEs resulting in pediatric hospitalizations. The use of these approaches can reduce the misclassification of study outcomes and improve the rigor of future studies of NPE. In particular, these approaches may aid in the timely evaluation of NPEs related to new exposures, such as SARS-CoV-2 infections,^19,21  and investigating NPE signals in adverse events reporting databases.^9,11,22,23  These approaches may also be used to evaluate the effectiveness and safety of treatments for chronic diseases such as depression, bipolar disorder, or epilepsy.