A Prediction Model to Determine Childhood Epilepsy After 1 or More Paroxysmal Events

We retrospectively collected clinical data of children referred after 1 or more paroxysmal events, suspected of having an epileptic origin. Data from 451 children who visited the outpatient department of pediatric neurology, University Medical Center Utrecht, between January 2008 and May 2013 were used to build a clinical decision model (development cohort). Data of 187 children who visited the outpatient department of pediatrics, Martini Hospital Groningen, between January 2013 and July 2016 were collected as an external cohort to validate the model (validation cohort). All children were referred by a general practitioner, pediatrician, neurologist, or via the emergency department. In addition to clinical history and physical examination, the report of a standard EEG recording performed at time of first consultation was used. At least 1 year of clinical follow-up was available for all children who were eventually diagnosed with epilepsy and for those whose diagnosis remained inconclusive after initial evaluation. Children in whom diagnosis of epilepsy was discarded directly after first consultation were referred back to their general practitioner with explicit instructions to report new possible seizures. Excluded were children in whom the diagnosis of epilepsy had already been confirmed by the referring health care professional before consultation at the outpatient department. The institutional ethical committees of both hospitals approved the study and concluded that the Dutch Medical Research Involving Human Subjects Act did not apply, and written informed consent was not required.

The outcome measure was presence or absence of epilepsy, defined as: (1) at least 2 unprovoked seizures within 1 year, judged by an experienced child neurologist to be epileptic in origin; or (2) 1 unprovoked seizure and epileptiform EEG abnormalities at first consultation or at later sleep-deprived or prolonged EEG that confirm the diagnosis of an epilepsy syndrome, or the demonstration of a typical epileptogenic brain lesion on MRI. If diagnosis of epilepsy was discarded on the basis of clinical history, EEG results, other ancillary investigations, or at least 1 year of uneventful follow-up, we attempted to reappoint the diagnosis. Children in whom a definitive diagnosis was inconclusive at the end of follow-up, or who were lost to follow-up, were classified as having no epilepsy. The consulting pediatric neurologist always made the definitive diagnosis.

Potential predictor variables were selected from previous cohort studies in children with newly diagnosed epilepsies.^16,–20 Predictor variables were required to be routinely available at the time of first consultation. Information on age at consultation, sex, age at first event, medical history, relevant family history, psychomotor development, number of events and detailed event description, and the EEG report was available and extracted into separate variables in a database. Age at consultation and at first event were truncated at month level. Medical history of a child was categorized into 3 predictor variables, namely “neurologic history” in the presence of a history of perinatal asphyxia, congenital or acquired brain lesions, head trauma, central nervous system infections, or migraine; “psychiatric history” in the presence of a diagnoses of autism, attention-deficit/hyperactivity disorder or other psychiatric disorders; and “known metabolic or genetic syndrome.” Relevant family history was categorized into the presence or absence of epilepsy, febrile seizures, migraine, and seizures in first- or second-degree relatives; or as “other” when consanguinity of parents, cerebrovascular accidents, or developmental delay was present in first- or second-degree relatives. Psychomotor development was categorized as normal or delayed on the basis of IQ (IQ of ≤70 was considered as intellectual disability) and type of schooling. Categorization was not mutually exclusive in these clinical categorical variables. EEG registrations were evaluated by 2 experienced neurophysiologists, and EEG reports were categorized as the presence (or absence) of focal epileptiform abnormalities if (multi)focal spikes or spike-wave-complexes were reported, generalized epileptiform abnormalities when generalized spikes or spike-wave complexes were reported, and aspecific nonepileptiform abnormalities if no clear epileptiform activity was stated but focal or generalized slowing or other aspecific abnormalities were reported.

Clinical symptoms extracted from the event descriptions provided by caregivers and children were categorized into multiple predictor variables, namely the presence or absence of (1) staring or nonattendance, (2) bilateral jerking or shaking, (3) stiffening or tonic posturing, (4) lateralizing motor symptoms, (5) weakness or loss of muscle tone, (6) sensory signs (such as a sensation of tingling and visual or auditory symptoms), (7) automatism, or (8) autonomic symptoms. Standard seizure classification, as recently proposed by the International League Against Epilepsy (ILAE),²¹ could not be applied to categorize event description because the majority of the events were eventually nonepileptic in nature.

Two clinicians (E.v.D. and H.J.L.), blinded for the diagnosis at follow-up, classified the event description into the aforementioned predictor variables on the basis of the clinical report. The reproducibility of this classification was tested in a random sample of 20% of patients by an experienced pediatric epileptologist (F.E.J.). Interobserver agreement was calculated with Cohen’s κ coefficient.²² A κ of 1 indicates perfect agreement; 0 indicates agreement equivalent to chance.

Data were missing for the event frequency in 167 (37%) and 31 (17%) children in the training cohort and validation cohort, respectively, and for the age at first event in 147 (33%) and 28 (15%) children, respectively. All other variables were missing in <5 cases (≤1%). We used single imputation for missing values to prevent biased results toward the complete case subjects.^23,24 On the basis of the imputed data set, a logistic regression model was built to estimate the log odds of the epilepsy probability as a linear function of the predictor variables. Backward selection using the Akaike information criterion was applied to select the strongest contributing predictor variables that were included in the full model.

The regression model was fitted on the basis of the clinical data from the test cohort (n = 451). The clinical data from the validation cohort (n = 187) was used for external model validation. We tested whether fitting the continuous predictor variables as restricted splines improved the logistic model, which was not the case, so these were kept included with their original values.²⁵ 

To assess the contribution of the EEG to the predictive value of multivariate models, we developed 2 submodels in addition to the full linear logistic regression model: 1 based only on clinical predictor variables and 1 with only EEG predictor variables.

The performance of the model was assessed by using a receiver operating characteristic (ROC) curve. Sensitivity, specificity, balanced accuracy, positive predictive values (PPVs), and negative predictive values (NPVs) were computed for different epilepsy probability thresholds, covering the relevant range for clinical decision-making (20%–80%). Analyses were performed in R (http://www.R-project.org).

Data of the training and validation cohort are summarized in Table 1. Accurate information on age at first event was available for 304 children (67%) in the training cohort and 159 children (85%) in the validation cohort. The number of events could be reliably determined for 275 children (61%) and 156 children (83%), respectively. In 203 children (45.0%) of the training cohort, the diagnosis was inconclusive after first consultation and ancillary investigations or follow-up was required; in the validation cohort, the diagnosis was originally inconclusive in 54 children (29%). After at least 1 year of follow-up, a definite classifying diagnosis (epilepsy or not) was available for 425 children (94.2%), and for 26 (5.8%) it remained inconclusive in the training cohort. For the validation cohort, the definite diagnosis was available for 172 children (92%) and remained uncertain for 15 (8%) children (Fig 1). Eventually, 30% of the children from the training cohort received the diagnosis of epilepsy compared with 84 (45%) from the validation cohort. In 1 patient, the initial diagnosis (no epilepsy) changed into epilepsy during follow-up. We provided the etiology (Supplemental Table 4) according to the proposed ILAE classification when possible.²⁶ 

The interobserver agreement of event description was excellent, with a Cohen’s κ coefficient of 0.91. The multivariable odds ratio of each predictor variable that contributed to the full model (ie, clinical and EEG data) is provided in Table 2. The predictor variables with the highest odds ratios were those from the EEG report: the presence or absence of generalized epileptiform activity, focal epileptiform activity, or aspecific nonepileptic abnormalities. High odds ratios were also found for the event description predictor variables on automatisms and weakness or loss of muscle tone and for the predictor variable on the presence of a neurologic history.

Figure 2 shows ROC curves for the full multivariate model and the 2 submodels as tested on the external validation cohort of 187 children. Excellent discrimination was found for the full model, combining clinical and EEG predictor variables: area under the receiver operating characteristic curve (AUC) of 0.86 (confidence interval [CI] 0.80–0.92). The corresponding sensitivity, specificity, PPV, and NPV are provided in Table 3 for different risk thresholds. The submodel based on only EEG predictor variables outperformed the model based on clinical predictor variables only: AUC of 0.82 (CI 0.77–0.88) versus AUC of 0.67 (CI 0.59–0.74), respectively.

Model performance based on the subgroup of children in whom diagnosis was inconclusive after first consultation from our validation cohort (n = 54) was good, with an AUC of 0.73 (CI 0.58–0.87) when combining EEG and clinical factors (ROC curve in Supplemental Fig 4). Corresponding sensitivity, specificity, PPV, NPV, and balance accuracy values are provided in Supplemental Table 5.

We built a Web application (http://epilepsypredictiontools.info/first-consultation) (Supplemental Table 6) that enables clinicians to explore the added clinical value of the model in daily practice and compute the probability for the diagnosis of epilepsy for an individual child who presents with 1 or more paroxysmal events.

In this study, we show that the diagnosis of epilepsy can be computed with a high likelihood in children at first consultation after 1 or more paroxysmal events, with a limited set of predictor variables and the availability of a first interictal EEG.

Early and accurate diagnosis (or exclusion) of epilepsy after a suspected event will diminish patients’ ignorance but also wrongly applied stigmata, will improve self-management, and prevent unnecessary antiepileptic drug treatment and the burden of noncontributing ancillary investigations in children who are wrongly diagnosed. Our model provides a rational approach to assist clinicians during the diagnostic process by combining routinely available clinical information in a multivariate way. More specifically, we expect our model to be useful as an “independent” screening tool to assess the likelihood of a possible seizure to be epileptic in origin and to help the clinician decide on the need for ancillary investigations or refer to an epileptologist.

We consciously do not propose a single cutoff value for clinical decision-making because this is a decision rather than a diagnostic model. Our prediction tool can help the clinician to decide whether ancillary investigations or referral to an epileptologist are necessary, which is especially preferable for children where the risk is neither high nor low. Additionally, high-risk cases are identified quickly, and appropriate actions can be taken early in the process (see Fig 3). Therefore, the merit of the model lies mainly with the general (pediatric) neurologist or pediatrician with access to EEG. In addition, it could be of use for the epileptologist to add in the prognosis of clinically more complicated cases in which diagnosis remains unclear after first consultation.

Predictor variables obtained from the EEG report strongly contributed to the discriminative power of the full multivariate model, which reflects daily practice. History taking is without doubt an important diagnostic instrument because it provides a coherent sequence of all the information provided.²⁸ However, subjective reporting might obscure its value for clinical decision-making. The submodel based on EEG predictors alone showed a superior performance as compared with the submodel with only clinical predictor variables. Nevertheless, after assessing the most important aspect, whether the event is epileptic in origin, a treating physician aims to classify the specific seizure type and epilepsy syndrome to initiate adequate treatment and predict prognosis. Here, clinical information is indispensable, and a standard EEG recording alone will be insufficient. Conversely, EEG predictors showed an inferior performance compared with clinical predictors when tested on children in whom diagnosis remained uncertain after first consultation (Supplemental Fig 4). Again, this reflects the actual clinical situation in which incongruent EEG findings might obfuscate clinical diagnosis.

A main strength of this study is the clinical applicability of the model. We collected data from a cohort of children after 1 or more paroxysmal events with at least 1 year of follow-up, rather than restricting ourselves to a predefined patient cohort, and built the model with clinical data that are routinely collected in standard care. We included all newly presented patients irrespective of the number and nature of possible events before consultation. We believe that this approach will allow a better translation to daily practice because actual first seizures are often underrecognized and may only be recalled in retrospect.^4,7,15 Secondly, clinical follow-up was available in all children diagnosed with epilepsy. This follow-up adds crucial information to the model and prevents it from a circular reasoning, particularly in children who presented with a single seizure and had epileptiform EEG activity. Finally, model performance was tested with an external cohort and a subcohort of clinically more difficult cases. The discrimination of the model was good in these validations. We do not claim to present a diagnostic tool; obviously, it is up to the clinician to finally conclude whether a child may truly have epilepsy and whether ancillary diagnostics or treatment is indicated.

This study has limitations. The incidence of epilepsy in both our patients’ cohorts was lower than in other comparable cohorts,^7,29,30 which might reflect a more liberal referral and selection policy to our clinics and may hamper its applicability to other medical centers. Second, although PPV and NPV remain excellent at higher-risk thresholds (Table 3), sensitivity is lower at high thresholds. This is a limitation of the model but not necessarily a problem in daily practice. If, for a specific patient, the clinician wants to minimize the risk of falsely rejecting the diagnosis of epilepsy, clinical follow-up and ancillary investigations will likely be performed. Increasing the number of children in future validation studies allows the inclusion of more predictors that could be useful to boost the sensitivity of the model at higher-risk thresholds. Third, we used generic categories based on unambiguous symptoms provided by the children or their caregivers to classify the event description. Although the interobserver variability was low, suggesting a reliable classification, an accurate event description depends on a good observation, and some symptoms might be difficult to recognize or recall. Information on event description was collected for most cases through a standardized questionnaire at the outpatient department and processed retrospectively for our study. Collecting data prospectively could further diminish missing data and improve uniformity of data, thereby possibly improving strength of predictor variables. No explicit follow-up data were collected for the children in whom the diagnosis of epilepsy was discarded directly after first consultation, which could have introduced false-negatives in our outcome variable. Nonetheless, we think the percentage of false-negatives is low because these children were referred back to their general practitioner or referral specialist with explicit instructions to report new suspected events, if any, and there is a close collaboration with health care providers in the region of the 2 institutions that participated in this study. In addition, previous research in a similar cohort has revealed that the risk of false-negatives is small.²⁹ Similarly, a subset of children in whom a single seizure was evaluated to be epileptic in origin was not diagnosed with epilepsy if ancillary investigations did not reveal epileptiform abnormalities or underlying epileptogenic brain lesions. Some of those children might develop epilepsy in the time beyond our follow-up period.^13,14 

Clinical models to assess the diagnosis of epilepsy based on clinical information available at first consultation in children with one or paroxysmal events are currently lacking. Our results indicate that a multivariable decision model may be valuable at first consultation in children presenting with possible seizures. Future research should evaluate the added value of a decision model in clinical practice to manage expectations of patients and caregivers to prompt interventions.

AUC

area under the receiver operating characteristic curve

CI

confidence interval

ILAE

International League Against Epilepsy

NPV

negative predictive value

PPV

positive predictive value

ROC

receiver operating characteristic