Parent-Child Agreement on Postconcussion Symptoms in the Acute Postinjury Period

This was a planned secondary analysis of a data set collected in the derivation phase of the 5P study.^11,14  The 5P study was a prospective multicenter cohort study that recruited participants in 9 Canadian pediatric emergency departments (PEDs), members of the Pediatric Emergency Research Canada network. A total of 2006 children were enrolled in the derivation phase between August 2013 and September 2014. Eligible patients were aged 5 to 17.99 years, presented to a participating PED with a head injury that occurred within the preceding 48 hours, and met criteria for a diagnosis of concussion as per the 2012 Zurich consensus statement. Details about patient selection and general procedures have been published previously.^11,14  Our study was approved by the research ethics board of each participating institution, and written consent and assent were obtained from all participants and their parents as appropriate.

At the PED, both patients and parents quantified preinjury and current symptoms using the PCSI^15,17  and provided information on their history and general levels of functioning. In our previous studies, we developed a clinical prediction rule for PPCSs in children and adolescents,^11,14  in which PPCSs were classified as 3 new or worsening symptoms lasting at least 4 weeks after a concussion (International Statistical Classification of Diseases and Related Health Problems, 10th Revision definition¹⁸ ). Because the child PCSI has 3 separation versions, the initial clinical prediction rule for PPCSs used the parent PCSI reports to avoid differences in symptoms listed and response scaling present across the 3 versions for children of different age groups. Nine factors assessed in the PED were found to predict PPCSs at 28 days in children and adolescents by using parent reports. We sought to compare the predictive power of this previously derived and validated clinical rule with the predictive power of a model using child PCSI reports.

The PCSI is a set of symptom scales for parents (20-item, 7-point Guttman scale) and developmentally specific self-report forms for children aged 5 to 7 (5-item, 3-point scale), children aged 8 to 12 (17-item, 3-point scale), and 13- to 18-year-olds (19-item, 7-point scale). The symptoms are loaded onto 4 different factors (ie, clusters), which include physical, cognitive, emotional, and fatigue domains.¹⁵  A summary question is also included to rate, on a scale of 0 to 4, what degree the child is acting differently from before the injury. This question encompasses a holistic view of recovery after a concussion and reflects the kind of question primary care providers not using specific symptom scales would rely on. Patients and parents completed the PCSI in PEDs using the standard PCSI administration procedures and instructions. In this study, we used the most up to date PCSI version at the time study enrollment began in August 2013. A newer iteration of the PCSI that appends the symptom “sleep more” into the 13- to 18-year-old assessment has been subsequently developed; however, this item was not evaluated in the PCSI version used in this study. The PCSI is a commonly used symptom scale and is a recommended common data element according to the National Institute of Neurological Disorders and Stroke and the US Department of Defense.¹⁹ 

The means and SDs for each item on the scale were computed overall and by age group. Item scores for the 5 to 7 and 8 to 12 age groups, whose responses were on a 0 to 2 rating, were scaled by a factor of 3 to allow comparisons across age groups and with parent scores. Summary statistics were computed for the total PCSI score, factors (physical, fatigue, emotional, and cognitive), and the “feels different” summary question for each age group. Items, factors, and overall scores were examined by using the reported preinjury, postinjury, and an adjusted score, the latter of which is defined as the difference between the preinjury score and postinjury score.

To assess agreement between child and parent reports, 2-way mixed, absolute measure intraclass correlation coefficients (ICCs) were calculated. ICCs were computed for all items, factors, and overall scores reported for preinjury, postinjury, and adjusted scores for all age groups. ICCs could only be calculated for those items in which both parent and child items were available. We considered ICC < 0.50 as poor, 0.50 < ICC < 0.75 as fair, 0.75 < ICC < 0.90 as moderate, and ICC > 0.90 as excellent.²⁰ 

The 5P rule¹¹  was reevaluated by using child reports for PCSI items included in the original rule. Therefore, a multiple logistic regression was performed with 9 items: age group, sex, duration of previous concussion, personal history of migraine headaches, Balance Error Scoring System tandem stance errors, answers questions slowly (Acute Concussion Evaluation), headache (child PCSI), sensitivity to noise (child PCSI), and fatigue (child PCSI). Missingness was handled via listwise deletion. Adjusted odds ratios and the respective 95% confidence intervals (CIs) were calculated. Validation was evaluated with a correlated receiver operating characteristic (ROC) analysis. Delong’s ROC analysis was also used to compare the child PCSI report model with the parent-report model and a model with physician prediction alone. Analyses were performed by using SPSS Statistics versions 21 and 23 (IBM SPSS Statistics, IBM Corporation) and R version 3.0.2. Two-sided P values <.05 were considered statistically significant.

A total of 2006 children and adolescents participated in this study. Descriptive characteristics of the sample are presented in Table 1.

Descriptive statistics for child and parent PCSI reports are shown in Table 2 (child) and Table 3 (parent). Mean preinjury scores (scaled) for the child PCSI ranged from 0.06 (answers questions more slowly) to 0.48 (headache). The mean child PCSI postconcussion scores (scaled) ranged from 0.99 (irritable) to 4.02 (headache). The largest mean difference in child PCSI scores could be found for the item headache (mean difference score = 3.53). The “feels different” child summary score, which was only used to assess postinjury, ranged from 1.47 (5–7-year-old version) to 2.19 (13–18-year-old version). Parent mean item preinjury scores ranged from 0.06 (dizziness) to 0.39 (difficulty concentrating), and postconcussion scores (scaled) ranged from 0.88 (nervous) to 4.05 (headache). The largest mean difference adjusted score was for the item headache (mean adjusted score = 3.39). The “feels different” parental summary score, which only assessed postinjury, ranged from 1.70 (8–12-year-old version) to 1.83 (13–18-year-old version).

The ICC for the total PCSI adjusted score was fair (ICC = 0.66). Physical (ICC = 0.71), cognitive (ICC = 0.62), and fatigue (ICC = 0.59) factor scores also had fair agreement, whereas emotional factors revealed poor agreement (ICC = 0.44). Between age groups, the ICC for the total PCSI adjusted score was highest for the 13- to 18-year-olds (ICC = 0.84) and lowest for the 5- to 7-year-olds (ICC = 0.12). For all age groups, the physical factor always generated the greatest agreement.

The lowest agreement among preinjury scores across age groups was found for answers questions more slowly (ICC = 0.25) and the highest agreement was found for headache and sensitivity to light (both ICC = 0.56). Postconcussion score agreement was generally higher. The lowest postconcussion agreement was found for irritable (ICC = 0.42) and the highest for nausea (ICC = 0.70). Between age groups, agreement in preinjury scores, postconcussion scores, and adjusted scores was consistently greatest for the 13- to 18-year-olds and lowest for the 5- to 7-year-olds.

Overall agreement for this question across age group was poor, with an ICC of 0.39. As per other PCSI items, 13- to 18-year-olds and their parents reported higher agreement (ICC = 0.55) than 8- to 12- or 5- to 7-year-olds, with ICCs of 0.34 and 0.22, respectively. The ICCs for all parent-child agreement outcomes can be found in Table 4.

The clinical prediction rule, using child-reported PCSI, had similar predictive power to the parent-reported model (child report: area under the curve [AUC] = 0.70 [95% CI: 0.67–0.72]; parent report: AUC = 0.71 [95% CI: 0.68–0.74]), as presented in Fig 1. DeLong’s test for 2 correlated ROC curves yields a nonstatistically significant P value of .23 (D = −1.1977; degree of freedom = 1641.657) when comparing the rule with parent-reported and child-reported PCSI items, demonstrating the similarity between the models. When comparing the child-reported model with physician prediction alone¹¹  (AUC = 0.55; 95% CI: 0.50–0.59), Delong’s test yields a statistically significant difference (P < .001; D = −4.72672; degree of freedom = 11.98.963).

Table 5 presents the odds ratios and 95% CIs for each predictor in the parent- and child-reported PCSI models. Although headache and fatigue were both statistically significant in the model using parent-reported PCSI, they were not statistically significant in the model using child-reported PCSI (P = .08 and P = .30, respectively).

Overall, parent and child agreement on symptom reporting was fair, particularly for total and factor scores. Parent-child agreement was greater (1) when asked about postconcussion symptoms, (2) for physical symptoms, and (3) for older children. Despite some differences in individual items, applying the clinical prediction rule using the child-reported PCSI maintained similar predictive power to the original rule developed with the parent-reported PCSI and had better predictive abilities than physician opinion alone. Overall, the poor agreement for younger children (≤12 years) suggests that it is important for an experienced interviewer to interact with both the parent and child to obtain a complete picture of functioning. Although clinicians could likely rely on parental reporting for physical or cognitive symptoms, emotional impairments may be better ascertained by directly interviewing the child regardless of age.

The overall fair parent-child agreement observed in this study was largely driven by postinjury scores within the physical and cognitive factors. The emotional domain consistently displayed the poorest parent-child agreement, confirming earlier work.¹⁵  In both preinjury and postconcussion contexts, physical health typically produces more observable symptoms that can be recognized by the parent than mental or emotional health, which reflect more internal states.²¹  Children, especially young children, may not recognize or conceptualize subtle symptoms. This contributes to poorer agreement in emotional domains, illustrates the difficulty for children to express more complex concepts, and represents a challenge for parents to observe behaviors that would illustrate impairments in the emotional spheres. Our findings are similar to those reported in studies by researchers investigating quality of life or reporting on the experience of traumatic events, in which agreement is consistently greater for domains that are easily observed.^22–25 

Younger children show less agreement with their parents than adolescents at both preinjury and postconcussion time points. This could be explained by developmental difficulties in self-appraisal of impairments in younger children. Young children may have difficulty with accurately verbalizing pain intensity and location,²⁶  and this ability is refined throughout childhood and adolescence, especially for more complex concepts. Adolescents also had better parental agreement than younger children for the PCSI “feels different” summary question. The overall fair agreement for this question, as well as better agreement with increasing age, replicate previous findings.²⁷  The interpretation of this question may vary between parent and child, in turn influencing responses. Additional exploration of what both children and parents understand or conceptualize from this question is needed.

Because postconcussion symptoms have an important role in injury prognosis, evaluating parent-child agreement on concussion outcomes, such as persistent symptom risk, is an important addition to the literature. The original 5P rule was developed to help clinicians orient children to appropriate concussion management in the acute period and includes 9 predictor variables, each providing risk points for the presence of PPCSs 28 days after injury.^11,14  Of the 9 variables, 3 are derived from the PCSI (headache, sensitivity to noise, and fatigue). Although 2 of the significant symptoms observed on the parent PCSI reports did not reach significance when using the child PCSI reports, the predictive ability of the overall models were not significantly different. Similar to the parent-derived model, the child model also led to better predictive ability that physician opinion in the PED. This may provide reassurance that symptom reports from either the parent or the child in the context of the busy PED would yield similar results when implementing the 5P clinical prediction rule.

In previous studies, authors evaluated parent-child agreement on the PCSI in subacute and chronic settings^13,15,16 ; in our study, we are the first to assess agreement at an acute time point (median: 2.8 hours after concussion). Understanding parent-child agreement and establishing PPCS risk by using the clinical prediction rule acutely can help clinicians direct treatment and refer patients to rehabilitation centers as necessary to prevent prolonged recovery. In this study, we enrolled a large, diverse sample of children and youth. Patients were included under a wide age range (5–17 years), under various mechanisms of injury, and with premorbid behavioral, learning, and psychological problems. This heterogeneous sample increases the generalizability of our findings. The PCSI is a concussion symptom checklist with developmentally appropriate versions that have been validated specifically for children and adolescents.

Although the psychometric properties and developmental appropriateness of the PCSI are strengths, the PCSI is limited by the lack of equivalence among the different child versions. All responses on the 5- to 7- and 8- to 12-year-old versions (0–2 scale) were multiplied by 3 to make equivalent ratings (0–6 scale) for all parent and child PCSI assessments. This assumes that a 1-point discrepancy on the 0 to 2 scale is equal to a 3-point discrepancy on a 0 to 6 scale, which may not be an appropriate assumption and may contribute to the lower ICC values observed in the younger age groups. However, we believe that rescaling is a sensible strategy to overcome this inherent limitation of our assessment tool.

Patients and parents completed the PCSI in the same room and were allowed to communicate throughout the assessment, including for clarification if the child did not comprehend a symptom. This may limit the internal validity of findings; however, this improves external validity because these are the standard administration procedures used in clinical settings nationally and internationally. The authors additionally feel that this did not meaningfully inflate agreement metrics because agreement between parents and the youngest children, who would most likely need to communicate with their parent, was the lowest of any age group. The PCSI was specifically designed and validated to have 3 versions, such that the symptom scales are developmentally appropriate for children aged 5 to 18. However, the child’s comprehension of the PCSI was not directly assessed in this study. Because younger children are more likely to have difficulty comprehending the PCSI, this may have contributed to poorer parent-child agreement among the youngest participants and should be evaluated further in future studies. Therefore, although younger children did display poorer agreement with their parents than adolescents, comparing agreement between the age groups (5–7, 8–12, and 13–18 years) should be done cautiously because of the different rating scales and potential issues with comprehension.

Because patients were recruited in the PED, they may have different symptom burdens, PPCS risk, and patient characteristics from individuals who do not report to the PED. However, we observed similar rates of PPCSs as those established in outpatient settings.^6,28  No objective biomarkers or gold standard is currently available for concussion or PPCS diagnosis,¹  and the risk prediction score generated by the 5P rule may be less precise than those available for other illnesses or diseases. However, the outcome measures we used are validated tools created by using the best evidence currently available. Patients and parents were asked to retrospectively rate the child’s preinjury level of symptoms. This may induce recall bias, but this methodology has been used in the literature^11,29  when a baseline assessment of healthy functioning is unavailable.

Self-assessment of postconcussion symptoms is a critical component of acute concussion assessment and monitoring of recovery. In the absence of specific fluid or imaging biomarkers, clinicians rely not only on a thorough neurologic and physical examination but also on patient-reported symptoms to establish a diagnosis and make appropriate management decisions. Although obtaining information from both children and parents is considered best practice, this may not be feasible in busy clinical settings (such as PEDs). Knowing whether informant-related differences exist and, if so, for which symptoms could help clinicians orient their interview more efficiently.

Parent and child agreement on symptom reporting by using the PCSI was generally fair, particularly for total and factor scores. Two symptom items (fatigue and headache) included in the original 5P clinical prediction rule using parent-reported PCSI failed to reach significance when using the child-reported PCSI; however, the child-reported PCSI still had similar predictive power to the original model. Although it remains best practice to obtain both parent and child symptom reports whenever possible, the high level of agreement between parent and child reports and the similar results when implementing the clinical prediction rule can give clinicians confidence that comparable information is gathered when only interviewing the parent or child is possible.

Additional Pediatric Emergency Research Canada concussion team members include Martin H. Osmond, MDCM (Department of Pediatrics, Children’s Hospital of Eastern Ontario, University of Ottawa, Ottawa, Ontario, Canada); Stephen B. Freedman, MDCM, MSc (Department of Pediatrics, Alberta Children’s Hospital, Alberta Children’s Hospital Research Institute, University of Calgary, Alberta, Canada); Jocelyn Gravel, MD (Department of Pediatrics, Centre Hospitalier Universitaire Sainte-Justine, University of Montreal, Montreal, Quebec, Canada); Gurinder Sangha, MD (Department of Pediatrics, Children’s Hospital of Western Ontario, Western University, London, Ontario, Canada); Kathy Boutis, MD (Department of Pediatrics, The Hospital for Sick Children, Toronto, Ontario, Canada); Darcy Beer, MD (Department of Pediatrics, The Children’s Hospital of Winnipeg, Winnipeg, Manitoba, Canada); William Craig, MDCM (Department of Pediatrics, Stollery Children’s Hospital, Edmonton, Alberta, Canada); Emma Burns, MD (Department of Pediatrics, Izaak Walton Killam Health Center, Halifax, Nova Scotia, Canada); and Ken J. Farion, MD (Department of Pediatrics, Children’s Hospital of Eastern Ontario, University of Ottawa, Ottawa, Ontario, Canada).

5P

Predicting and Preventing Postconcussive Problems in Pediatrics

AUC

area under the curve

CI

confidence interval

ICC

intraclass correlation coefficient

PCSI

Postconcussion Symptom Inventory

PED

pediatric emergency department

PPCS

persistent postconcussion symptom

ROC

receiver operating characteristic