BACKGROUND AND OBJECTIVES:

Children born extremely preterm (EP) (<26 weeks’ gestation) have lower cognitive scores and an increased rate of cognitive impairment compared with their term-born peers. However, the neuropsychological presentation of these EP individuals in adulthood has not been described. The aim of this study was to examine neuropsychological outcomes in early adulthood after EP birth in the 1995 EPICure cohort and to investigate if the rate of intellectual impairment changed longitudinally.

METHODS:

A total of 127 young adults born EP and 64 term-born controls had a neuropsychological assessment at 19 years of age examining general cognitive abilities (IQ), visuomotor abilities, prospective memory, and aspects of executive functions and language.

RESULTS:

Adults born EP scored significantly lower than term-born controls across all neuropsychological tests with effect sizes (Cohen’s d) of 0.7 to 1.2. Sixty percent of adults born EP had impairment in at least 1 neuropsychological domain; deficits in general cognitive functioning and visuomotor abilities were most frequent. The proportion of EP participants with an intellectual impairment (IQ <70) increased by 6.7% between 11 and 19 years of age (P = .02). Visuospatial functioning in childhood predicted visuomotor functioning at 19 years.

CONCLUSIONS:

Adults born EP continue to perform lower than their term-born peers in general cognitive abilities as well as across a range of neuropsychological functions, indicating that these young adults do not show improvement overtime. The prevalence of intellectual impairment increased from 11 years into adulthood.

What’s Known on This Subject:

Children born extremely preterm (EP) are at increased risk of cognitive impairment compared with their term-born peers, and the prevalence of serious disability remains stable throughout childhood.

What This Study Adds:

Young adults born EP continue to perform below the level of their term-born peers across a range of neuropsychological functions. The rate of intellectual impairment increased from childhood into adulthood among those born EP.

Researchers examining outcomes of preterm birth in infancy and childhood have reported a high prevalence of neurodevelopmental impairments, with disability increasing with decreasing gestational age.14  The EPICure studies have followed a cohort of infants born extremely preterm (EP) (<26 weeks’ gestation) in the United Kingdom and Ireland during 1995. Assessments of this population at 2.5, 6, and 11 years of age found that as a group, children born EP performed 1.1 to 1.6 SDs lower on measures of general cognitive function in comparison with standardized norms and/or term-born controls57  and were at increased risk of cognitive impairment.6,7 

The prevalence of neurodevelopmental disability in the EPICure cohort at 6 and 11 years, defined as impairment in cognitive, motor, or sensory function, was similar at 45% and 46%, respectively, with cognitive impairment being the most common deficit.7  Furthermore, longitudinal investigation revealed that mean cognitive scores among EP participants remained stable from 2.5 to 19 years of age.8  This is consistent with studies of very preterm (<32 weeks’ gestation) and/or very low birth weight (<1500 g birth weight) individuals,9,10  which confirm that adults born very preterm/very low birth weight continue to function below the level of their peers.11,12  Individuals born very preterm and/or very low birth weight are also at risk for impairment in other neuropsychological domains such as executive function (EF),1315  language,1619  prospective memory,20  and visuomotor skills.21  However, the range and extent of neuropsychological deficits in adulthood after birth before 26 weeks’ gestation have not been described. Understanding the longer-term outcomes for individuals born EP will help to guide the developmental expectations of both caregivers and professionals as well as to identify potential areas in which to target interventions.

Here, we present neuropsychological outcomes at 19 years of age in the 1995 EPICure cohort. The study aimed to address the following questions. (1) Do adults born EP perform significantly below the level of their term-born peers across neuropsychological domains? (2) In which neuropsychological domains do adults born EP display the greatest prevalence of impairment? (3) Does the rate of intellectual impairment remain stable from childhood into early adulthood? (4) Does visuospatial functioning in childhood predict later visuomotor abilities in early adulthood?

Of 306 participants from the EPICure birth cohort, 129 adults born EP (61 male participants) along with 65 term-born controls (25 male participants) participated in the 19-year follow-up. Of these, 127 adults born EP (59 male participants) and 64 controls (25 male participants) had a neuropsychological assessment. This represents 42% of EP participants from the original cohort and 42% of term-born controls assessed at 11 years. Neuropsychological assessment was not completed for 2 EP participants (1 missed appointment; 1 acute psychiatric episode) and 1 term-born control (missed appointment). Recruitment and participation through each of the previous study phases has been described elsewhere.57  EP and term-born participants not seen at 19 years had either declined participation, had asked not to be contacted by the study team, or did not respond to several attempted contacts. Nine EP participants had died since being discharged from the hospital after birth.

All participants gave informed written consent to take part in the 19-year assessment. For participants with severe cognitive impairment, consent was obtained from a parent or guardian. The neuropsychological assessment was conducted by a single psychologist (H.O.) at 19 years and took place at University College Hospital, London. The study was designed to be examiner blind; however, the majority of participants disclosed their group allocation during the course of the assessment. Eleven participants were assessed at home because of disability or other personal commitments. Socioeconomic status (SES) was classified at 19 years on the basis of parent occupation by using the UK Office for National Statistics’ Socioeconomic Classification 2010 system and categorized as (1) higher managerial, administrative, and professional occupations; (2) intermediate occupations; (3) routine and manual occupations; or (4) other (long-term unemployed, student, unclassifiable because of missing data). Ethical approval was granted by the South Central – Hampshire A Research Ethics Committee (reference 13/SC/0514).

General cognitive functioning was assessed by using the Wechsler Abbreviated Scale of Intelligence, Second Edition (WASI-II).22  The 4-subtest version was administered comprising block design, matrix reasoning, vocabulary, and similarities, from which an estimate of full-scale IQ (FSIQ) was derived (mean: 100; SD: 15), with FSIQ being the primary outcome measure. In addition, verbal comprehension index (VCI) and perceptual reasoning index (PRI) standardized scores were derived (mean: 100, SD: 15). Three EP participants had severe cognitive impairment; these participants were assigned an FSIQ score of 40 and a VCI and PRI score of 45, corresponding to the lowest score on each scale. Scores were not imputed for any other measure. Additionally, 2 EP participants with severe motor impairment could not complete the block design subtest; their FSIQ scores were calculated by using the 2-subtest version of the WASI-II.

Three tasks were used to assess components of EFs.

  1. The digit span forward and backward tasks from the Wechsler Adult Intelligence Scale, Fourth Edition,23  were used to measure verbal short-term memory and working memory, respectively. The examiner read out a series of numbers that the participant had to verbally repeat (forward condition), beginning with a 2-number sequence and increasing to a 9-number sequence. For the backward task, the examiner read out a series of numbers that the participant had to repeat in reverse order. A scaled score (mean: 10; SD: 3) was calculated for each task.

  2. Two subtests from the Automated Working Memory Assessment24  (AWMA), dot matrix and spatial recall, were selected to measure visuospatial short-term memory and working memory, respectively. Participants completed this task on a laptop computer. The short-term memory task involved recalling the location of a series of dots presented within a 4 × 4 matrix on the computer screen. The number of dots to recall increased sequentially. Visuospatial working memory was assessed through a mental rotation judgement task combined with simultaneous dot location recall. A standardized score was derived for each task (mean: 100; SD: 15).

  3. Verbal fluency tasks are considered a measure of both language ability and EFs.25,26  The verbal fluency test consisted of 2 tasks: the F-A-S version of phonemic verbal fluency27  (PVF) and semantic verbal fluency (SVF) using the category "animals". For PVF, the participant had 1 minute to generate as many words as possible beginning with each of the letters “F,” “A,” and “S” separately. The number of correct words generated for each letter alone and summed together was recorded. For SVF, the participant had 1 minute to spontaneously generate as many words as possible from the word category “animal,” and the total number of correct responses was recorded.

Visuomotor integration was assessed by using the Beery-Buktenica Developmental Test of Visual-Motor Integration, Sixth Edition (Beery VMI).28  This test was composed of 3 tasks. For the visuomotor integration task, participants copied drawings of geometric shapes. The visual perception task involved matching a target shape within similar distractor shapes. The motor coordination task involved copying geometric drawings within a trail. Each task produced a standardized score (mean: 100; SD: 15).

Prospective memory, comprising both memory and EF skills, involves remembering to conduct a required task or action in the future when cued. This was assessed by using an event-based task with a 2-hour time delay. Participants were requested to write their name on an envelope unprompted when handed one by the examiner later that day. The participant’s response was scored by using the following nominal categories: (1) does not remember, (2) remembers something but does not know what, (3) remembers after being asked, and (4) remembers immediately.

At 6 and 11 years, visuomotor integration was assessed by using 2 subtests from the Developmental Neuropsychological Assessment (NEPSY)29 , design copying and arrows, which comprise the visuospatial processing core domain score (mean: 100; SD: 15). IQ was assessed by using the Kaufman Assessment Battery for Children30  (mean: 100; SD: 15).

Data were analyzed by using Stata 15.1 (Stata Corp, College Station, TX). Mean scores with SDs were calculated for all neuropsychological measures for EP participants and term-born controls. Effect sizes were calculated by using Cohen’s d with a large effect size classified as ≥0.8. The performance of EP participants was compared to that of the controls by using linear regression models. Unadjusted and adjusted (sex and SES) mean differences between groups and their 95% confidence intervals (CIs) are reported. Similar analyses were conducted to examine sex differences within the EP group. Differences in verbal working memory and visual working memory were examined after further controlling for verbal short-term memory and visual short-term memory, respectively. Odds ratios (ORs) for rates of impairment across all measures for EP participants compared to those of term-born controls were estimated by using binary logistic regression models. Term-born controls were used as a reference group, and impairment was classified as a score >2 SDs below their mean score. This was only completed for those measures in which the controls displayed scores >2 SDs below the mean. Similar adjusted analyses were performed. In addition, ORs of intellectual impairment in EP and term-born participants were examined by using the traditional cutoff value of an IQ score of <70. The McNemar test was used to analyze whether the rate of intellectual impairment remained stable from childhood into early adulthood. To examine whether EP participants with cognitive impairment at 11 years were at increased risk of impairment at 19 years, relative risks (RRs) were estimated by using generalized linear models. Correlation analyses and linear regression analyses were used to examine visuospatial scores longitudinally.

No difference was found in birth weight, gestational age, or sex between the young adults born EP who participated in the 19-year study (n = 129) and dropouts (n = 177) (see Supplemental Table 3). Compared with dropouts, a greater proportion of the 19-year EP cohort came from higher socioeconomic backgrounds as recorded at 2.5, 6, and 11 years. Furthermore, dropouts had lower mean IQ and/or cognitive scores at each of the previous assessment points, and a greater proportion scored in the intellectual disability range (IQ score <70) at the 6- and 11-year visits compared with those who participated at 19 years.

There was no difference between the term-born controls who participated in the study at 19 years compared with dropouts in terms of SES or cognitive function measured at 11 years; however, a greater proportion of dropouts were from low to medium SES at 6 years of age.

The sample characteristics of the EP and term participants are shown in Supplemental Table 3; there were no between-group differences in age, sex, or SES at 19 years (calculated by using a t test or χ2 test; all P values >.05). In comparison with their term-born peers, adults born EP had significantly lower scores for FSIQ, VCI, and PRI; visuomotor integration, visual perception, and motor coordination; verbal and visual short-term and working memory; and verbal fluency (Table 1). Adjustment for multiple comparisons was made by using Bonferroni correction (critical value = 0.004) with all findings remaining significant. In comparison to term-born controls, significantly fewer EP participants correctly completed the prospective memory task (Table 2). All between-group differences remained statistically significant after controlling for sex and SES at 19 years. A large effect size was reported across the following measures: the greatest effect was in FSIQ followed by PRI, visuospatial working memory, motor coordination, PVF, visual-motor integration, visuospatial short-term memory, visual perception, VCI, and verbal working memory.

TABLE 1

Neuropsychological Performance of EP Young Adults and Term-Born Controls

EP (n = 127), Mean ± SDTerm-Born Controls (n = 64), Mean ± SDEP Females Versus Male Participants, Mean Difference (95% CI)EP Versus Term, Mean Difference (95% CI)
MaleFemaleAllMaleFemaleAllUnadjustedAdjusted for SESUnadjusted**Adjusted for Sex and SES**Adjusted for Sex, SES, and OthersaCohen’s d
IQ WASIb             
 FSIQc 82.8 ± 17.7 (n = 59) 88.6 ± 15.5 (n = 68) 85.9 ± 16.7 (n = 127) 104.4 ± 11.9 (n = 25) 103.6 ± 9.2 (n = 39) 103.9 ± 10.2 (n = 64) 5.8 (−0.0 to 11.6) 5.3 (−0.8 to 11.5) −18.0 (−22.5 to −13.5) −17.8 (−22.3 to −13.2) — 1.21 
 VCIc 85.9 ± 16.9 (n = 59) 91.5 ± 13.2 (n = 68) 88.9 ± 15.2 (n = 127) 102.4 ± 11.3 (n = 25) 102.3 ± 9.1 (n = 39) 102.4 ± 9.9 (n = 64) 5.6 (0.3 to 10.9)* 5.1 (−0.4 to 10.7) −13.4 (−17.6 to −9.3) −13.2 (−17.4 to −9.0) — 0.98 
 PRIc 83.0 ± 18.6 (n = 57) 87.7 ± 17.6 (n = 68) 85.6 ± 18.1 (n = 125) 105.4 ± 12.8 (n = 25) 103.9 ± 10.3 (n = 39) 104.5 ± 11.3 (n = 64) 4.7 (−1.7 to 11.1) 4.6 (2.2 to 11.4) −18.9 (−23.8 to −14.0) −19.0 (−24.0 to −14.0) — 1.17 
Beery VMI             
 Visuomotor integration 73.4 ± 17.1 (n = 55) 79.7 ± 16.9 (n = 68) 76.9 ± 16.9 (n = 123) 90.4 ± 11.5 (n = 25) 92.8 ± 8.3 (n = 39) 91.8 ± 9.7 (n = 64) 6.3 (0.3 to 12.4)* 5.9 (−0.6 to 12.3) −15.0 (−19.5 to −10.4) −14.4 (−19.0 to −9.8) — 1.00 
 Visual perception 82.7 ± 15.2 (n = 55) 83.8 ± 17.1 (n = 68) 83.3 ± 16.2 (n = 123) 98.4 ± 7.2 (n = 25) 96.0 ± 7.6 (n = 39) 97.0 ± 7.5 (n = 64) 1.2 (−4.7 to 7.0) 0.3 (−5.9 to 6.4) −13.7 (−17.9 to −9.4) −13.6 (−18.0 to −9.3) — 0.98 
 Motor coordination 67.7 ± 17.2 (n = 55) 78.7 ± 15.7 (n = 68) 73.8 ± 16.9 (n = 123) 85.8 ± 11.0 (n = 25) 93.3 ± 8.6 (n = 39) 90.3 ± 10.2 (n = 64) 11.0 (5.2 to 16.8)* 10.6 (4.5 to 16.6)* −16.5 (−21.1 to −12.0) −15.5 (−19.9 to −11.1) — 1.10 
Verbal fluency             
 Letter F 7.2 ± 3.3 (n = 56) 7.4 ± 3.8 (n = 68) 7.3 ± 3.6 (n = 124) 11.0 ± 4.9 (n = 25) 11.2 ± 3.8 (n = 39) 11.1 ± 4.2 (n = 64) 0.2 (−1.1 to 1.5) 0.3 (−1.0 to 1.7) −3.8 (−4.9 to −2.6) −3.8 (−5.0 to −2.6) — 0.99 
 Letter A 6.4 ± 3.1 (n = 56) 6.6 ± 3.4 (n = 68) 6.5 ± 3.3 (n = 124) 10.0 ± 4.2 (n = 25) 9.1 ± 3.6 (n = 39) 9.5 ± 3.8 (n = 64) 0.2 (−1.0 to 1.3) 0.2 (−1.0 to 1.4) −3.0 (−4.1 to −1.9) −3.0 (−4.1 to −2.0) — 0.86 
 Letter S 8.7 ± 3.5 (n = 56) 10.0 ± 4.5 (n = 68) 9.4 ± 4.1 (n = 124) 13.4 ± 3.8 (n = 25) 12.6 ± 3.9 (n = 39) 12.9 ± 3.8 (n = 64) 1.3 (−0.2 to 2.7) 1.4 (−0.2 to 2.9) −3.5 (−4.8 to −2.3) −3.6 (−4.8 to −2.3) — 0.87 
 PVF total 22.3 ± 8.5 (n = 56) 24.0 ± 10.6 (n = 68) 23.2 ± 9.7 (n = 124) 34.4 ± 11.6 (n = 25) 33.0 ± 9.7 (n = 39) 33.5 ± 10.4 (n = 64) 1.7 (−1.8 to 5.1) 1.9 (−1.7 to 5.5) −10.3 (−13.4 to −7.3) −10.4 (−13.5 to −7.3) — 1.03 
 SVF 16.2 ± 5.6 (n = 56) 17.1 ± 5.9 (n = 67) 16.7 ± 5.8 (n = 123) 22.3 ± 5.7 (n = 25) 20.1 ± 5.9 (n = 39) 21.0 ± 5.9 (n = 64) 1.0 (−1.1 to 3.1) 1.2 (−0.8 to 3.1) −4.3 (−6.0 to −2.5) −4.3 (−6.0 to −2.5) — 0.73 
Verbal memoryd             
 Short-term memory 8.6 ± 2.8 (n = 56) 8.1 ± 2.8 (n = 68) 8.3 ± 2.8 (n = 124) 10.2 ± 3.3 (n = 25) 10.4 ± 3.0 (n = 39) 10.3 ± 3.1 (n = 64) −0.5 (−1.5 to 0.5) −0.7 (−1.7 to 0.3) −2.0 (−2.9 to −1.1) −2.0 (−2.9 to -1.1) — 0.69 
 Working memory 8.2 ± 2.5 (n = 56) 8.5 ± 2.6 (n = 67) 8.3 ± 2.6 (n = 123) 10.1 ± 3.0 (n = 25) 10.7 ± 2.9 (n = 39) 10.5 ± 2.9 (n = 64) 0.3 (−0.7 to 1.2) 0.3 (−0.6 to 1.3) −2.1 (−2.9 to −1.3) −2.1 (−2.9 to −1.3) −1.1 (−1.9 to −0.4)* 0.78 
Visuo-spatial memorye             
 Short-term memory 82.3 ± 14.8 (n = 52) 82.2 ± 13.5 (n = 65) 82.2 ± 14.1 (n = 117) 97.9 ± 15.5 (n = 24) 95.7 ± 15.0 (n = 39) 96.5 ± 15.1 (n = 63) −0.1 (−5.3 to 5.1) −0.9 (−6.4 to 4.6) −14.4 (−18.8 to −9.9) −14.0 (−18.5 to −9.4) — 0.99 
 Working memory 87.8 ± 12.4 (n = 49) 87.4 ± 12.5 (n = 64) 87.6 ± 12.4 (n = 113) 103.3 ± 15.9 (n = 23) 102.5 ± 15.7 (n = 39) 102.8 ± 15.6 (n = 62) −0.5 (−5.2 to 4.2) −1.5 (−6.3 to 3.5) −15.2 (−19.5 to −11.0) −15.1 (−19.5 to −10.7) −8.1 (−12.0 to −4.2)** 1.11 
EP (n = 127), Mean ± SDTerm-Born Controls (n = 64), Mean ± SDEP Females Versus Male Participants, Mean Difference (95% CI)EP Versus Term, Mean Difference (95% CI)
MaleFemaleAllMaleFemaleAllUnadjustedAdjusted for SESUnadjusted**Adjusted for Sex and SES**Adjusted for Sex, SES, and OthersaCohen’s d
IQ WASIb             
 FSIQc 82.8 ± 17.7 (n = 59) 88.6 ± 15.5 (n = 68) 85.9 ± 16.7 (n = 127) 104.4 ± 11.9 (n = 25) 103.6 ± 9.2 (n = 39) 103.9 ± 10.2 (n = 64) 5.8 (−0.0 to 11.6) 5.3 (−0.8 to 11.5) −18.0 (−22.5 to −13.5) −17.8 (−22.3 to −13.2) — 1.21 
 VCIc 85.9 ± 16.9 (n = 59) 91.5 ± 13.2 (n = 68) 88.9 ± 15.2 (n = 127) 102.4 ± 11.3 (n = 25) 102.3 ± 9.1 (n = 39) 102.4 ± 9.9 (n = 64) 5.6 (0.3 to 10.9)* 5.1 (−0.4 to 10.7) −13.4 (−17.6 to −9.3) −13.2 (−17.4 to −9.0) — 0.98 
 PRIc 83.0 ± 18.6 (n = 57) 87.7 ± 17.6 (n = 68) 85.6 ± 18.1 (n = 125) 105.4 ± 12.8 (n = 25) 103.9 ± 10.3 (n = 39) 104.5 ± 11.3 (n = 64) 4.7 (−1.7 to 11.1) 4.6 (2.2 to 11.4) −18.9 (−23.8 to −14.0) −19.0 (−24.0 to −14.0) — 1.17 
Beery VMI             
 Visuomotor integration 73.4 ± 17.1 (n = 55) 79.7 ± 16.9 (n = 68) 76.9 ± 16.9 (n = 123) 90.4 ± 11.5 (n = 25) 92.8 ± 8.3 (n = 39) 91.8 ± 9.7 (n = 64) 6.3 (0.3 to 12.4)* 5.9 (−0.6 to 12.3) −15.0 (−19.5 to −10.4) −14.4 (−19.0 to −9.8) — 1.00 
 Visual perception 82.7 ± 15.2 (n = 55) 83.8 ± 17.1 (n = 68) 83.3 ± 16.2 (n = 123) 98.4 ± 7.2 (n = 25) 96.0 ± 7.6 (n = 39) 97.0 ± 7.5 (n = 64) 1.2 (−4.7 to 7.0) 0.3 (−5.9 to 6.4) −13.7 (−17.9 to −9.4) −13.6 (−18.0 to −9.3) — 0.98 
 Motor coordination 67.7 ± 17.2 (n = 55) 78.7 ± 15.7 (n = 68) 73.8 ± 16.9 (n = 123) 85.8 ± 11.0 (n = 25) 93.3 ± 8.6 (n = 39) 90.3 ± 10.2 (n = 64) 11.0 (5.2 to 16.8)* 10.6 (4.5 to 16.6)* −16.5 (−21.1 to −12.0) −15.5 (−19.9 to −11.1) — 1.10 
Verbal fluency             
 Letter F 7.2 ± 3.3 (n = 56) 7.4 ± 3.8 (n = 68) 7.3 ± 3.6 (n = 124) 11.0 ± 4.9 (n = 25) 11.2 ± 3.8 (n = 39) 11.1 ± 4.2 (n = 64) 0.2 (−1.1 to 1.5) 0.3 (−1.0 to 1.7) −3.8 (−4.9 to −2.6) −3.8 (−5.0 to −2.6) — 0.99 
 Letter A 6.4 ± 3.1 (n = 56) 6.6 ± 3.4 (n = 68) 6.5 ± 3.3 (n = 124) 10.0 ± 4.2 (n = 25) 9.1 ± 3.6 (n = 39) 9.5 ± 3.8 (n = 64) 0.2 (−1.0 to 1.3) 0.2 (−1.0 to 1.4) −3.0 (−4.1 to −1.9) −3.0 (−4.1 to −2.0) — 0.86 
 Letter S 8.7 ± 3.5 (n = 56) 10.0 ± 4.5 (n = 68) 9.4 ± 4.1 (n = 124) 13.4 ± 3.8 (n = 25) 12.6 ± 3.9 (n = 39) 12.9 ± 3.8 (n = 64) 1.3 (−0.2 to 2.7) 1.4 (−0.2 to 2.9) −3.5 (−4.8 to −2.3) −3.6 (−4.8 to −2.3) — 0.87 
 PVF total 22.3 ± 8.5 (n = 56) 24.0 ± 10.6 (n = 68) 23.2 ± 9.7 (n = 124) 34.4 ± 11.6 (n = 25) 33.0 ± 9.7 (n = 39) 33.5 ± 10.4 (n = 64) 1.7 (−1.8 to 5.1) 1.9 (−1.7 to 5.5) −10.3 (−13.4 to −7.3) −10.4 (−13.5 to −7.3) — 1.03 
 SVF 16.2 ± 5.6 (n = 56) 17.1 ± 5.9 (n = 67) 16.7 ± 5.8 (n = 123) 22.3 ± 5.7 (n = 25) 20.1 ± 5.9 (n = 39) 21.0 ± 5.9 (n = 64) 1.0 (−1.1 to 3.1) 1.2 (−0.8 to 3.1) −4.3 (−6.0 to −2.5) −4.3 (−6.0 to −2.5) — 0.73 
Verbal memoryd             
 Short-term memory 8.6 ± 2.8 (n = 56) 8.1 ± 2.8 (n = 68) 8.3 ± 2.8 (n = 124) 10.2 ± 3.3 (n = 25) 10.4 ± 3.0 (n = 39) 10.3 ± 3.1 (n = 64) −0.5 (−1.5 to 0.5) −0.7 (−1.7 to 0.3) −2.0 (−2.9 to −1.1) −2.0 (−2.9 to -1.1) — 0.69 
 Working memory 8.2 ± 2.5 (n = 56) 8.5 ± 2.6 (n = 67) 8.3 ± 2.6 (n = 123) 10.1 ± 3.0 (n = 25) 10.7 ± 2.9 (n = 39) 10.5 ± 2.9 (n = 64) 0.3 (−0.7 to 1.2) 0.3 (−0.6 to 1.3) −2.1 (−2.9 to −1.3) −2.1 (−2.9 to −1.3) −1.1 (−1.9 to −0.4)* 0.78 
Visuo-spatial memorye             
 Short-term memory 82.3 ± 14.8 (n = 52) 82.2 ± 13.5 (n = 65) 82.2 ± 14.1 (n = 117) 97.9 ± 15.5 (n = 24) 95.7 ± 15.0 (n = 39) 96.5 ± 15.1 (n = 63) −0.1 (−5.3 to 5.1) −0.9 (−6.4 to 4.6) −14.4 (−18.8 to −9.9) −14.0 (−18.5 to −9.4) — 0.99 
 Working memory 87.8 ± 12.4 (n = 49) 87.4 ± 12.5 (n = 64) 87.6 ± 12.4 (n = 113) 103.3 ± 15.9 (n = 23) 102.5 ± 15.7 (n = 39) 102.8 ± 15.6 (n = 62) −0.5 (−5.2 to 4.2) −1.5 (−6.3 to 3.5) −15.2 (−19.5 to −11.0) −15.1 (−19.5 to −10.7) −8.1 (−12.0 to −4.2)** 1.11 

—, not applicable.

a

Mean differences in verbal working memory and visuospatial working memory when further adjusting for verbal short-term memory and visuospatial short-term memory, respectively.

b

IQ WASI indicates the WASI-II.

c

PRI.

d

Verbal short-term and working memory indicates digit span forward and backward tasks from the Wechsler Adult Intelligence Scale, Fourth Edition.

e

Visuospatial short-term and working memory indicates the dot matrix and spatial recall tasks from the AWMA.

*P < .05

**P < .001.

TABLE 2

Neuropsychological Impairment in EP Young Adults and Term-Born Controls

EP, % (n of N)Term-Born Controls, % (n of N)Unadjusted OR (95% CI)Adjusted for Sex and SES, OR (95% CI)
IQ WASIa     
 FSIQb 44.9 (57 of 127) 3.1 (2 of 64) 25.2 (5.9 to 107.7)* 48.5 (6.5 to 362.6)* 
 FSIQc 15.8 (20 of 127) 0.0 (0 of 64) — — 
 VCIb 33.1 (42 of 127) 1.6 (1 of 64) 31.1 (4.1 to 232.3)* 30.8 (4.1 to 232.0)* 
 VCIc 11.0 (14 of 127) 0.0 (0 of 64) — — 
 PRIb 36.0 (25 of 125) 1.6 (1 of 64) 35.4 (4.8 to 264.2)* — 
 PRIc 19.2 (24 of 125) 0.0 (0 of 64) — — 
Beery VMI     
 Visuomotor integration 36.6 (45 of 123) 4.7 (3 of 64) 11.7 (3.5 to 39.6)* 10.9 (3.2 to 37.4)* 
 Visual perception 39.8 (49 of 123) 6.3 (4 of 64) 9.9 (3.4 to 29.1)* 10.2 (3.4 to 30.2)* 
 Motor coordination 31.7 (39 of 123) 4.7 (3 of 64) 9.4 (2.8 to 32.0)* 8.7 (2.5 to 30.3)* 
Verbal fluency     
 Letter F 8.1 (10 of 124) 0.0 (0 of 64) — — 
 Letter A 3.2 (4 of 124) 0.0 (0 of 64) — — 
 Letter S 17.7 (22 of 124) 1.6 (1 of 64) 13.6 (1.8 to 103.3)* 12.6 (1.6 to 96.5)* 
 PFV total 15.3 (19 of 124) 0.0 (0 of 64) — — 
 SVF 12.2 (15 of 123) 1.6 (1 of 64) 8.8 (1.1 to 67.8)* 8.6 (1.1 to 67.0)* 
Verbal memoryd     
 Short-term memory 8.1 (10 of 124) 1.6 (1 of 64) 5.5 (0.7 to 44.2) 5.4 (0.7 to 43.9) 
 Working memory 9.8 (12 of 123) 0.0 (0 of 64) — — 
Visuospatial memorye     
 Short-term memory 12.0 (14 of 117) 0.0 (0 of 63) — — 
 Working memory 0.0 (0 of 113) 0.0 (0 of 62) — — 
Prospective memory     
 Score <4 17.9 (22 of 123) 4.7 (3 of 64) 4.4 (1.3 to 15.4)* 4.0 (1.1 to 14.1)* 
EP, % (n of N)Term-Born Controls, % (n of N)Unadjusted OR (95% CI)Adjusted for Sex and SES, OR (95% CI)
IQ WASIa     
 FSIQb 44.9 (57 of 127) 3.1 (2 of 64) 25.2 (5.9 to 107.7)* 48.5 (6.5 to 362.6)* 
 FSIQc 15.8 (20 of 127) 0.0 (0 of 64) — — 
 VCIb 33.1 (42 of 127) 1.6 (1 of 64) 31.1 (4.1 to 232.3)* 30.8 (4.1 to 232.0)* 
 VCIc 11.0 (14 of 127) 0.0 (0 of 64) — — 
 PRIb 36.0 (25 of 125) 1.6 (1 of 64) 35.4 (4.8 to 264.2)* — 
 PRIc 19.2 (24 of 125) 0.0 (0 of 64) — — 
Beery VMI     
 Visuomotor integration 36.6 (45 of 123) 4.7 (3 of 64) 11.7 (3.5 to 39.6)* 10.9 (3.2 to 37.4)* 
 Visual perception 39.8 (49 of 123) 6.3 (4 of 64) 9.9 (3.4 to 29.1)* 10.2 (3.4 to 30.2)* 
 Motor coordination 31.7 (39 of 123) 4.7 (3 of 64) 9.4 (2.8 to 32.0)* 8.7 (2.5 to 30.3)* 
Verbal fluency     
 Letter F 8.1 (10 of 124) 0.0 (0 of 64) — — 
 Letter A 3.2 (4 of 124) 0.0 (0 of 64) — — 
 Letter S 17.7 (22 of 124) 1.6 (1 of 64) 13.6 (1.8 to 103.3)* 12.6 (1.6 to 96.5)* 
 PFV total 15.3 (19 of 124) 0.0 (0 of 64) — — 
 SVF 12.2 (15 of 123) 1.6 (1 of 64) 8.8 (1.1 to 67.8)* 8.6 (1.1 to 67.0)* 
Verbal memoryd     
 Short-term memory 8.1 (10 of 124) 1.6 (1 of 64) 5.5 (0.7 to 44.2) 5.4 (0.7 to 43.9) 
 Working memory 9.8 (12 of 123) 0.0 (0 of 64) — — 
Visuospatial memorye     
 Short-term memory 12.0 (14 of 117) 0.0 (0 of 63) — — 
 Working memory 0.0 (0 of 113) 0.0 (0 of 62) — — 
Prospective memory     
 Score <4 17.9 (22 of 123) 4.7 (3 of 64) 4.4 (1.3 to 15.4)* 4.0 (1.1 to 14.1)* 

—, not applicable.

a

IQ WASI indicates the WASI-II.

b

PRI.

c

Using traditional cutoffs to define intellectual disability (score <70).

d

Verbal short-term and working memory indicates digit span forward and backward tasks from the Wechsler Adult Intelligence Scale, Fourth Edition.

e

Visuospatial short-term and working memory indicates the dot matrix and spatial recall tasks from the AWMA.

*P < .05.

Of 119 EP participants assessed at both 11 and 19 years, 10 EP participants (8.4%) at 11 years and 18 EP participants (15.1%) at 19 years had an IQ score <70 (McNemar’s x2 = 5.33; P = .021), which is the standard clinical cutoff for classifying intellectual disability (see Supplemental Fig 3); those who scored in the intellectual disability range at 11 years were at increased risk of scoring in this range at 19 years (RR [95% CI]: 8.72 [4.48 to 16.99]; P < .001). None of the controls had an IQ score <70 at either 11 or 19 years. Impairment was also classified by using scores <−2 SDs with the controls as the reference (see Table 2). Using these criteria, 42 (35.3%) EP participants had a cognitive impairment at 11 years and 53 (44.5%) had a cognitive impairment at 19 years (McNemar’s x2 = 4.84; P = .028); those who had a cognitive impairment at 11 years were at increased risk of deficit at 19 years (RR [95% CI]: 3.56 [2.32 to 5.46]; P < .001). None of the term-born controls had a cognitive impairment at 11 years and 2 (3.1%) had a cognitive impairment at 19 years. Deficits among EP participants were most common in general cognitive functioning and visuomotor abilities. Sixty percent of EP participants had impairment in at least 1 domain compared with 21% of term-born controls, with 35% of EP participants displaying deficits in ≥4 domains (see Fig 1).

FIGURE 1

Number of neuropsychological domains in which impairment is present.

FIGURE 1

Number of neuropsychological domains in which impairment is present.

Close modal

Longitudinal analyses revealed that Beery VMI scores at 19 years were highly correlated with NEPSY visuospatial processing core domain scores at 6 years (EP [n = 109]: r = 0.484, P < .001; control [n = 53]: r = 0.417, P = .002) and 11 years (EP [n = 115]: r = 0.695, P < .001; control [n = 64]: r = 0.389, P = .002). Even after adjustment for sex and SES, NEPSY visuospatial processing core domain scores at 6 and 11 years were both significant predictors of Beery VMI score at 19 years for both control and EP participants (control: P = .001; EP: P < .001; see Supplemental Table 4).

Among EP participants, sex differences were present in the WASI-II VCI and the Beery VMI and motor coordination tasks, with EP female participants achieving higher mean scores than those of male participants (Table 1). After adjustment for SES, only the difference in motor coordination remained significant. IQ scores were evenly distributed across SES categories for both EP participants and term-born controls (see Fig 2).

FIGURE 2

IQ versus SES for EP participants and term-born controls at 19 years of age.

FIGURE 2

IQ versus SES for EP participants and term-born controls at 19 years of age.

Close modal

At 19 years of age, the young adults born EP in the EPICure cohort continued to function below their term-born peers, particularly in general cognitive functioning (IQ) and visuomotor skills, as well as in prospective memory and aspects of language and EFs. Even after adjustment for sex and SES, group differences remained significant. These results are in line with studies of very preterm and/or very low birth weight individuals921 ; but here, we extend these findings to an EP population for the first time. Underperformance relative to their term-born peers extended across a range of neuropsychological functions, with 35% of EP participants displaying deficits in ≥4 domains, again corroborating previous studies of very preterm and/or very low birth weight.13,18  Mean cognitive performance of EP participants at 19 years was comparable to that reported at the previous follow-up visits in childhood,57  indicating that these young adults do not show substantial catch-up over time.8 

Although the EP participants were found to be functioning significantly below the level of their peers, on average, their performance on standardized measures was within the low average range. Pyhälä et al11  suggest that although the scores of preterm individuals may fall within the normal range, poorer neuropsychological functioning could result in lower educational attainment, earning potential and poorer physical health. Indeed, general cognitive abilities were found to predict adult wealth in a study of very preterm and/or very low birth weight, with the very preterm and/or very low birth weight adults also having significantly lower wealth than the term-born controls.31  Previous research by the EPICure study group has revealed that EP children are at markedly high risk of poor academic attainment32  and that cognitive ability is a predictive factor.33 

Sixteen percent of the EP participants scored in the intellectual disability range (IQ <70) at 19 years. However, when using the term-born controls as a reference group with impairment classified as a score <−2 SD, 44.5% of the EP participants were found to have a cognitive impairment in comparison with 3% of controls. Using the term-born controls as a reference group allowed us to identify EP participants with deficits relative to their same-aged peers and who may not be receiving the support they need because they do not fall in clinically defined range of disability. Studies of school-aged EP children have reported that despite presenting with learning difficulty or intellectual disability, a large proportion are not receiving additional school support, suggesting they have unmet educational needs.34,35 

The number of EP participants scoring in the intellectual disability range (IQ <70) increased from 11 to 19 years (8.4% vs 15.1%, respectively). For a small proportion of EP individuals, intellectual impairment may only become apparent later in adolescence or adulthood when cognitive demands become more complex, highlighting the need for ongoing neuropsychological assessment over the course of childhood and adolescence. The majority of EP individuals who shifted into the impaired range of IQ functioning at 19 years had borderline scores (IQ score: 70–79) at the 11-year assessment (Supplemental Fig 3), which would account for the reported high correlation in mean cognitive functioning over time.8  Visuospatial abilities were also found to be highly correlated from childhood to young adulthood. These findings are in keeping with previous research in which authors reported that cognitive function remains stable from early childhood onward in those born very preterm and/or very low birth weight.9,11 

Sex differences in cognitive function were reported in the EPICure 11-year cohort,7  with EP male participants demonstrating lower IQ scores in comparison with EP female participants. Similar differences were seen at 19 years on the VCI, visual-motor integration, and motor coordination. However, after adjustment for SES, only the difference in motor coordination between EP female and male participants remained significant. This suggests that EP males may catch up to their EP female peers in general cognitive function over the course of adolescence.

A total of 57.8% of the EP participants were lost to follow-up by 19 years. The dropout analysis revealed that those who were lost to follow-up were from lower socioeconomic backgrounds and had lower IQ and/or cognitive scores; additionally, a greater proportion of dropouts scored in the intellectual disability range at 6 and 11 years compared with those who participated. This suggests that the cognitive disparity observed between the term-born controls and EP participants at 19 years may be underestimated. Analysis using multiple imputation at 11 years to account for such selective loss to follow-up suggested that cognitive disability was 5% greater than reported.7  In addition, some of the 95% CIs are wide, as seen in Table 2, reflecting the small sample size. An additional limitation was the breached blinding of the study due to accidental group disclosure by participants. However, because objective standardized instruments were used to assess neuropsychological outcomes, the likelihood of bias is minimized but cannot be excluded.36 

Young adults born EP continue to function below the level of their term-born peers in general cognitive functioning, with the prevalence of intellectual impairment increasing significantly from 11 to 19 years. These adults born EP had impairment in multiple neuropsychological domains, with deficits in general cognitive functioning and visuomotor abilities being the greatest. The current results highlight the need for early and ongoing neuropsychological and educational assessment in EP children to ensure these children receive appropriate support in school and for planned educational pathways.

We thank the EPICure study group and all the EPICure participants and their families for their participation in this study.

Dr O’Reilly assisted in the design of the 19-year follow-up study, collected the data, and drafted and revised the manuscript; Dr Johnson conceptualized and designed the study, obtained funding, and critically reviewed and revised the manuscript for intellectual content; Dr Ni conducted the statistical analyses and critically reviewed and revised the manuscript for intellectual content; Dr Wolke conceptualized and designed the 19-year follow-up study, collected the 6-year follow-up data, and critically reviewed and revised the manuscript; Mr Marlow conceptualized and designed the study, obtained funding, supervised data collection, and critically reviewed and revised the manuscript; and all authors approved the final manuscript as submitted and agree to be accountable for all aspects of the work.

FUNDING: Funded by the Medical Research Council United Kingdom (Medical Research Council reference MR/J01107X/1).

COMPANION PAPER: A companion to this article can be found online at www.pediatrics.org/cgi/doi/10.1542/peds.2019-3359.

AWMA

Automated Working Memory Assessment

Beery VMI

Beery-Buktenica Developmental Test of Visual-Motor Integration, Sixth Edition

CI

confidence interval

EF

executive function

EP

extremely preterm

FSIQ

full-scale IQ

NEPSY

Developmental Neuropsychological Assessment

OR

odds ratio

PRI

perceptual reasoning index

PVF

phonemic verbal fluency

RR

relative risk

SES

socioeconomic status

SVF

semantic verbal fluency

VCI

verbal comprehension index

WASI-II

Wechsler Abbreviated Scale of Intelligence, Second Edition

1
Marret
S
,
Marchand-Martin
L
,
Picaud
JC
, et al;
EPIPAGE Study Group
.
Brain injury in very preterm children and neurosensory and cognitive disabilities during childhood: the EPIPAGE cohort study
.
PLoS One
.
2013
;
8
(
5
):
e62683
2
Serenius
F
,
Källén
K
,
Blennow
M
, et al;
EXPRESS Group
.
Neurodevelopmental outcome in extremely preterm infants at 2.5 years after active perinatal care in Sweden
.
JAMA
.
2013
;
309
(
17
):
1810
1820
3
Larroque
B
,
Ancel
PY
,
Marret
S
, et al;
EPIPAGE Study group
.
Neurodevelopmental disabilities and special care of 5-year-old children born before 33 weeks of gestation (the EPIPAGE study): a longitudinal cohort study
.
Lancet
.
2008
;
371
(
9615
):
813
820
4
Anderson
P
,
Doyle
LW
;
Victorian Infant Collaborative Study Group
.
Neurobehavioral outcomes of school-age children born extremely low birth weight or very preterm in the 1990s
.
JAMA
.
2003
;
289
(
24
):
3264
3272
5
Wood
NS
,
Marlow
N
,
Costeloe
K
,
Gibson
AT
,
Wilkinson
AR
.
Neurologic and developmental disability after extremely preterm birth. EPICure Study Group
.
N Engl J Med
.
2000
;
343
(
6
):
378
384
6
Marlow
N
,
Wolke
D
,
Bracewell
MA
,
Samara
M
;
EPICure Study Group
.
Neurologic and developmental disability at six years of age after extremely preterm birth
.
N Engl J Med
.
2005
;
352
(
1
):
9
19
7
Johnson
S
,
Fawke
J
,
Hennessy
E
, et al
.
Neurodevelopmental disability through 11 years of age in children born before 26 weeks of gestation
.
Pediatrics
.
2009
;
124
(
2
).
8
Linsell
L
,
Johnson
S
,
Wolke
D
, et al
.
Cognitive trajectories from infancy to early adulthood following birth before 26 weeks of gestation: a prospective, population-based cohort study
.
Arch Dis Child
.
2018
;
103
(
4
):
363
370
9
Breeman
LD
,
Jaekel
J
,
Baumann
N
,
Bartmann
P
,
Wolke
D
.
Preterm cognitive function into adulthood
.
Pediatrics
.
2015
;
136
(
3
):
415
423
10
Mangin
KS
,
Horwood
LJ
,
Woodward
LJ
.
Cognitive development trajectories of very preterm and typically developing children
.
Child Dev
.
2017
;
88
(
1
):
282
298
11
Pyhälä
R
,
Lahti
J
,
Heinonen
K
, et al
.
Neurocognitive abilities in young adults with very low birth weight
.
Neurology
.
2011
;
77
(
23
):
2052
2060
12
Hack
M
,
Flannery
DJ
,
Schluchter
M
,
Cartar
L
,
Borawski
E
,
Klein
N
.
Outcomes in young adulthood for very-low-birth-weight infants
.
N Engl J Med
.
2002
;
346
(
3
):
149
157
13
Eryigit Madzwamuse
S
,
Baumann
N
,
Jaekel
J
,
Bartmann
P
,
Wolke
D
.
Neuro-cognitive performance of very preterm or very low birth weight adults at 26 years
.
J Child Psychol Psychiatry
.
2015
;
56
(
8
):
857
864
14
Luu
TM
,
Ment
L
,
Allan
W
,
Schneider
K
,
Vohr
BR
.
Executive and memory function in adolescents born very preterm
.
Pediatrics
.
2011
;
127
(
3
).
15
Anderson
PJ
,
Doyle
LW
;
Victorian Infant Collaborative Study Group
.
Executive functioning in school-aged children who were born very preterm or with extremely low birth weight in the 1990s
.
Pediatrics
.
2004
;
114
(
1
):
50
57
16
Northam
GB
,
Liégeois
F
,
Tournier
JD
, et al
.
Interhemispheric temporal lobe connectivity predicts language impairment in adolescents born preterm
.
Brain
.
2012
;
135
(
pt 12
):
3781
3798
17
Barre
N
,
Morgan
A
,
Doyle
LW
,
Anderson
PJ
.
Language abilities in children who were very preterm and/or very low birth weight: a meta-analysis
.
J Pediatr
.
2011
;
158
(
5
):
766
774.e1
18
Woodward
LJ
,
Moor
S
,
Hood
KM
, et al
.
Very preterm children show impairments across multiple neurodevelopmental domains by age 4 years
.
Arch Dis Child Fetal Neonatal Ed
.
2009
;
94
(
5
):
F339
F344
19
Putnick
DL
,
Bornstein
MH
,
Eryigit-Madzwamuse
S
,
Wolke
D
.
Long-term stability of language performance in very preterm, moderate-late preterm, and term children
.
J Pediatr
.
2017
;
181
:
74
79.e3
20
Ford
RM
,
Griffiths
S
,
Neulinger
K
,
Andrews
G
,
Shum
DH
,
Gray
PH
.
Impaired prospective memory but intact episodic memory in intellectually average 7- to 9-year-olds born very preterm and/or very low birth weight
.
Child Neuropsychol
.
2017
;
23
:
954
979
21
Sripada
K
,
Løhaugen
GC
,
Eikenes
L
, et al
.
Visual-motor deficits relate to altered gray and white matter in young adults born preterm with very low birth weight
.
Neuroimage
.
2015
;
109
:
493
504
22
Wechsler
D
.
Wechsler Abbreviated Scale of Intelligence
, 2nd ed.
Bloomington, MN
:
Pearson
;
2011
23
Wechsler
D
.
Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV) Administration and Scoring Manual
.
San Antonio, TX
:
Psychological Corporation
;
2008
24
Alloway
TP
.
Automated Working Memory Assessment
.
London, United Kingdom
:
Pearson Assessment
;
2007
25
Whiteside
DM
,
Kealey
T
,
Semla
M
, et al
.
Verbal fluency: language or executive function measure?
Appl Neuropsychol Adult
.
2016
;
23
(
1
):
29
34
26
Shao
Z
,
Janse
E
,
Visser
K
,
Meyer
AS
.
What do verbal fluency tasks measure? Predictors of verbal fluency performance in older adults
.
Front Psychol
.
2014
;
5
:
772
27
Benton
AL
,
Hamsher
K
,
Sivan
AB
.
Multilingual Aphasia Examination
.
Iowa City, IA
:
AJA Associates
;
1994
28
Beery
KE
,
Beery
NA
.
The Beery-Buktenica Developmental Test of Visual-Motor Integration (Beery VMI) With Supplemental Developmental Tests of Visual Perception and Motor Coordination and Stepping Stones Age Norms: Administration, Scoring and Teaching Manual
, 6th ed.
Minneapolis, MN
:
NCS Pearson
;
2010
29
Korkman
M
,
Kirk
U
,
Kemp
S
.
Manual for the NEPSY: A Developmental Neuropsychological Assessment
.
San Antonio, TX
:
Psychological Testing Corporation
;
1998
30
Kaufman
AS
,
Kaufman
NL
.
Kaufman Assessment Battery for Children
.
Circle Pines, MN
:
American Guidance Service, Inc
;
1983
31
Jaekel
J
,
Baumann
N
,
Bartmann
P
,
Wolke
D
.
General cognitive but not mathematic abilities predict very preterm and healthy term born adults’ wealth
.
PLoS One
.
2019
;
14
(
3
):
e0212789
32
Johnson
S
,
Hennessy
E
,
Smith
R
,
Trikic
R
,
Wolke
D
,
Marlow
N
.
Academic attainment and special educational needs in extremely preterm children at 11 years of age: the EPICure study
.
Arch Dis Child Fetal Neonatal Ed
.
2009
;
94
(
4
):
F283
F289
33
Johnson
S
,
Wolke
D
,
Hennessy
E
,
Marlow
N
.
Educational outcomes in extremely preterm children: neuropsychological correlates and predictors of attainment
.
Dev Neuropsychol
.
2011
;
36
(
1
):
74
95
34
Johnson
S
,
Strauss
V
,
Gilmore
C
,
Jaekel
J
,
Marlow
N
,
Wolke
D
.
Learning disabilities among extremely preterm children without neurosensory impairment: comorbidity, neuropsychological profiles and scholastic outcomes
.
Early Hum Dev
.
2016
;
103
:
69
75
35
Taylor
HG
,
Klein
N
,
Anselmo
MG
,
Minich
N
,
Espy
KA
,
Hack
M
.
Learning problems in kindergarten students with extremely preterm birth
.
Arch Pediatr Adolesc Med
.
2011
;
165
(
9
):
819
825
36
Holman
L
,
Head
ML
,
Lanfear
R
,
Jennions
MD
.
Evidence of experimental bias in the life sciences: why we need blind data recording
.
PLoS Biol
.
2015
;
13
(
7
):
e1002190

Competing Interests

POTENTIAL CONFLICT OF INTEREST: The authors have indicated they have no potential conflicts of interest to disclose.

FINANCIAL DISCLOSURE: The authors have indicated they have no financial relationships relevant to this article to disclose.

Supplementary data