The Reading House: A Children’s Book for Emergent Literacy Screening During Well-Child Visits

TRH is a full-color, board-format book for young children, measures 6 × 6 in, and is 14 pages long. It has a simple, rhyming narrative (Flesch-Kincaid estimated kindergarten level) and illustrated content playfully modeling early reading skills as children of diverse ethnicity and sexes go about their day (Fig 1). Pediatrician-advocated behaviors such as wearing a bicycle helmet and healthy eating are also reinforced. Before publication, draft versions were pilot tested and psychometrically revised.³⁴ 

The TRH assessment follows, yet is distinct from, the book’s narrative and is intended to be administered by a clinical provider. Content was developed on the basis of a conceptual model of literacy skills expected to emerge between 3 and 4 years of age^14,36: print concepts and knowledge (15%),³⁷ alphabet and letter sound knowledge (29%),^38,–40 phonological awareness (rhyming, alliteration, syllable blending; 21%),^41,42 expressive language and vocabulary (21%),³⁶ and emergent writing (14%).⁴³ The scripted scoring form has 9 items corresponding to the covers and 7 interior page spreads. Wording was refined in consultation with experts in measure development and pilot tested with families and is estimated at a Flesch-Kincaid kindergarten reading level. Administration and scoring are summarized in Table 1.⁴⁴ 

The Get Ready to Read! (GRTR) measure was our criterion-referenced standard. GRTR consists of 25 items in tabletop, monochrome format, with an estimated administration time of 10 to 15 minutes. GRTR is validated for ages 3 to 6 years, intended for use by early childhood educators.³⁴ Component skills assessed are print knowledge (25%), alphabet and letter sound knowledge (25%), phonological awareness (rhyming, alliteration, syllable blending, elision; 35%), and emergent writing (15%). GRTR total scores are normalized for age and performance level,⁴⁵ and the GRTR has been shown to predict reading outcomes.^46,47 

Healthy children between 3 and 4 years old were recruited at 7 pediatric primary care clinics in a large Midwestern city. Two clinics were affiliated with an academic children’s hospital serving almost exclusively families of lower socioeconomic status (SES) (>90% were eligible for Medicaid; 75% were African American). Five clinics were private practices serving demographically and geographically diverse communities. Eligibility criteria for all children were as follows: (1) gestation of at least 36 weeks, (2) age at screening of 36 to 52 months, (3) no documented history of global developmental delay or neurobehavioral disorder, and (4) native English–speaking custodial parent. Verbal informed consent was obtained from a custodial parent to expedite screening and minimize interference with clinic flow, and families were provided a different children’s book for participation. Our study was approved by our institutional review board.

Clinical research coordinators (CRCs) practiced administration of TRH via role play sessions and were instructed to adhere to screening scripts verbatim. After informed consent was given, CRCs administered TRH and GRTR in examination rooms before or after seeing the pediatric provider but before vaccine administration to be mindful of patient flow. No identifying or sensitive demographic information was collected per institutional review board restrictions for verbal consent. Order of administration was determined via a random assignment list to control for child fatigue. The time of TRH administration was noted, and after screening, the caregivers were asked 3 questions regarding the time required for TRH screening, whether TRH screening helped them to learn more about their child’s early reading skills, and if it influenced plans to read with their child at home. The child was asked if sharing TRH was fun. Clinical providers (physicians, nurses, and medical assistants) were also surveyed regarding whether screening interfered with clinic flow, their impression of feasibility and usefulness of adopting TRH for everyday practice, perceived barriers to the adoption of TRH, and whether they believe that literacy screening belongs in primary care. Families were not allowed to take TRH home at this preliminary stage. Response data were entered into a secure Research Electronic Data Capture (REDCap) database.⁴⁸ 

Descriptive statistics were computed for demographic variables for all children. Descriptive statistics were also computed for all TRH variables at item and scale levels. Modern theory Rasch rating scale methods were used for all TRH items,^49,50 which were evaluated for smoothness, modality, difficulty, polarity, and sufficiency of density. Model fit was tested for each item to identify any that were markedly influencing scale-level distributions. Bivariate correlations were computed among pairs of items as well as between each item and TRH total score and GRTR total score by using Spearman-ρ correlation coefficients. Preliminary estimates of TRH reliability and validity were then computed, beginning with Cronbach’s coefficient α (α_Cr) as our measure of reliability, and Spearman-ρ (r_ρ) correlation coefficients for criterion-related validity between TRH and GRTR total scores. Spearman-ρ coefficients were also computed between TRH total score and selected demographic variables: child age and sex and clinic type (hospital-affiliated versus private practice). Spearman-ρ correlations were deemed as most appropriate given the non-normal nature of distributions for both TRH and GRTR scores. Ordinal logistic regression models were used to estimate the relationship between GRTR (below average, average, and above average) and TRH scores, controlling for age, sex, and clinic type, respectively, to identify the best-fitting model. The criterion for statistical significance was the unadjusted α = .05 level. Finally, descriptive statistics were obtained for responses to acceptance, feasibility, and usability questions administered to children, parents, and providers across clinics. Statistics for TRH administration time were also determined. All analyses were conducted by using SAS version 9.4 (SAS Institute, Inc, Cary, NC) and Winsteps version 4.0 software (Rasch Measurement Analysis, Beaverton, OR).

A total of 278 children were screened. Ages ranged from 36 to 52 months (mean: 43.1 ± 5.6 months; 58% were 36–47 months; 42% were 48–52 months), with 125 boys and 153 girls. A total of 109 children (39%) were from hospital-affiliated clinics, and 169 were from private practices (61%).

TRH item–level information is provided in Table 2. Rasch estimates of item difficulty ranged from −3.69 (less difficult; vocabulary) to 2.22 (more difficult; rhyming). Item-fit statistics using empirically derived z values were within the traditional ±2 SDs, which suggests that no outliers likely influenced the distributions.⁵¹ Point-measure correlations ranged from 0.21 (alliteration) to 0.77 (vocabulary), which suggests a small-to-moderate relationship between each of the items and the entire scale. Bivariate TRH interitem correlation coefficients ranged from r_s = −0.02 (blending-alliteration; none) to 0.58 (letter name–letter sounds; large)⁵² but were generally small between 0.1 and 0.3 (Table 3). The mean TRH score was 4.2 (±2.9; range: 0–14), and mean GRTR was 11.1 (±4.4; range: 1–25). A scatter plot is shown in Fig 2.

For reliability, internal consistency was estimated at r_Co-α = 0.68, considered “good” for new measures.⁵³ For criterion-related validity, the correlation between TRH and GRTR scores was r_s = 0.66 (P < .001; high). There were low-to–moderately positive correlations between TRH total scores and female child sex (r_s = 0.18), private practice clinic type (r_s = 0.31), and child age (r_s = 0.34; all P < .001). In the ordinal logistic regression model, TRH total score (odds ratio: 1.7; 95% confidence interval: 1.5–1.9; P < .001) and child age (odds ratio: 0.9; 95% confidence interval: 0.8–0.9; P < .001) were included in the best-fitting model. Child sex and clinic type were not statistically significant.

A total of 72 clinical providers were surveyed (20 hospital affiliated and 52 from private practice), including 18 medical assistants, 11 nurses, and 43 physicians. Most believed that literacy screening belongs in primary care (85% yes, 15% not sure, 0% no). Few reported that TRH interfered with patient flow (93% not at all, 6% somewhat, 1% very much). Most believed that TRH would be feasible to administer (49% yes, 43% not sure, 8% no), clinically useful (67% yes, 31% not sure, 2% no), and useful for families (85% yes, 14% not sure, 1% no). Responses were not significantly different between physician and nonphysician providers, although nominally more physicians rated TRH as clinically useful and nominally more nonphysicians rated TRH as feasible to administer. The most commonly cited potential barrier to adopting TRH in everyday practice was not enough provider time (68%), with 8% noting staffing concerns and 3% noting no reimbursement.

A total of 248 children and 243 parents responded to surveys regarding their impression of TRH screening, summarized in Fig 3. Most parents reported the time required for TRH screening as reasonable (88% just right, 9% a bit much, 3% too much). Most reported that TRH screening helped them learn about their child’s early reading skills (34% very much, 37% somewhat, 29% not at all) and influenced plans to read with their child at home (29% very much, 29% somewhat, 42% no). Responses for the time required were equivalent between clinic types and more favorable for learning about reading skills and influencing plans to read in hospital-affiliated clinics (P < .05; χ² test). The majority of children rated sharing TRH as fun (65% yes, 23% kind of, 10% no, 2% no response), which was equivalent between clinic types. The mean TRH administration time was 5:25 minutes (±0:55; range: 3:34–8:32) and was not significantly different between 3- and 4-year-olds or between clinic types.

Recent insights into early brain development suggest that potential deficits in emergent literacy skills can be identified and addressed proactively well before kindergarten.^25,54,55 Concerns with screeners reliant on parent report^10,32,33 include susceptibility to social desirability bias, use of complex language and concepts, low reliability at <4 years of age, and, importantly, nonengagement of the child. This is a particular concern in families of low-SES backgrounds who may have fewer opportunities to share books with their child and accurately gauge their abilities. Although parent report is widely used and appealing for practical reasons,^56,–58 direct screening is ideal. Our objective was to develop and validate a children’s book designed to directly screen emergent literacy skills during well-child visits as early and enjoyably as possible, providing a potential catalyst for anticipatory guidance that complements existing reading programs.⁹ 

The internal psychometric properties of TRH were strong, particularly for a novel measure attempting to capture a dynamic developmental construct with multiple components. We attribute this performance to an evidence-based conceptual model of emergent literacy used to guide development.^14,15,36 TRH items exhibited a good range of difficulty, with phonological items (rhyming, alliteration) scoring most difficult and vocabulary and letter naming the easiest (Table 2), consistent with expected trajectories of these skills.³⁶ The relative ease of the letter naming item may reflect its open-ended nature, providing for differences in how children learn letters (eg, those in their own name).^40,59,60 The ease of syllable blending relative to the other phonological items may be attributable to the manipulation of larger sound units⁶¹ and/or pictures of blended words that are visual cues for the child. By contrast, the rhyming and alliteration items have multiple visually appealing choices, and alliteration requires the processing of smaller sound units.⁶¹ Despite word selection from an age-normalized list,⁶² difficulty of the vocabulary item was exceedingly low (<−3 SD units). We plan to incorporate more difficult word objects into a revised version.

TRH items revealed small-to-moderate interitem correlations, which suggests that each item contributed uniquely to the total score. All correlations were positive, consistent with variable skill trajectories, which emerge in parallel before their integration,³⁶ with some lagging because of neurobiological constraints.^17,63 Letter naming and letter sound knowledge were highly correlated with each other and with vocabulary, consistent with previous research.^40,64 Interestingly, rhyming and alliteration were weakly correlated, possibly attributable to a high variability of phonological abilities at younger ages³⁶ and likely contributing to relatively weak correlation between these items and total TRH score. Weak correlation between these items and syllable blending may be attributable to relative ease of larger sound unit blending at this age.⁶¹ Name writing was positively correlated with almost all other items, consistent with evidence of broad interrelationship of this skill with other emergent skills.^65,66 

TRH total scores were positively correlated with age, female sex, and private practice clinic type (our predetermined proxy for SES), consistent with the expected influence of these factors.^14,67,68 TRH was strongly correlated with GRTR (concurrent validity), which has been shown to predict reading outcomes and risk for reading difficulties.^34,46 This relationship remained statistically significant in our logistic regression model after controlling for age, sex, and clinic type. This robust relationship is unsurprising because TRH components mirror GRTR; however, in addition to format, there are important content differences. Print concepts and emergent writing are ecologically assessed in TRH by inviting the child to identify the front cover and write his or her name on the back. To assess alphabet knowledge, the child is invited to identify letters that they know rather than a prescribed set. TRH is used to assess alliteration as its most advanced phonological skill because we considered elision to be prohibitively difficult for our target age. TRH includes assessment of expressive vocabulary, the earliest emergent skill,^36,37 with less weighting of phonological skills. Despite incorporating easier items, TRH scores skewed low, with only 1 child achieving a maximum score and 12 receiving a score of 0. We believe that this reflects the relative difficulty of TRH fueled by its ecological format and minimal use of multiple-choice items (2 of 9 items, each involving 2 trials requiring correct responses to receive credit) to reduce the influence of prompts and guessing. Refinement of future versions may involve the revision of items to make them easier for our intended age range or alternate forms for younger and older children.

Although psychometric properties are critical, effective screening should be perceived as useful and not burdensome or invasive. Responses to parent, child, and provider surveys were favorable, which suggests that TRH screening may be an enjoyable and valuable addition to well-child visits. This seems particularly appealing for families of lower SES, who reported learning more about their child’s skills and a greater influence on plans to read at home. An administration time of ∼5 minutes seems to be a reasonable investment if staff and families view time spent as worthwhile, as suggested in our data. This appeal should be even greater if and/or when reimbursement for literacy screening is implemented and providers are able to use TRH as a teaching tool. The 3-year-old well-child visit may be ideal for TRH administration because vaccines are not typically given, and guidance can be provided proximal to preschool entry.⁵ Beyond skill assessment, an appealing aspect of TRH involves modeling of reading and other positive health messages in the narrative. Children’s books for anticipatory guidance have been proven effective for this purpose.^69,–71 Inviting the child to write his or her name on the book exemplifies this dual function of assessing a core skill while providing a sense of ownership and empowerment for families.

This study has limitations. Although construct validity was established, its cross-sectional nature cannot be used to establish predictive validity, which would require a longitudinal design. Because of feasibility constraints, we did not explore test-retest reliability, although this is planned. To expedite screening in busy clinical practices, we collected limited demographic information. However, this served the aims of this study, and clinic type is a reasonable proxy for SES given the population served by our hospital-based clinics. TRH was administered by CRCs, and it is possible that the results would not generalize to clinical providers, although at this preliminary stage, fidelity with administration was paramount and provider impression was surveyed. Indirect assessment of feasibility and utility was likely a major driver of the sizable percentage of “not sure” responses for these items, which we believe would skew favorably in actual practice given the appeal of book distribution and discussion of reading documented in the Reach Out and Read program.^9,72 Similarly, we suspect that more parents may have rated TRH as useful if it was administered by providers^1,72 rather than CRCs trained to not discuss results or give them the book to take home. GRTR was our sole external criterion,^34,47 and exploring relationships between TRH and other standardized measures would be useful. Although it revealed solid internal consistency and reliability, TRH was administered to a broad age range when skills evolve rapidly. In addition to determination of risk strata for older and younger age ranges, development of alternate forms for 3- and 4-year-old children may be useful.

Our study also has important strengths. Our sample was large and well characterized, drawn from diverse practice sites. Screening was conducted during normal pediatric practice by using a book that is scalable and low cost within existing programs such as Reach Out and Read.⁷² Analyses involved rigorous psychometric methods^49,51 and an external criterion shown to predict reading outcomes. Feasibility was explored via large samples of providers, parents, and children. A future study involving TRH administration by clinical providers will allow the exploration of important questions, including optimal age of administration, integration into clinic workflows, and use of TRH for anticipatory guidance. We are developing training materials to encourage fidelity.⁴⁴ We have also generated estimates of risk thresholds for TRH corresponding to performance categories in the GRTR (below average, average, above average) for children aged 36 to 47 months and 48 to 52 months. Although we will provide these to those interested in using TRH with caveats of their preliminary nature, longitudinal studies are needed to determine predictive validity. Altogether, TRH reveals early promise as a valid and enjoyable means to directly screen emergent literacy skills in 3- and 4-year-old children with potential as a catalyst for reading guidance consistent with AAP screening and literacy recommendations.^1,5 

In this preliminary validation study involving emergent literacy screening in 3- and 4-year-old children during well-child visits, TRH revealed promising psychometric properties, including internal consistency and validity referenced to an established standard. Provider, parent, and child impressions of screening in terms of time, enjoyment, and utility were favorable, and administration time was feasible for primary care practice. The children’s book format of TRH also suggests potential as a catalyst for provider-parent reading guidance and is scalable within existing programs. Longitudinal studies are needed to refine risk strata for different age ranges, improve items, assess predictive validity, and test performance when administered by clinical providers.

We thank Arielle Wilson for her diligence in collecting data and the providers at the 7 clinical sites, Pediatric Primary Care Clinic, Hopple Street Clinic, Springdale-Mason Pediatrics, Pediatricians of Hyde Park, Northeast Pediatrics, Group Health Associates: Western Hills, and Pediatric Associates of Mount Carmel, who graciously hosted this research. We also thank the Cincinnati Children’s Hospital Medical Center Research Foundation for their support of early-career investigators and this work. For more information about TRH or to order it for research or clinical use, visit http://www.bluemanateepress.com/nonprofit/ or e-mail press@bluemanateebooks.com or john1.hutton@cchmc.org.

AAP

American Academy of Pediatrics

CRC

clinical research coordinator

GRTR

Get Ready to Read!

SES

socioeconomic status

TRH

The Reading House