Pediatric psychoSocial Risk Index: Factor Analysis and Item Reduction Study Free

An instrument validation study was conducted in the pediatric medicine inpatient unit of a tertiary care hospital in a major city in North America. The hospital has >300 inpatient beds and 15 000 annual admissions.

There were 100 primary caregivers (a term that includes parents and/or legal guardian of the child) of hospitalized children aged 0 to 18 years that completed an 86-item PSRI between May and July 2014, on their first or second day of hospitalization. PSRI was then readministered on day 3 or 4 of hospitalization. PSRI was readministered at least 2 days after the initial instrument completion. All primary caregivers whose child was admitted to the pediatric inpatient unit were eligible to participate in this study. For participants who prefer to have medical discussions in a language other than English, the instrument was administered verbally by a research assistant with support for language interpretation that was recorded on a paper version. For families who communicate with English, the instrument was completed in writing. Retest was completed on any family that was admitted for at least 2 days to the inpatient unit.

To reduce redundancy, items were combined from single-order items into multiple part items when possible. For example, a question asked in the first part, “Will the parent have to stop working because of their child’s condition? (yes/no),” and in the second part, “For how long will the parent have to stop working? (Less than a month/1–3 months/4–26 months/26–52 months/permanently) were combined into a single question with 6 ordered levels, from 0 weeks of no work (1) to permanently no work (6). Questions were not removed by this process but were rather rearranged. Second, the 86th item on the questionnaire will not be included in analysis because it is an open-ended question, namely “what are the main concerns for your child?” This item will be added to the end of the updated questionnaire for families to offer any further unanswered information.

Validity evidence testing was accomplished using several statistical tests including percentile distribution to identify frequency of endorsement, item-total correlation to assess each question’s discriminatory power, and Cronbach α to assess for internal consistency. Items that most respondents answer in the same way or that have an item-total correlation <0.20 will be removed to attain a Cronbach α of at least 0.7, indicating that all items contribute to the same construct. For these analyses, 100 respondents is usually a sufficient sample size.¹⁴ 

Principal component analysis (PCA) was performed to reduce redundancy in the PSRI by removing items that met the mathematical criteria of redundancy in several steps. First, the Spearman correlation coefficient was measured to account for correlations between nonnormally distributed PSRI item responses. The Spearman correlation statistical test was applied to determine which items have item–item correlations <|0.30| (this signified that the items were not closely related to the any of the other items within the PSRI) were removed from the instrument. Third, we examined the Kaiser Meyer-Olkin (KMO) test of sampling adequacy to determine the appropriateness of the sample for factor analysis.¹⁴  KMO assesses if items were closely related to responses from other items that would signify that the items are redundant to include. Items were removed with individual KMOs with an a priori threshold <0.50, which indicated that they represented a lower proportion of variance than the other items. Fourth, the items were included in the orthogonal PCA. The orthogonal PCA informs item reduction by identifying the items that are contributing to the measurement of the construct—and consequently the items that are not contributing to the measurement of the construct of interest and can therefore be removed from the instrument. Therefore an orthogonal PCA means that the extracted components must explain unique variance, that is, they must not be correlated with each other and representing unique constructs.^15,16  Then, item-total correlations were reviewed, and individual items with a correlation <0.45 were removed. Item-total correlations were examined with Pearson correlation. For the remaining items, the data were examined to ensure that there was less than 10% missingness, indicating that the participants did not refuse to answer the questions slated for inclusion. PCA was then repeated on the final set of items to confirm that they met these criteria when retested together.

Cronbach α was used to evaluate the internal consistency of PSRI. Items that most respondents answered in the same way (low variance in item responses across participants) or had an item-total correlation <0.20 (items were not associated with all other questions in the instrument) were removed. We set a minimum threshold for Cronbach α at 0.7, indicating that all items contributed to the same construct.

Test-retest reliability assessed the consistency of responding to the PSRI over time. This was accomplished by comparing PSRI responses among 50 participants who recompleted the PSRI 3 to 4 days after completing it at baseline. Test-retest reliability was examined by evaluating the correlation between the mean between-person across-item reliability and mean within-person across-item reliability.

Item-total correlations were examined on the final selection of items with Pearson correlations. This helps confirm that the remaining items were all correlated with the total of the other items and therefore measuring the same underlying latent psychosocial construct. This also helps confirm that the individual items were correlated with the total of all the other items and therefore representing adequately assessing the same underlying construct, albeit within the various psychosocial themes.

All analyses were completed in R software for statistical computing version 3.6.0¹⁶  and R Studio version 1.2.1335¹⁷  using the psych¹⁸  and multilevel^19–21  packages. All participants provided written informed consent. This study was approved by the local institutional Ethics Review Board.

One hundred families completed the 86-item PSRI and 50 completed retesting. The average age of the children was 5.6 years and 5.8 years in the retest sample (Table 1). Two items were reorganized together to reduce question burden by creating 1 multiple choice questions instead of 2 single order questions, leaving 85 items remaining (Fig 1).

Before running the PCA, 3 items were removed because they had >10% missing data; 82 items remained. A correlation matrix indicated that 16 items had item-total correlations <|0.30|; therefore, these 16 items were removed, leaving a total of 66 items remaining. For the analysis of individual KMO testing of sampling adequacy, 14 items had individual KMOs <0.50, which were removed, leaving 52 items remaining. After running the first orthogonal PCA, 24% of the variance was explained by 26 items; these items were each loaded at 0.50 or higher on the first component. The scree plot indicated the inflection point after 1 component (Fig 2). The remaining 26 items explained <10% of the variance. The orthogonal PCA indicated that the first 26 items explained enough of the variance in the construct that they could meaningfully contribute to measuring that construct. Thus, the remaining 26 items, that explained <10% of the variance in the construct of interest would simply add to participant burden without meaningfully adding to the measure of the construct under consideration. The first 26 items loaded on the first component were retained for subsequent analysis. Analysis of the individual item-total correlations of the 26 retained items revealed that 6 had item-total correlations <0.45, which were removed, leaving 20 items (Fig 1).

We examined the individual item-total correlations for the 20 remaining items in the retest data. Two items had individual item-total correlations <0.50 and were removed. Of the remaining 18 items, 2 had missingness ≥10%, resulting in 16 retained items representing 9 social risk themes (number of items/theme): food insecurity (2), medical complexity (2), home environment (2), behavioral issues (1), financial insecurity (1), parenting confidence (1), parental mental health (2), social support (2), and unmet medical needs (3).

For the remaining 16 items, the test-retest reliability was 0.98, mean within-person across-item dependability was 0.95, and the mean between-person across-item reliability was 0.96, indicating excellent test-retest reliability. Cronbach α was 0.89 using the test data and 0.9 using the retest data, indicating a high degree of internal consistency. Please refer to Supplemental Table 3 which describes all items removed from PSRI along with the reason for removal based on the analysis.

After completion of PSRI by 100 parents of hospital inpatients at a major pediatric hospital, the PSRI was refined from 86 items into a brief, multidimensional 16-item instrument with excellent internal consistency and test-retest reliability (Supplemental Table 2). Remaining items met all a priori specified criteria and, while still addressing 9 social risk themes, represented the same overall construct of social risk based on PCA. Cronbach α was run as a test of internal consistency on the final 16 items. The value of 0.89 is for the 16 test items and 0.9 in the retest items. Although α is high, which generally indicates internal consistency of the items, it does not exceed the 0.9 benchmark that may indicate redundancy. Cronbach α also increased with more items in a scale. As an exercise in item reduction and scale construction, we expected a higher α in the initial sample.²²  PSRI is the first pediatric social-risk screening instrument that is brief and multidimensional with evidence of validity specifically in the pediatric inpatient setting.^11,23  Multidimensional social risk screening is optimal because marginalized families tend to experience multiple interacting social risk factors at the same time.^8,24  For instance, a 2003 study of 102 353 US children found that more than half had 2 or more social risk factors, with one quarter having 4 or more.²⁴  Social risk factors such as education and maternal mental health were independently associated with negative child health outcomes but also were associated with each other, as is found in other studies^24,25  Additionally, having a greater number of social risk factors was found to have a cumulative negative effect on child health. The percentage of children with poorer oral health, obesity, and social/emotional challenges increased with the number of social risk factors.²⁴  These findings demonstrate the importance of addressing multiple social needs at the same time. For instance, efforts to improve parental mental health may be less effective if care providers fail to attend to lack of social support. Multidimensional risk screening instruments such as the PSRI may help identify overlapping social risk and treat patients more holistically. Although the PSRI is not intended to replace existing comprehensive assessments such as those performed by social workers, it may be particularly helpful in settings where such assessments are limited or not available.

Multidimensionality also makes the PSRI a time-efficient alternative to completing multiple social risk instruments each on different risk dimensions.^5,10  With families accessing health care already experiencing high distress, especially those in inpatient settings, it is important that screening instruments such as the PSRI pose the least burden possible while still capturing the nuances of a family’s social needs.^9,10  Shorter instruments also reduce health care provider workload, leaving more time for other priorities such as the medical care of patients. Time saving would be of particular importance in acute care settings where more resource intensive social risk assessments such as those conducted by a social worker may be less available or practical.⁹  Based on our findings for PSRI, given that it is brief yet multidimensional, it can be useful for families to complete the tool during admission to support the medical team to gather details regarding social history and identify needs so they can support families by connecting them with resources.

Other screening instruments have shown promise in addressing social risk factors in the pediatric outpatient setting.^8,10  For example the Well Child Care, Evaluation, Community Resources, Advocacy, Referral, Education social risk instrument was found to increase maternal employment, reduce the number of families living in homeless shelters, improve childcare access, and increase referral to and enrollment in community-based social resource programs.¹⁰  The PSRI has the alternative benefit of demonstrating evidence of validity in the pediatric inpatient setting, a particularly appropriate location for social risk screening.

Limitations of this study include test-retest data collection that included a sample of hospitalized patients only, which may limit generalizability of the tool in other health care settings. Detailed demographic data were kept anonymous, a decision of the research team at the time to not further burden families with additional questionnaires because they completed the 86-item PSRI. Future studies will include this component as it relates to results of PSRI. In future refinements of the instrument, we will be collecting test-retest data for both outpatient and inpatient pediatric populations. Future studies will aim to validate PSRI in the outpatient setting as well.

After further validation across multiple settings, we hope that PSRI will be helpful for health care teams in both inpatient and outpatient settings to identify the social needs of children and their families that will aide connecting families with needed social resources, and ultimately improve long-term health and reduce health service utilization for vulnerable children.

The 16-item PSRI demonstrates adequate validity evidence to screen social risk factors for pediatric patients admitted to the hospital.