Driving Skills at Licensure and Time to First Crash

This study used a deidentified analytic dataset of linked VDA, licensing, and crash record data (see Data Sources section below for more detail) prepared by a data operations team at the Children’s Hospital of Philadelphia (CHOP), in accordance with data privacy agreements between CHOP and the State of Ohio and Diagnostic Driving Inc. Thus, this study was considered exempt from Institutional Review Board oversight by CHOP. Our analytical sample consists of first-time license applicants in Ohio who were <25 years of age and took the VDA immediately before passing the state license examination, for which they were awarded their license, between July 2017 and December 2019. Ohio license and crash database records (up to March 15, 2020, and before the coronavirus pandemic lockdown) were exact-matched to VDA records by using a unique driver ID, which was then deidentified through an Honest Broker at CHOP. Through the deidentification process, all dates were converted to age in days from birthdate, including age at permit issuance, license examination attempt, license issuance, and any police-reported crash. From these, we derived the time since licensure and the time to first crash. The crash follow-up time for individuals was also rounded to 15-day intervals to protect the privacy of the data. The driver’s license addresses were geocoded into Federal Information Processing codes by using Esri geocoder and then matched to American Community Survey data²⁸  to obtain Census tract-level sociodemographic variables.

Our analytical sample consisted of 16 914 newly licensed drivers who were <25 years of age, completed the VDA immediately before passing the license examination in Ohio, and were awarded their first license within a month of their license examination. See Fig 1 for sample derivation. These individuals were located at 5 licensing sites in Ohio in which the VDA was implemented (3 sites in the urban/suburban Columbus area, 1 other suburban site, and 1 rural site).

The Ohio licensing database contains detailed information on each driver’s interactions with the Ohio Bureau of Motor Vehicles (BMV), including driver demographics (date of birth, gender, address) and licensing history (permit dates, license examination dates, license examination outcomes, license issue dates, and license status). Of note, in this sample, the “gender” variable only includes “male” and “female” and, thus, hereafter is referred to as “sex.” Ohio crash records are collected from law enforcement agencies via a statewide uniform crash report, in compliance with current Model Minimum Uniform Crash Criteria standards.²⁹  In Ohio, a crash must be reported if any personal injury or fatality occurs, and/or there is at least $1000 worth of property damage.

The VDA data were acquired from Diagnostic Driving Inc., which collected it via the VDA software (Ready-Assess^TM) that was implemented in Ohio BMVs. The Ready-Assess VDA is a ∼15-minute-long self-directed workflow including a driving route that incorporates common serious crash risk scenarios,^30,31  such as rear-end events, intersections, curved roads, merges, and hazard zones. These routes also have varied settings (urban and suburban), physical road features, and other road potential hazards (eg, crosswalks, merges, construction zones, vehicles, and pedestrians). The Ohio VDA dataset had 69 variables that capture operational and tactical driving skills across the drive, including aberrant or hazardous behaviors like simulated crashes and traffic light violations, as well as performance in several known domains of driving (see Table 1). Our team previously used a subset of these VDA metrics (see Table 1 and Walshe et al 2022)³²  to derive the VDA Skill Clusters and 4 major Driving Classes: (1) No Issues (ie, careful and skilled drivers showing no issues on the VDA), (2) Minor Issues (ie, an average new driver with minor vehicle control skill deficits), (3) Major Issues (ie, drivers with more control issues and who take more risks), and (4) Major Issues with Dangerous Behavior (ie, drivers with even more control issues with “jackrabbit,” sudden, jerky movements, and more reckless and risk-taking behavior). See Table 2 for Skill Cluster and Driving Class labels and their weight on domains of VDA skill performance that differentiate them (for more specific details on Skill Cluster weightings on each variable (see Supplemental Appendix A plots in Walsche et al³² ). Of note, the original paper labeled the worst performing Driving Class as “Major Issues with Aggression,” which has since been relabeled as “Major Issues with Dangerous Behavior” to be more descriptive and avoid implying underlying personality traits.

The outcome is time to first crash from the first day of licensure, determined as the total number of days after a license was issued to the first involvement in a crash. Although measured to the day, total follow-up time was rounded to 15-day intervals to protect driver privacy.

The key predictor in this crash risk model is the 20 Skill Clusters and the 4 summary Driving Classes (see Table 2).

The potentially confounding adjustment variables included in our model include the 5 licensing sites noted above, age at the time of licensure (16 to <17, 17 to <18, 18 to <19, and 19 to <25 years of age), sex (male/female), as well as sociodemographic summary measures of each county (as described previously^32,33 ): income/education, transportation, and urbanicity.

A Cox proportional hazard model was used to estimate the instantaneous risk of a police-reported crash during the follow-up period as a function of Skill Cluster and Driving Class membership. Effect coding was also used to estimate the change in risk associated with cluster assignment so that each category is compared with the overall average crash risk. All analyses are adjusted for licensing center location at which the test was taken; we treat these as “unadjusted” (that is, unadjusted for substantive confounders of interest). Analyses adjusting for individual-level age, sex and census-tract-level sociodemographic characteristics were also conducted. To account for potential correlations among drivers at each center, a generalized estimating equation approach was used.³⁴  All analyses were completed by using SAS V9.4. The proportional hazard assumption was tested by using a bootstrap test of cumulative martingale residuals.³⁵  For models that failed to follow the proportional hazard assumption, interactions with time were considered to assess how risk might have changed over follow-up within certain classes. Interaction with the age of licensure among the Driving Classes was also considered.

The majority (78%) of newly licensed drivers in this sample were tested in the Columbus area (Urban Sites 1, 2; Suburban Site 3), 14% were at a rural location (Site 5), and 8.4% were at another suburban location (Site 4; Table 3).

Table 4 presents sample characteristics. Overall, almost one-half of the newly licensed drivers were aged 16 years (46%), with fewer 19- to 25-year-old new drivers (22%). Relative to state-wide distributions of license applicants,¹⁸  this sample was from more urban tracts and was slightly more educated (and more so among drivers with no crash).

Table 2 reveals the distribution of the sample across the 4 Driving Classes and their subsets of Skill Clusters.

In this newly licensed driver sample, 2 301 (13.6%) were observed to have at least 1 crash. Figure 2 reveals the Kaplan-Meier curve. In addition, 1.5% of these new drivers had 2 or more crashes (10.7% of those who had at least 1 crash) during follow-up. The maximum total number of crashes for an individual was 4. Table 5 reveals the mean number of crashes per year by VDA Driving Classes and the various Skill Clusters they represent. However, only time to first crash is used in our estimation of how the risk of crash varies by VDA classification.

Table 5 provides the hazard ratio estimates for each of the Skill Clusters and Driving Classes, relative to the overall average risk of crash. After adjusting only for licensing center location, the No Issues Driving Class had a significantly lower risk of crash, at 10% less than average (95% confidence interval [CI] 13% to 6%), whereas the Major Issues with Dangerous Behavior Class had a significantly higher risk of crash, 11% greater than average (95% CI 1% to 22%). In addition, all Skill Clusters in the No Issues Class had below-average risks of a crash, with the Cautious skill cluster drivers reaching statistical significance, with a crash risk 17% less than the average (95% CI 28% to 4%). On the other hand, all Skill Clusters in the Major Issues with Dangerous Behavior Class had above-average risk of crash, with 2 reaching statistical significance: the Risky, No Control, Jackrabbit drivers had a 20% increase in risk (95% CI 2% to 42%), and Extremely Aggressive, Reckless drivers had an 8% increase in risk (95% CI 1% to 16%). The Minor Issues and Major Issues Classes did not differ from the average crash risk. The Skill Clusters in the Minor Issues and Major Issues Classes were both above and below-average risk, although most were close to 1. The exception to this is the Speeder, Tailgater, Rulebreaker Skill Cluster in the Minor Issues Class, in which there was a 7% increase in risk relative to the overall risk (95% CI 2% to 14%). Adjusting for covariates (age, sex, and tract-level SES indicators) had little impact on crash hazard ratios.

There was evidence of an interaction between age and the Minor Issues Class (P = .005); in particular, drivers licensed at age 18 had an elevated crash risk 16% higher than average (95% CI 6% to 27%), but no other age group differed from average crash risk. Specifically, the Skilled Average Cluster within the Minor Issues Class revealed a strong interaction with the 18-year-old age group (who are not required to complete driver education and training before licensure): the risk of crash was 38% higher than average in this age group (95% CI 13% to 68%).

The proportional hazards assumption, that the relative risk difference is constant over time, generally held for the Driving Classes, but this was not the case for 3 of the Skill Clusters: Below Average Control (P = .025), Quick with Controlled Braking (P = .045) and Risky, Poor Control, Jackrabbit (P = .005; See Fig 3). The Below Average Control Cluster had significantly elevated risk in 12 to 18 months after licensing (47% increase, 95% CI 7% to 101%). The Quick with Controlled Braking Cluster started out with near-average risk but, after 18 months of follow-up, had above-average risk (74% increase, 95% CI 35% to 126%). A similar pattern was seen for the Risky, Poor Control, Jackrabbit Cluster (112% increase at 12 to 18 months, 95% CI 64% to 174%; 53% increase after 18 months, 95% CI 11% to 162%).

This population-level study confirmed that new driver skills at licensure (measured via a virtual driving assessment) were a predictor of time to crash early in licensure.

Specifically, compared with average risk, the best-performing No Issues Driving Class had a lower risk of crashing post-licensure, and the worst-performing Major Issues with Dangerous Behavior Driving Class was associated with higher crash risk. The more moderate Minor Issues Driving Class and the Major Issues Class did not have statistically significant differences from the overall risk of crash. However, within the Minor Issues Driving Class, the Speeder, Tailgater, Rule Breaker cluster had a significantly elevated risk of crash. Although this cluster had a small proportion of drivers (0.8%), it is a worrisome behavior pattern that could be targeted for interventions if identified at the time of licensure via the VDA. Given these findings, this Skill Cluster would likely be better placed in the Major Issues with Dangerous Behavior Driving Class and will be reclassified in subsequent studies and use of this tool.

Controlling for the covariates of age, sex, and tract-level sociodemographic indicators did not add any predictive power. However, there was an interaction between age and the Minor Issues Driving Class, whereby those aged 18 were at an elevated risk of a crash (especially prominent in the Skilled Average cluster). Of note, in Ohio, license applicants aged 18 years are only required to pass the licensing examination and are not subject to the rigorous licensing policies of those <18 years of age (a 6-month learner permit, mandated driver education and behind-the-wheel training). Future studies should explore the interaction between training, exposure and skills, and time-varying risk post-licensure.

Although the VDA does not capture an individual’s entire risk profile, it could be used in combination with other measures of crash-risk skills, attitudes, and behaviors to advance the development of better diagnostic prediction models of crash risk. With this information, programs can be developed that provide personalized and targeted interventions that mitigate skill deficits before licensure and in the early period of licensure when crash risk is highest for new drivers. For example, some online education interventions have been developed for improving hazard awareness and prevention that have been validated by observed reduction in crashes^25,36  that could be provided at the time of licensure.

One important limitation that we could not control for is miles driven, future studies should account for differences in driving exposure. In addition, the total follow-up time was rounded to 15-day intervals to protect data privacy, meaning the time to crash could have errors of up to 14 days. However, this error would be expected to be random and, thus, not likely to impact these results. Although the demographic profile of this analytical sample from 5 Ohio BMVs is rather similar to that of the state-wide license applicant population of Ohio in the same years (per our previous work³³ ), this sample was skewed slightly older with slightly fewer 16-year-old newly licensed drivers (46% vs 54% state-wide), and more 19- to 25-year-old newly licensed drivers (23% vs 18% state-wide). Finally, the VDA database available to us from the Ohio BMV implementation did not contain visual-scanning metrics that could better indicate situational awareness, a key component for driver safety.

This population-level study reveals that new driver skills measured by the VDA at the time of licensure contribute to crash risk post-licensure. These findings take us a step closer to identifying the drivers at risk of crashing early in licensure and some of the skills that are protective of future crashes. These results can inform targeted interventions that could be delivered at the time of licensure, providing actionable feedback or training to newly licensed drivers on areas needing continued improvement post-licensure.

The authors would like to acknowledge Arcus and the Data Science and Biostatistics Unit at CHOP for their support in data management and geocoding the license address data.

BMV

Bureau of Motor Vehicles

CHOP

Children’s Hospital of Philadelphia

CI

confidence interval

VDA

virtual driving assessment