The effect of driver improvement interventions on crash involvement; has it been under-estimated?

The available evidence suggests that driver improvement interventions (with the aim to increase driver safety, most often by education or training) do not work. The average effect calculated in several meta-analyses is close to, and not always possible to distinguish from, zero, despite total samples sizes of several hundred thousand drivers. However, it is possible that all studies included in these meta-analyses have under-estimated the effect, due to a methodological error; all crashes have been used as dependent variable, instead of only those that the targeted drivers have caused. This error is expected to have considerably deflated the effect sizes, but it is not known how large this effect could be.

Using crash data for bus drivers in which culpability had been reliably established, a simple simulation was performed to determine the difference between using culpable and all crashes as an estimator of a safety effect. Using data for six years, calculations were made on single years. About ten percent of culpable crashes in each year were deleted to simulate a safety effect, where after the difference between the original and the simulated variable were calculated, using culpable only and all crashes in parallel. The effects using these two different kinds of datasets could then be compared and the under-estimation effect estimated.

Culpable crashes, as compared to all crashes, yielded larger differences in means between time periods, and smaller standard deviations. In between-subjects comparisons resulted in 15–30 percent larger effects for culpable crashes. Within-subjects calculations yielded larger but not as systematic effects.

The effect of driver improvement on crash involvement has been systematically under-estimated, as extremely few evaluation studies seem to have taken culpability for crashes into account. Therefore, new evaluations need to be undertaken, and/or old data re-analysed, to calculate a better estimate of the true effect of training and education in driving safety.