How reliable are self-report measures of mileage, violations and crashes?

Abstract

The use of self-reported driver mileage, violations and crashes is very popular in traffic safety research, but their validity has been questioned. One way of testing validity is with an analysis of test–retest reliability. Three mechanisms might influence reliability in self report; actual changes in the variable over time, stable systematic reporting bias, and random error. Four samples of drivers who had responded twice to an online questionnaire asking them to report their mileage, violations and crashes were used and correlations between self reports for this data were calculated. The results for crashes were compared to expected correlations, calculated from the error introduced by the non-overlapping periods and the variable means. Reliability was fairly low, and controlling for mileage in the violations and crashes calculations did not strengthen the associations. The correlation between self reports of crashes in different time periods was found to be much larger than expected in one case, indicating a report bias, while the other correlation agreed with the predicted value. The correlations for overlapping time periods were much smaller than expected. These results indicate that drivers’ self reports about their mileage, violations and crashes are very unreliable, but also that several different mechanisms are operating. It is uncertain exactly under what circumstances different types of self report bias is operating. Traffic safety researchers should treat the use of self-reported mileage, violations and crashes with extreme caution and preferably investigate these variables with the use of objective data.