Motorcycle Safety Puzzle Piece: Training, Part II
As I wrote in the previous entry, motorcyclists believe that training is a key part of the motorcycle safety puzzle. Rider educators are particularly committed to the notion–even though they rarely know how their students perform after leaving the course. The Motorcycle Safety Foundation itself has not claimed for more than a decade that training is effective in reducing crashes. Then again, few motorcyclists are aware that rider training–and, in particular, MSF’s training–has been studied over and over again.
Here’s a list with summary[i] of twenty-three training and/or licensing studies that have been done over the past thirty years. Bibliographic detail is given in footnotes. Quotations are drawn either from the study or abstract itself or from “Evaluation Of Rider Training Curriculum In Victoria”.[ii]
A twenty-third paper, the most recent, will be presented in more detail in the next entry.
1. In 1979, a British medical journal reported, “A University of Salford team tried to assess the effectiveness of training. Overall, 65% of the riders had accidents in their first three years, the untrained group faring slightly better than the trained. But the groups may not have been sufficiently comparable, Raymond pointed out; and intrinsically poor riders who would have given up without training could have affected the results.”[iii]
2. In 1978, Canadian researchers Jonah, et. al, evaluated the Motorcycle Operator Skill Test (MOST).[iv] “It was expected that the greater the riding experience and training, the higher would be the skill scores….skill was greater among the more experienced riders (i.e. miles ridden motorcycle)but unexpectedly it was lower among trainingcourse graduates. Further analyses revealed that course graduates had less skill than untrained riders even when experience differences were controlled.”
3. A Ph.D. dissertation published in 1980 on the South Dakota training program[v] found that, “Survival Rate Analyses indicated that Course riders who did not pass the course were not more likely to have accidents than riders who passed. Course graduates had a higher accident rate for mileage covered before and after the course than the untrained subjects.”
“This result also occurred when subjects were matched on relevant background variables. It was concluded that MSF Rider Course graduates are as likely to become involved in accidents as untrained riders.”
When the MSF course scores were analyzed in conjunction with the Motorcycle Operator Skills Test (MOST), “There were no relationships found between skill test item scores and types of accidents which implied some deficiency in those skills.”
4. Also in 1980, R.S. Satten produced a report for the Illinois Department of Transportation[vi] based on a group of 200 riders. He found MRC riders were less likely to have had crashes or citations but they also tended to be female and had fewer years of experience and rode less per week.
5. Another 1980 study by J.W. Anderson on 40,000 San Diego license-applicants at six months and a year out[vii] and found, “The improved procedures programs had significantly lower motorcycle accident rates after 6 months than the control group and the lowest rate belonged to the group with remedial training (30 minutes classroom and 2.5 hrs. skill training). After one year, riders in the group which included remedial training for those who failed had 14% fewer total fatal and injury accidents than those in the current procedures program. This was still true when controlling for riding exposure.”
6. In 1981, Jonah, et. al.[viii] did a study to determine whether trained riders (MTP) “were less likely to have had an accident or committed a traffic violation while riding a motorcycle compared to informally trained (IT) motorcyclists.” However, they found that those who had successfully passed MOST were 42% more likely to be crash-involved than those who failed.
However, “Multivariate analyses, controlling for the differences in [sex, age, time licensed, distance travelled [sic], education and BAC] revealed that the MTP graduates and IT riders did not differ in accident likelihood but the MTP graduates were significantly less likely to have committed a traffic violation than the IT riders. Although the lower incidence of traffic violations among graduates could be attributed to the training program, it is possible that the graduates sought formal training because they were safety conscious and this attitude also influenced their riding behaviour.”
7. In 1984, in a study involving 516 trained and a control group of untrained riders over three years, Rudolf G.Mortimer found that, ”when controlling for age and years licensed, those who took the course did not have a lower accident rate than the control group; (b) there were no differences in the violation rates between the groups; (c) the cost of damage to the motorcycles per million miles was not less for those who took the course; and (d) the estimated cost of medical treatment of injuries per million miles was not significantly less for the group which took the course.”[ix]
8. There was another study done in 1984 for the Illinois Department of Transportation that “compared a group of participants in the training program with a control group of people who had a current valid motorcycle licence. Some members of each group never actually rode. The trained riders rode less often, rode less powerful machines, were less likely to own a motorcycle and were less likely to hold a licence. Not surprisingly, trained riders were less likely to report having been involved in an accident or obtaining a moving violation or infringement notice. They had, however, been involved in fewer accidents per mile ridden.”[x]
9. A study published in 1986 comparing almost 60,000 riders who took either the California test or Alt-MOST[xi] and either took training (basic or remedial) or not. It found, “applicants assigned to the MOST II group had more fatal and injury motorcycle accidents and motorcycle convictions than applicants assigned to the Standard Test. When the analysis was restricted to the licensed riders, the MOST II riders had more total motorcycle accidents at the 2-year stage and more motorcycle and total convictions than the riders in the Standard Test group.”
10. Wisconsin produced a report in 1987.[xii] It used three-years of data involving almost 3,000 MRC graduates and about 43,000 untrained riders. When “Z” tests were applied, it found, “[T]here seem[s] to be no significant differences between the group with MRC training and the group without MRC training. Based upon this analysis, the effectiveness of the MRC program in reducing accidents among motorcyclists has been very small or not significant at all.”
MRC graduates did have a significantly lower ratio of citations—but once again, the authors point out that they weren’t able to control for things that might change that—and they point out that 50 percent of graduates did not go on to get their endorsements.
The study points out that the majority (56.8%) of MRC graduates were female—and that significantly affected the ratios of citations and crashes in a positive way while the control group was overwhelming male (93.4%). Women had much better safety records than males in both groups—but the disparity between men and women in the two groups did make a difference. When just male trained and untrained riders are compared, the accident ratio of all male MRC trained was 0.043 compared to 0.53 for untrained males. Trained women had a 0.10 and untrained women 0.11 ratio for crash-involvement as percentages of the whole.
11. A study on the New York licensing system that compared the state test with MOST[xiii] was published in 1988. While the attrition/failure rates were similar between both groups, it found there was no differences between groups that received shorter or longer training or no training or took one test or the other and got their license. Nor were there differences in crash severity between licensed and unlicensed groups one year later.
“Neither the skill test nor the training course was shown to be any more effective for riders who had previous riding experience compared to novice riders.
“Riders who attempted the MOST II had higher failure rates on their first attempt at a licence than control group riders who attempted the current New York test. Trained riders did not do better on their first attempt at the MOST II than untrained riders. Riders in the twenty-hour training group did worse than those in the three-hour group on their first attempt at the MOST II. The untrained riders performed better than the trained riders on their first attempt at the sub-tests of the MOST II that assessed correct braking procedures and obstacle avoidance.
“Those riders who were assessed by the MOST II as showing higher skill levels were not significantly less likely to be involved in subsequent motorcycle accidents.”
12. In 1988, Mortimer published the results of a further 913 graduates and a control group of 500 untrained riders. Once again he found that graduates of the MRC did not have a lower violation rate, accident rate, total cost of damage to accident-involved motorcycles, a significantly lower mean cost of injury treatment per accident, or a lower total cost of injury treatment. Mortimer’s study found that 30% of those who took the course did not ride afterwards. MRC graduates had more loss-of-traction crashes (gravel/low-friction pavement). Untrained riders had twice as many multi-vehicle crashes as trained riders. [xiv]
13. A dissertation published in 1989 on the Texas program[xv] matched trained and untrained riders and followed 988 of them. At the end of four years it found, “…the trained respondents were not significantly different from the comparison respondents.”
“The trained motorcyclists had 2.4 times the rate of a motorcycle crash compared with untrained motorcyclists,” and that “Most of the excess risk experienced by the trained group occurred within two years of training.”
14. A 1989 study, this time by McDavid, et. al,[xvi] was done on the British Columbia’s Safety Council’s 37-hour training program. It found, “Trained riders tend to have fewer accidents of all kinds (all motor vehicle accidents combined), fewer motorcycle accidents, and less severe motorcycle accidents. Although these differences are not large in a statistical sense, they suggest that when care is taken to carefully match trained and untrained riders, training is associated with a reduction in accidents.
15. In 1990, a paper, “The promotion of motorcycle safety: training, education, and awareness”[xvii] found, “Very little support for the beneficial impact of education/training can be found in the evaluation literature.” And suggested one of the reasons for that is “the need to focus more on rider motivations and attitudes than on skills….”
16. A 1990 evaluation of the Ohio state program with 2,000 trainees and a 6,000 licensed but untrained control group[xviii] found that, “A higher percentage of the trainees who had scored in the highest skill category had been involved in a motorcycle crash than those in all other skill test categories. However, those trainees who obtained scores above 85% on the knowledge test appeared to have a lower motorcycle crash involvement rate in 1989.”
17. In 1991, Billheimer, et. al., published his report on the California program.[xix] Overall, trained students had less crash-involvement six month after they began riding—however, most of that effect came from true novices—those who had not ridden more than 500 miles before training. Those who had more than 500 miles of experience had a slightly higher rate of crash-involvement than untrained riders, but it was not a statistically significant difference.
At one year, comparisons “show no significant differences between the accident rates of trained and untrained riders one year after training.” Nor were there any differences at two years.
18. In a 1994 report to the California state legislature on the MRC:RSS published in 1995 Wilson, et. al,[xx] found that 44% of students failing to complete the course said they no longer rode when interviewed a year after training, compared with 24% of those who passed and of those who no longer rode, 16 percent said taking the RSS influenced their decision. “Preliminary analyses showed that accident rates for untrained riders appeared to be 10% higher than for their trained counterparts in the six months after training.”
19. Though not a study on training or licensing, the European accident causation study, Motorcycle Accident In-Depth Study (MAIDS) found: “”When the accident population and the exposure population are compared, the data indicates that a similar number of riders in both groups have received no PTW training (40.1% of the accident population and 48.4% of the petrol station population). However, it is important to note that the PTW training status for 93 riders was coded as unknown. … The data indicates that 47.2% of those riders without any type of training failed to attempt a collision avoidance manoeuvre. Similarly, the data indicates that 33.2% of those riders who had compulsory training also failed to attempt a collision avoidance manoeuvre. These results are difficult to interpret since there were many cases in which there was insufficient time available for the PTW rider to perform any kind of collision avoidance.”
20. Motorcycle safety researchers at Monash University Accident Research Centre in Melbourne, Australia, produced a review of current licensing and training practices around the world[xxi] in 2005. Assessing all the evidence in a multitude of countries (as well as their previous literature review published in 2000), the authors conclude, “There is no real evidence of particular programs or components leading to reductions in crash risk….” “Standard motorcycle training courses leading to standard motorcycle tests have not been shown to result in reductions in crash involvement.”
21. Another Monash study published in 2006[xxii] done on older riders found “no significant relationship between involvement in one or more crashes in the past five years and having completed a training course at some time for fully-licensed riders … although there was a trend towards (p<.01) an association between having completed training and involvement in multiple vehicle crashes.” Nor was there any relationship between trained or untrained and crash severity. Nor was there a significant relationship between how recently or distantly one took a course and crashing.
22. In 2007, the Australians did a focus group study on 40 riders that had taken adult-centered training (various curriculum based on Q-ride).[xxiii] Participants felt/believed that training helped them avoid crashes however, in reality, there were several crashes and near misses where training didn’t help. The authors suggested “This suggests either a lack of learning transfer, a decay over time of information learnt, or that other factors not addressed in training (e.g. of an attitudinal or motivational nature) influenced rider behaviour once licensed. Training is therefore arguably not enough to always keep riders safe in the traffic environment unless skills are practised, honed and tempered with self-control.”
It also found that “that the most salient information from training is that which has been subsequently experienced on-road. 1) information from training may decay unless subsequently reinforced by experience; 2) learners may be more able to integrate information from training once they have had some riding experience as opposed to the pre-license stage where there is potential for ‘information overload’ due to the cognitive resources required in initial skill acquisition; and 3) the information may become more personally relevant to novice riders once some experience has been gained.”
23. Also in 2007, a study was done on Indiana riders.[xxiv] The authors found that riders who took “beginning rider training courses are more likely to be accident involved than those that do not – and that those that take the beginning course more than once are much more likely to be accident involved.”
“The Motorcycle Safety Foundation’s Basic Rider Course was found to be significant with three variables in the accident model. For the first variable, those that completed the Basic Rider Course were found to be 44% more likely to be accident involved. This may reflect the ineffectiveness of the course, the fact that the course is attracting an inherently less skilled set of riders and/or the post-course skill set is being used to ride more aggressively….Commenting on the effectiveness of the material taught in the Basic Rider Course is beyond the scope of this paper. In terms of the course attracting inherently less skilled riders, we do control for a wide range of variables in our model. However, it is possible that unobservable variables that are not correlated with those included in the model are still influencing our estimates here.”
The second course-related finding was that those that completed the Basic Rider Course multiple times were an additional 180% more likely to be accident involved. This finding may reflect the fact that people that take the course repeatedly are trying to improve an inherently diminished skill set (or one that changes over time) that affects their accident likelihoods. Thus, this variable may be capturing one’s inherent ability to master, or the need to refresh, the relatively complex physical and mental skills necessary to operate a motorcycle. Interestingly, there was no significant age difference between people that took the Basic Rider Course once and those that took it multiple times (both roughly 45 years of age). However, those that took the course multiple times had, on average, almost 12 more years of experience. It appears that more experienced riders –perhaps those noting a decline in their skills or those having had recent experiences with near misses – are more likely to take the Basic Riding Course repeatedly.
“People that cited no need for taking the Basic Rider Course were 51% less likely to be accident involved (the average age of these riders was 24.4 years, and 85% of these riders had 5 or more years of experience). This seems to provide some supporting evidence that the people taking the beginner course may be inherently less-skilled riders. It is also interesting to note that 12% of our sample took the Motorcycle Safety Foundation’s Experienced Rider Course (the sequel to the Basic Rider Course) but this did not have a statistically significant effect, positive or negative, on accident probabilities.”
In the next entry, we’ll look at the most recent work and what it has to say about training and licensing.
[i] Usually, the summaries are my own or drawn directly from the studies or abstracts of the studies. On rare occasions, I preferred to use a quote from Narelle Haworth, et. al.’s Evaluation Of Rider Training Curriculum In Victoria. Monash Accident Research Centre. 2000.
[ii] Haworth, Narelle and Rob Smith, Naomi Kowadlo. Evaluation Of Rider Training Curriculum In Victoria. Monash University Accident Research Centre. 2000.
[iii] Motorcycle And Bicycle Accidents Source: The British Medical Journal, Vol. 1, No. 6155 (Jan. 6, 1979), pp. 39-41 Published by: BMJ Publishing Group. The report referred to was, Raymond, S and Tatum, S (1977). An evaluation of the effectiveness of the RAC/ACU motorcycle training scheme. A Final Report. University of Salford, Department of Civil Engineering, Road Safety.
Research Unit. Salford.
[iv] Jonah Brian A. and Nancy E. Dawson. Validation of the motorcycle operator skill test. Road and Motor Vehicle Traffic Safety Branch, Transport Canada. 1978.
[v] Osga, Glenn Arthur. An Investigation Of The Riding Experiences Of Msf Rider Course Participants. University Of South Dakota.
[vi] Satten, R.S. Analysis and evaluation of the motorcycle rider courses in thirteen Illinios counties. Proceedings of the International Motorcycle Safety Conference, Washington DC, Vol. 1, 145-193. 1980.
[vii] Anderson, J.W. The effects of motorcycling licensing and skills training on the driver records of original applicants. Proceedings of the International Motorcycle Safety
Conference, Washington DC, USA, Vol. 1, 381-401. 1980.
[viii] Jonah, B.A., Dawson, N.E., & Bragg, W.E.. Are formally trained motorcyclists safer?
Accident Analysis and Prevention, 14(4), 247-255. 1982.
[ix] Mortimer, Rudolf G. Evaluation of the motorcycle rider course. Accident Analysis & Prevention, Volume 16, Issue 1. February 1984. Pages 63-71.
[x] As reported in Evaluation Of Rider Training Curriculum In Victoria. Narelle Haworth, et. al. Monash University Accident Research Centre. 2000. Lakener, E. A survey of motorcycle riders in Illinois. A report submitted to the Traffic Safety Division, Illinios Department of Transportation. 1984.
[xi] Kelsey, S.L., Liddicoat, C., & Ratz, M. Licensing novice motorcyclists: A comparison of California’s standard test and the MOST II (Motorcycle Operator Skill Test) administered at centralised testing offices. Research Report of the California Department of Motor Vehicles, Research and Development Office. 1986.
[xii] Leung, Kam S. and Vernon A. Reding. Evaluation of the Wisconsin Motorcycle Rider Course. Wisconsin Department of Transportation. 1987.
[xiii] Buchanan, L.S. (1988). Motorcycle rider evaluation project. Report prepared for the US
Department of Transportation, National Highway Traffic Safety Administration,
[xiv] Henderson, Michael. Education, Publicity and Training in Road Safety: A Literature Review. Monash University Accident Research Study.. 1991.
[xv] Lloyd, Linda Elizabeth. An evaluation of the Texas motorcycle operator training course. 1989. The University of Texas at Austin.
[xvi] McDavid, James C.; Lohrmann, Barbara A.; and Lohrmann, George. Does Motorcycle Training Reduce Accidents? Evidence for a Longitudinal Quasi-Experimental Study. Journal of Safety Research, Vol. 20, pp. 61-72, 1989.
[xvii] Simpson, . M. and D. R. Mayhew. The promotion of motorcycle safety: training, education, and awareness. Oxford Univ Press. 1990.
[xviii] Rockwell, T.H., Kiger, S.M., & Carnot, M.J. An evaluation of the Ohio motorcyclists enrichment program, Phase II initial assessment report. Prepared for the Ohio Department of Highway Safety. 1990.
[xix] Billheimer, J.W. California Motorcyclists Safety Program; Final evaluation report. Prepared for California Highway Patrol, under contract to Crain and Associates. 1991.
[xx] Wilson, P., Dunphy, D. & Hannigan, M.J. (1995). The California Motorcyclist Safety Program: 1994 Annual Report to the State Legislature.
[xxi] Haworth, N. & Mulvihill, C. Review of motorcycle licensing and training (Report No. 240). Melbourne: Monash University Accident Research Centre. 2005
[xxii] Haworth, Narelle and Mulvihill, Christine and Rowden, Peter. Teaching old dogs new tricks? Training and older motorcyclists.. In Proceedings Australasian Road Safety Research, Policing and Education Conference, Gold Coast, Queensland.. 2006.
[xxiii] Rowden, Peter J. and Watson, Barry C. and Haworth, Narelle L. What can riders tell us about motorcycle rider training? A view from the other side of the fence. In Proceedings 2007 Australasian Road Safety Research, Policing and Education Conference, Melbourne, Australia. 2007.
[xxiv] Savolainen, Peter and Fred Mannering. Additional evidence on the effectiveness of motorcycle training and motorcyclists’ risk-taking behaviour. TRB 2007 Annual Meeting CD-ROM. 2007.