Archive for the ‘State Motorcycle Safety Programs’ category

Another death in Motorcycle Safety Foundation Training

July 18, 2010

There’s been another death as the result of an MSF-curriculum rider training class. There may have been more—but this is the latest one I have heard about from an alert reader and loyal friend:

Fifty-five year-old James Lawrence Smith was taking a MSF-curriculum rider training course at Florence-Darlington Technical College in Florence, SC on Saturday, July 10, 2010 when he thrown from his bike after he lost control. According to his family, he died from his injuries. The same news story can be found here and here and here.

The articles do not say which course Smith was taking at the time of his crash.

Rider training is offered by the state motorcycle training program in South Carolina on community and technical college campuses.  The Florence-Darlington Technical College website says the continuing ed program offers beginning, intermediate and expert classes, however the available courses button only links to intermediate classes.

That is all the information I have at this time.

This is the 12th death that we know about in training in the past eight years. From 1973-2001 and almost 1.5 million riders trained, there had only been one death due to rider error. From 2002 until now, there have been at least 12 and several other near-fatal injuries.

Motorcycle puzzle piece: training, part III

January 14, 2010

The twenty-third study is a 2008 Centre for Accident Research & Road Safety – Queensland report, “Identifying Programs To Reduce Road Trauma To Act Motorcyclists”[i] While it deals with many issues a significant part of it looks at motorcycle training and licensing programs.

The report is for the Australian Capital Territory (ACT), a tiny little bean-shaped area  surrounded by New South Wales. Canberra, the capital of Australia is in ACT. The population of ACT is roughly the size of Raleigh, NC or Tulsa, OK or Minneapolis, MN.

It has relatively few riders and few deaths since most riders crash in New South Wales. This report outlines the best motorcycle safety program for ACT.

It highlights two ways to reduce crashes: exposure reduction and risk reduction. Exposure reduction limits the number of riders and the miles they ride—something that neither riders nor the motorcycle industry would support. Risk reduction cuts down on the hazards and numbers of them that riders take/are exposed to. The report points out that risk reduction rather than exposure reduction “that can also work to reduce the severity of injury in the event of a crash.”

Training programs, the study points out can result in exposure reduction when people choose not to ride because of the difficulty of taking/passing a course. But it is in risk reduction where training programs would be expected to shine.

The situation in Australia is somewhat different than in the USA. It has a variety of programs—basic and beyond—available in the various states—and has graduated licensing—first a learner’s permit, then a provisional permit and then a full motorcycle license. There are training programs for the first and second level and in some states training for the first level is compulsory. Training programs to obtain the learner’s permit last between 6-16 hours and the second level of training lasts between 6 and 12 hours. Iow, Australian riders can take more than twice the training before being fully-licensed.

Nor is there one specified curriculum in a state as in the USA. In Queensland, for example, the state sets a strict set of standards that “quantify what a learner must do and how well it must be done to enable them to apply to Queensland Transport for the issue of the class of licence they have been trained and assessed for through Q-RIDE.”  But it does not publish a curriculum that every training provider must use.

The report finds that all programs are not created equal: there can be a positive, neutral or even negative effect on motorcycle safety:

“Programs which may possibly have a negative effect on safety are those that aim to, or are likely to increase exposure… [or] which knowingly or unknowingly promote or encourage increased riding,” or “produce over-confidence in riders” if it “lead[s] to riskier riding behaviour.”

The reports says that some training programs are “likely” to be “beneficial” if they are:

  • training programs that are research-based,
  • use risk reduction and/or exposure  measures and
  • reaches a large number of the audience for which it was intended.
  • Motorcycle safety should increase by addressing a combination of road user, vehicle, and environment-based measures as well as
  • a combination of crash prevention measures and the reduction in the severity of injury and treatment improvements.

Many would argue that the USA’s Motorcycle Safety Foundation curriculum does exactly that.

However, the report states, having the elements is not enough. The researchers pointed out that determining what programs could have a beneficial effect is difficult.

“In terms of identifying effective programs, the most serious limitation was the lack of evaluation of program effectiveness.”  The authors remarked it wasn’t surprising on a local level but that “many large statewide programs had only limited (or no) process evaluation available and very few had an outcome evaluation. Thus, very few programs can be said to be “proven beneficial,” although there are quite a few that are “likely beneficial”.”

The report later states, “There is no strong evidence in support of training leading to marked improvements in rider safety (Haworth & Mulvihill, 2005). An international review of motorcycle training concluded that compulsory training through licensing programs produces a weak but consistent reduction in crashes but voluntary motorcycle training programs do not reduce crash risk (TOI, 2003).  On the contrary, these programs seem to increase crash risk.  This may be due, in part, to the increased confidence felt by many riders who have completed training, despite minimal improvements in rider skill.  Such riders may therefore take more risks in situations where they lack the skills to safely avoid a crash.”

In short: while training has the potential to be beneficial, there’s little-to-no proof that it is:  “Many authors have concluded that the apparent lack of success of rider training in reducing accident risk or number of violations may stem from the content of the training programs (Chesham, Rutter & Quine, 1993; Crick & McKenna, 1991; Haworth, Smith & Kowadlo; 1999; Reeder, Chalmers & Langley, 1996; Simpson & Mayhew, 1990).   Rider training programs currently in use focus mainly on the development of vehicle control skills.  This is not necessarily through choice but is often brought about through time constraints and the need to prepare a rider for an end test that is skill-based.”

“In their review of motorcycle licensing and training methods throughout Australia, Haworth and Mulvihill (2005) argued that motorcycle riding requires higher levels of vehicle control and cognitive skills in comparison to car driving and suggested that future motorcycle safety initiatives need to incorporate activities promoting higher level cognitive and control skills.”

Based on years of intense, comprehensive and global research, the experts put forth the best practices in training and licensing:

Table 4.1    Summary of best practice components for motorcycle licensing system

Component Effect on crash risk Effect on crash severity Effect on amount of riding Reason for effect Is this current practice in the ACT?
No exemptions from licensing, training or testing requirements for older applicants Facilitates other measures Facilitates other measures Reduces it Older riders need to develop riding-specific skills.  May make licensing less attractive. NO:  Exemptions are made for older applicants and those who already hold a car licence.
Minimum age for learner and provisional motorcycle licences higher than for car licences Reduces it Reduces it Consistent with graduated licensing principles. Crash risk has been demonstrated to decrease with age among young novices.  Increasing the minimum age would also almost eliminate riding and therefore crashes among riders below this age. YES
Zero BAC for L and P Reduces it Reduces it Reducing drink riding will reduce crash risk.  Zero BAC will also reduce the amount of riding after drinking. NO: 0.02% for L & P
Restrictions on carrying pillion passengers for L and P Reduces it Reduces it Pillions have been shown to increase crash risk and severity. YES: for L, and P in first 12mths
Power-to-weight restrictions for L and P Reduces it(severe crashes) Reduces it Reduces it Crash risk may be reduced if less powerful motorcycles result in less deliberate speeding and risk taking or problems with vehicle control.  Restrictions may dissuade some potential high-risk riders from riding. YES
Minimum periods for L and P Facilitates other measures Facilitates other measures Unknown To ensure that other requirements have sufficient duration. YES

Australia already has a graduated licensing and power-to-weight ratios (that can be offset by training). Already there’s on-road testing in some of the states. Already, then, at least some states in Australia have stricter standards than almost all USA states.

The report then summarizes the best practices for motorcycle training:

Table 4.2    Summary of best practice components for motorcycle training

Component Effect on crash risk Effect on crash severity Effect on amount of riding Reason for effect Is this current practice in the ACT?
Compulsory training to obtain L and P Small reduction Unknown Reduces it Ensure a basic level of competency.  May make licensing less attractive. Yes for L, no for P
Comprehensive roadcraft training at both L and P (may require longer training duration and better educational skills of trainers) Reduces it Reduces it Reduces it Improved ability to detect and respond to hazards by novice riders.  Longer and potentially more expensive training may deter some applicants. NO
Off-road training for L, mix of on- and off-road training for P Reduces it Reduces it Reduces it Ensure a basic level of competency gained under situations that are appropriate for current level of competency.  Allow safe practice of responses to hazards.  Longer and potentially more expensive training may deter some applicants.

As we see, many of the components of both training and licensing that would lead to more competent and possibly safer riders on the road are also ones that would likely reduce exposure even if they don’t–or while they do–reduce risk.

The bottom line? The  best experts in motorcycle safety conclude that the best chance of motorcycle safety will have the side effect of reducing the number of riders.

[i] Greig Kristi, Narelle Haworth and Darren Wishart. “Identifying Programs To Reduce Road Trauma To Act Motorcyclists”, The Centre for Accident Research & Road Safety—Queensland. Australia, February 2008.

Motorcycle Safety Puzzle Piece: Training, Part II

January 12, 2010

As I wrote in the previous entry, motorcyclists believe that training is a key part of the motorcycle safety puzzle. Rider educators are particularly committed to the notion–even though they rarely know how their students perform after leaving the course. The  Motorcycle Safety Foundation itself has not claimed for more than a decade that training is effective in reducing crashes. Then again, few motorcyclists are aware that rider training–and, in particular, MSF’s training–has been studied over and over again.

Here’s a list with summary[i] of twenty-three training and/or licensing studies that have been done over the past thirty years. Bibliographic detail is given in footnotes. Quotations are drawn either from the study or abstract itself or from “Evaluation Of Rider Training Curriculum In Victoria”.[ii]

A twenty-third paper, the most recent, will be presented in more detail in the next entry.

1. In 1979, a British medical journal reported, “A University of Salford team tried to assess the effectiveness of training. Overall, 65% of the riders had accidents in their first three years, the untrained group faring slightly better than the trained. But the groups may not have been sufficiently comparable, Raymond pointed out; and intrinsically poor riders who would have given up without training could have affected the results.”[iii]

2. In 1978, Canadian researchers Jonah, et. al, evaluated the Motorcycle Operator Skill Test (MOST).[iv] “It was expected that the greater the riding experience and training, the higher would be the skill scores….skill was greater among the more experienced riders (i.e. miles ridden motorcycle)but unexpectedly it was lower among trainingcourse graduates. Further analyses revealed that course graduates had less skill than untrained riders even when experience differences were controlled.”

3. A Ph.D. dissertation published in 1980 on the South Dakota training program[v] found that, “Survival Rate Analyses indicated that Course riders who did not pass the course were not more likely to have accidents than riders who passed. Course graduates had a higher accident rate for mileage covered before and after the course than the untrained subjects.”

“This result also occurred when subjects were matched on relevant background variables. It was concluded that MSF Rider Course graduates are as likely to become involved in accidents as untrained riders.”

When the MSF course scores were analyzed in conjunction with the Motorcycle Operator Skills Test (MOST), “There were no relationships found between skill test item scores and types of accidents which implied some deficiency in those skills.”

4. Also in 1980, R.S. Satten produced a report for the Illinois Department of Transportation[vi] based on a group of 200 riders. He found MRC riders were less likely to have had crashes or citations but they also tended to be female and had fewer years of experience and rode less per week.

5. Another 1980 study by J.W. Anderson on 40,000 San Diego license-applicants at six months and a year out[vii] and found, “The improved procedures programs had significantly lower motorcycle accident rates after 6 months than the control group and the lowest rate belonged to the group with remedial training (30 minutes classroom and 2.5 hrs. skill training). After one year, riders in the group which included remedial training for those who failed had 14% fewer total fatal and injury accidents than those in the current procedures program. This was still true when controlling for riding exposure.”

6. In 1981, Jonah, et. al.[viii] did a study to determine whether trained riders (MTP) “were less likely to have had an accident or committed a traffic violation while riding a  motorcycle  compared to informally trained (IT) motorcyclists.” However, they found that those who had successfully passed  MOST were 42% more likely to be crash-involved than those who failed.

However, “Multivariate analyses, controlling for the differences in [sex, age, time licensed, distance travelled [sic], education and BAC] revealed that the MTP graduates and IT riders did not differ in accident likelihood but the MTP graduates were significantly less likely to have committed a traffic violation than the IT riders. Although the lower incidence of traffic violations among graduates could be attributed to the training program, it is possible that the graduates sought formal training because they were safety conscious and this attitude also influenced their riding behaviour.”

7. In 1984, in a study involving 516 trained and a control group of untrained riders over three years, Rudolf G.Mortimer found that, ”when controlling for age and years licensed, those who took the course did not have a lower accident rate than the control group; (b) there were no differences in the violation rates between the groups; (c) the cost of damage to the  motorcycles  per million miles was not less for those who took the course; and (d) the estimated cost of medical treatment of injuries per million miles was not significantly less for the group which took the course.”[ix]

8. There was another study done in 1984 for the Illinois Department of Transportation that “compared a group of participants in the training program with a control group of people who had a current valid motorcycle licence. Some members of each group never actually rode. The trained riders rode less often, rode less powerful machines, were less likely to own a motorcycle and were less likely to hold a licence. Not surprisingly, trained riders were less likely to report having been involved in an accident or obtaining a moving violation or infringement notice. They had, however, been involved in fewer accidents per mile ridden.”[x]

9. A study published in 1986 comparing almost 60,000 riders who took either the  California test or Alt-MOST[xi] and either took training (basic or remedial) or not. It  found, “applicants assigned to the MOST II group had more fatal and injury motorcycle accidents and motorcycle convictions than applicants assigned to the Standard Test. When the analysis was restricted to the licensed riders, the MOST II riders had more total motorcycle accidents at the 2-year stage and more motorcycle and total convictions than the riders in the Standard Test group.”

10. Wisconsin produced a report in 1987.[xii] It used three-years of data involving almost 3,000 MRC graduates and about 43,000 untrained riders. When “Z” tests were applied, it  found, “[T]here seem[s] to be no significant differences between the group with MRC training and the group without MRC training. Based upon this analysis, the effectiveness of the MRC program in reducing accidents among motorcyclists has been very small or not significant at all.”

MRC graduates did have a significantly lower ratio of citations—but once again, the authors point out that they weren’t able to control for things that might change that—and they point out that 50 percent of graduates did not go on to get their endorsements.

The study points out that the majority (56.8%) of MRC graduates were female—and that significantly affected the ratios of citations and crashes in a positive way while the control group was overwhelming male (93.4%). Women had much better safety records than males in both groups—but the disparity between men and women in the two groups did make a difference. When just male trained and untrained riders are compared, the accident ratio of all male MRC trained was 0.043 compared to 0.53 for untrained males. Trained women had a 0.10 and untrained women 0.11 ratio for crash-involvement as percentages of the whole.

11. A study on the New York licensing system that compared the state test with MOST[xiii] was published in 1988. While the attrition/failure rates were similar between both groups, it found there was no differences between groups that received shorter or longer training or no training or took one test or the other and got their license. Nor were there differences in crash severity between licensed and unlicensed groups one year later.

“Neither the skill test nor the training course was shown to be any more effective for riders who had previous riding experience compared to novice riders.

“Riders who attempted the MOST II had higher failure rates on their first attempt at a licence than control group riders who attempted the current New York test. Trained riders did not do better on their first attempt at the MOST II than untrained riders. Riders in the twenty-hour training group did worse than those in the three-hour group on their first attempt at the MOST II. The untrained riders performed better than the trained riders on their first attempt at the sub-tests of the MOST II that assessed correct braking procedures and obstacle avoidance.

“Those riders who were assessed by the MOST II as showing higher skill levels were not significantly less likely to be involved in subsequent motorcycle accidents.”

12. In 1988, Mortimer published the results of a further 913 graduates and a control group of 500 untrained riders. Once again he found that graduates of the MRC did not have a lower violation rate, accident rate, total cost of damage to accident-involved motorcycles, a significantly lower mean cost of injury treatment per accident, or a lower total cost of injury treatment. Mortimer’s study found that 30% of those who took the course did not ride afterwards. MRC graduates had more loss-of-traction crashes (gravel/low-friction pavement). Untrained riders had twice as many multi-vehicle crashes as trained riders. [xiv]

13. A dissertation published in 1989 on the Texas program[xv] matched trained and untrained riders and followed 988 of them. At the end of four years it found, “…the trained respondents were not significantly different from the comparison respondents.”

“The trained motorcyclists had 2.4 times the rate of a motorcycle crash compared with untrained motorcyclists,” and that “Most of the excess risk experienced by the trained group occurred within two years of training.”

14. A 1989 study, this time by McDavid, et. al,[xvi] was done on the British Columbia’s Safety Council’s 37-hour training program. It found, “Trained riders tend to have fewer accidents of all kinds (all motor vehicle accidents combined), fewer motorcycle accidents, and less severe motorcycle accidents. Although these differences are not large in a statistical sense, they suggest that when care is taken to carefully match trained and untrained riders, training is associated with a reduction in accidents.

15. In 1990, a paper, “The promotion of motorcycle safety: training, education, and awareness”[xvii] found, “Very little support for the beneficial impact of education/training can be found in the evaluation literature.” And suggested one of the reasons for that is “the need to focus more on rider motivations and attitudes than on skills….”

16. A 1990 evaluation of the Ohio state program with 2,000 trainees and a 6,000 licensed but untrained control group[xviii] found that, “A higher percentage of the trainees who had scored in the highest skill category had been involved in a motorcycle crash than those in all other skill test categories. However, those trainees who obtained scores above 85% on the knowledge test appeared to have a lower motorcycle crash involvement rate in 1989.”

17. In 1991, Billheimer, et. al., published his report on the California program.[xix] Overall, trained students had less crash-involvement six month after they began riding—however, most of that effect came from true novices—those who had not ridden more than 500 miles before training. Those who had more than 500 miles of experience had a slightly higher rate of crash-involvement than untrained riders, but it was not a statistically significant difference.

At one year, comparisons “show no significant differences between the accident rates of trained and untrained riders one year after training.” Nor were there any differences at two years.

18. In a 1994 report to the California state legislature on the MRC:RSS published in 1995 Wilson, et. al,[xx] found that 44% of students failing to complete the course said they no longer rode when interviewed a year after training, compared with 24% of those who passed and of those who no longer rode, 16 percent said taking the RSS influenced their decision. “Preliminary analyses showed that accident rates for untrained riders appeared to be 10% higher than for their trained counterparts in the six months after training.”

19. Though not a study on training or licensing, the European accident causation study, Motorcycle Accident In-Depth Study (MAIDS) found: “”When the accident population and the exposure population are compared, the data indicates that a similar number of riders in both groups have received no PTW training (40.1% of the accident population and 48.4% of the petrol station population). However, it is important to note that the PTW training status for 93 riders was coded as unknown. … The data indicates that 47.2% of those riders without any type of training failed to attempt a collision avoidance manoeuvre. Similarly, the data indicates that 33.2% of those riders who had compulsory training also failed to attempt a collision avoidance manoeuvre. These results are difficult to interpret since there were many cases in which there was insufficient time available for the PTW rider to perform any kind of collision avoidance.”

20. Motorcycle safety researchers at Monash University Accident Research Centre in Melbourne, Australia, produced a review of current licensing and training practices  around the world[xxi] in 2005. Assessing all the evidence in a multitude of countries (as well as their previous literature review published in 2000), the authors conclude, “There is no real evidence of particular programs or components leading to reductions in crash risk….” “Standard motorcycle training courses leading to standard motorcycle tests have not been shown to result in reductions in crash involvement.”

21. Another Monash study published in 2006[xxii] done on older riders found “no significant relationship between involvement in one or more crashes in the past five years and having completed a training course at some time for fully-licensed riders … although there was a trend towards (p<.01) an association between having completed training and involvement in multiple vehicle crashes.” Nor was there any relationship between trained or untrained and crash severity. Nor was there a significant relationship between how recently or distantly one took a course and crashing.

22. In 2007, the Australians did a focus group study on 40 riders that had taken adult-centered training (various curriculum based on Q-ride).[xxiii] Participants felt/believed that training helped them avoid crashes however, in reality, there were several crashes and near misses where training didn’t help. The authors suggested “This suggests either a lack of learning transfer, a decay over time of information learnt, or that other factors not addressed in training (e.g. of an attitudinal or motivational nature) influenced rider behaviour once licensed. Training is therefore arguably not enough to always keep riders safe in the traffic environment unless skills are practised, honed and tempered with self-control.”

It also found that “that the most salient information from training is that which has been subsequently experienced on-road. 1) information from training may decay unless subsequently reinforced by experience; 2) learners may be more able to integrate information from training once they have had some riding experience as opposed to the pre-license stage where there is potential for ‘information overload’ due to the cognitive resources required in initial skill acquisition; and 3) the information may become more personally relevant to novice riders once some experience has been gained.”

23. Also in 2007, a study was done on Indiana riders.[xxiv] The authors found that riders who took “beginning rider training courses are more likely to be accident involved than those that do not – and that those that take the beginning course more than once are much more likely to be accident involved.”

“The Motorcycle Safety Foundation’s Basic Rider Course was found to be significant with three variables in the accident model. For the first variable, those that completed the Basic Rider Course were found to be 44% more likely to be accident involved. This may reflect the ineffectiveness of the course, the fact that the course is attracting an inherently less skilled set of riders and/or the post-course skill set is being used to ride more aggressively….Commenting on the effectiveness of the material taught in the Basic Rider Course is beyond the scope of this paper. In terms of the course attracting inherently less skilled riders, we do control for a wide range of variables in our model. However, it is possible that unobservable variables that are not correlated with those included in the model are still influencing our estimates here.”

The second course-related finding was that those that completed the Basic Rider Course multiple times were an additional 180% more likely to be accident involved. This finding may reflect the fact that people that take the course repeatedly are trying to improve an  inherently diminished skill set (or one that changes over time) that affects their accident likelihoods. Thus, this variable may be capturing one’s inherent ability to master, or the need to refresh, the relatively complex physical and mental skills necessary to operate a motorcycle. Interestingly, there was no significant age difference between people that took the Basic Rider Course once and those that took it multiple times (both roughly 45 years of age). However, those that took the course multiple times had, on average, almost 12 more years of experience. It appears that more experienced riders –perhaps those noting a decline in their skills or those having had recent experiences with near misses – are more likely to take the Basic Riding Course repeatedly.

“People that cited no need for taking the Basic Rider Course were 51% less likely to be accident involved (the average age of these riders was 24.4 years, and 85% of these riders had 5 or more years of experience). This seems to provide some supporting evidence that the people taking the beginner course may be inherently less-skilled riders. It is also interesting to note that 12% of our sample took the Motorcycle Safety Foundation’s Experienced Rider Course (the sequel to the Basic Rider Course) but this did not have a statistically significant effect, positive or negative, on accident probabilities.”

In the next entry, we’ll look at the most recent work and what it has to say about training and licensing.

[i] Usually, the summaries are my own or drawn directly from the studies or abstracts of the studies. On rare occasions, I preferred to use a quote from Narelle Haworth, et. al.’s Evaluation Of Rider Training Curriculum In Victoria. Monash Accident Research Centre. 2000.

[ii] Haworth, Narelle and Rob Smith, Naomi Kowadlo. Evaluation Of Rider Training Curriculum In Victoria. Monash University Accident Research Centre. 2000.

[iii] Motorcycle And Bicycle Accidents Source: The British Medical Journal, Vol. 1, No. 6155 (Jan. 6, 1979), pp. 39-41 Published by: BMJ Publishing Group. The report referred to was, Raymond, S and Tatum, S (1977). An evaluation of the effectiveness of the RAC/ACU motorcycle training scheme. A Final Report. University of Salford, Department of Civil Engineering, Road Safety.

Research Unit. Salford.

[iv] Jonah Brian A. and Nancy E. Dawson. Validation of the motorcycle operator skill test. Road and Motor Vehicle Traffic Safety Branch, Transport Canada. 1978.

[v] Osga, Glenn Arthur. An Investigation Of The Riding Experiences Of Msf Rider Course Participants.  University Of South Dakota.

[vi] Satten, R.S. Analysis and evaluation of the motorcycle rider courses in thirteen Illinios counties. Proceedings of the International Motorcycle Safety Conference, Washington DC, Vol. 1, 145-193. 1980.

[vii] Anderson, J.W. The effects of motorcycling licensing and skills training on the driver records of original applicants. Proceedings of the International Motorcycle Safety

Conference, Washington DC, USA, Vol. 1, 381-401. 1980.

[viii] Jonah, B.A., Dawson, N.E., & Bragg, W.E.. Are formally trained motorcyclists safer?

Accident Analysis and Prevention, 14(4), 247-255. 1982.

[ix] Mortimer, Rudolf G. Evaluation of the motorcycle rider course. Accident Analysis & Prevention, Volume 16, Issue 1. February 1984. Pages 63-71.

[x] As reported in Evaluation Of Rider Training Curriculum In Victoria. Narelle Haworth, et. al. Monash University Accident Research Centre. 2000.  Lakener, E. A survey of motorcycle riders in Illinois. A report submitted to the Traffic Safety Division, Illinios Department of Transportation. 1984.

[xi] Kelsey, S.L., Liddicoat, C., & Ratz, M. Licensing novice motorcyclists: A comparison of California’s standard test and the MOST II (Motorcycle Operator Skill Test) administered at centralised testing offices. Research Report of the California Department of Motor Vehicles, Research and Development Office. 1986.

[xii] Leung, Kam S. and Vernon A. Reding. Evaluation of the Wisconsin Motorcycle Rider Course. Wisconsin Department of Transportation. 1987.

[xiii] Buchanan, L.S. (1988). Motorcycle rider evaluation project. Report prepared for the US

Department of Transportation, National Highway Traffic Safety Administration,

Washington DC.

[xiv] Henderson, Michael.  Education, Publicity and Training in Road Safety: A Literature Review. Monash University Accident Research Study.. 1991.

[xv] Lloyd, Linda Elizabeth. An evaluation of the Texas motorcycle operator training course. 1989. The University of Texas at Austin.

[xvi] McDavid, James C.; Lohrmann, Barbara A.; and Lohrmann, George. Does Motorcycle Training Reduce Accidents? Evidence for a Longitudinal Quasi-Experimental Study. Journal of Safety Research, Vol. 20, pp. 61-72, 1989.

[xvii] Simpson, . M. and D. R. Mayhew. The promotion of motorcycle safety: training, education, and awareness. Oxford Univ Press. 1990.

[xviii] Rockwell, T.H., Kiger, S.M., & Carnot, M.J. An evaluation of the Ohio motorcyclists enrichment program, Phase II initial assessment report. Prepared for the Ohio Department of Highway Safety. 1990.

[xix] Billheimer, J.W. California Motorcyclists Safety Program; Final evaluation report. Prepared for California Highway Patrol, under contract to Crain and Associates. 1991.

[xx] Wilson, P., Dunphy, D. & Hannigan, M.J. (1995). The California Motorcyclist Safety Program: 1994 Annual Report to the State Legislature.

[xxi] Haworth, N. & Mulvihill, C. Review of motorcycle licensing and training (Report No. 240).  Melbourne: Monash University Accident Research Centre. 2005

[xxii] Haworth, Narelle and Mulvihill, Christine and Rowden, Peter. Teaching old dogs new tricks? Training and older motorcyclists.. In Proceedings Australasian Road Safety Research, Policing and Education Conference, Gold Coast, Queensland.. 2006.

[xxiii] Rowden, Peter J. and Watson, Barry C. and Haworth, Narelle L. What can riders tell us about motorcycle rider training? A view from the other side of the fence. In Proceedings 2007 Australasian Road Safety Research, Policing and Education Conference, Melbourne, Australia. 2007.

[xxiv] Savolainen, Peter and Fred Mannering. Additional evidence on the effectiveness of motorcycle training and motorcyclists’ risk-taking behaviour. TRB 2007 Annual Meeting CD-ROM. 2007.

Motorcycle safety puzzle piece: training

January 8, 2010

I apologize for formatting errors in the chart–for some reason, changes the font size part way through and won’t allow me to fix it.

Training is the next piece that’s supposed to solve the motorcycle safety puzzle. It’s the most important piece in this way: while helmets have their fervent and often vehement supporters and detractors, everyone agrees that training is axiomatic as an effective solution to the motorcycle safety puzzle.

As motorcycle rights organizations are fond of saying–don’t legislate, educate. Training (and to an extent licensing), it’s believed, keeps one out of situations that could lead to crashes.

Training in some form or another has been around since the earliest days. One of the first how-to-ride manuals,  Boy Scouts on Motorcycles, was copyrighted in 1912, and the earliest official course, the British Metropolitan Police Hendon Training System began in 1934 with a civilian version taught by the 1950s.  By the time the Motorcycle Safety Foundation (MSF) began in 1973 there were over 30 different courses, curriculum  and manuals–including Montgomery Ward.

In 1974, MSF claims it trained 15,000 students. Today it claims that “5,422,315 students have graduated from MSF RiderCourses since 1974. 400,000 motorcyclists enroll in our courses each year.”

We’re going to look at training in more than one entry. The first uses MSF documentation to show how the range section has changed from MSF’s Motorcycle Rider Course through the Basic RiderCourse.

The second entry will be as comprehensive a list and summary of twenty-one studies I’ve tracked down on training and licensing.

Thirty-something years of MSF basic training curriculum

MSF produced a chart for the state administrators who were invited to a private preview of the Basic RiderCourse in the summer of 2000. It outlines what was taught in the range portion in the Motorcycle Rider Course, the MRC: RSS and the then-new BRC.

Comparing the curriculums, the MRC taught 43 skills, the MRC:RSS taught 22 skills with 8 optional skills and the BRC taught 16 skills. The MRC tested  8 skills, the RSS tested 5 and the BRC tests 4.

The course went from 22 hours to 15 during these years.

Some of the skills listed separately in the MRC were clumped in the RSS and BRC so there’s not as much disparity as it appears—however, some of the skills are not included in the clumped skill exercises. A skill test for swerving was added in the RSS (and kept in the BRC) however swerving itself was taught in the MRC.

There are those who argue that some skills previously taught but not mentioned in the BRC portion of the chart are still taught—like the sharp turn. However, there is no portion of the BRC range cards that teach students how to do a sharp turn or a sharp turn from a stop. All we can go on is what is actually in the range cards and this MSF-produced chart to compare the curriculums.

Also, even if some of the skills are still taught, the shorter course length means they’re taught (and practiced) for a shorter time.

Please note that there are differences in the order of the BRC exercises between what MSF planned to do in the summer of 2000 and the current order.

1 Mount/Dismount Getting Familiar with the motorcycle Motorcycle Familiarization
2 Posture Moving the Motorcycle Using the Friction Zone
3 Controls Starting and Stopping the Engine Starting and Stopping Drill
4 Start/Stop Engine Riding in a Straight Line Shifting and Stopping
5 Walking Motorcycle Riding the Perimeter and Large


Adjusting Speed and Turning
6 Buddy Push Weaving (30’) Control-skills Practice
7 Friction Point Turning on Different Curves and

Weaving (20’)

Pressing to Initiate Lean
8 Straight Line Riding Riding Slowly Cornering
9 Rectangle Making Sharp Turns Matching Gears to Speed
10 Large Circles Shifting in a Straight Line Stopping Quickly
11 Medium Circles Shifting and Turning on Different


Limited-Space Maneuvers
12 Cone Weave (20’) Shifting and Making Sharp Turns Cornering Judgment
13 Sharp Turns Stopping with Both Brakes Negotiating Curves
14 Shifting in a Straight Line Stopping Quickly on Command Stopping Quickly in a Curve
15 Turning at Higher Speeds Stopping on a Curve Lane Change and Obstacles
16 Riding Slowly Level 1 Evaluation: 1.

1. Stalling


3. Sharp Turns

4. Stopping on Command.

Avoiding Hazards
17 Principles of Braking Gap Selection Skills Practice
18 Stopping at a Designated Point Turning from a Stop and

Changing Lanes

Skills Test:1. U-turns

2. Swerve

3. Quick Stop

4. Cornering

19 Figure 8-Turning and Adjusting Speed Controlling Rear-Wheel Skids
20 Turning in Tight Circles Stopping in the Shortest Distance (maximum braking)
21 Weaving Between Cones(20’ X 10’) Swerving to Avoid Obstacles
22 Shifting and Acceleratingin a Turn Stopping Quickly on a Curve
23 Stopping Quickly withBoth Brakes Selecting a Safe Turning Speed
24 Sharp Turns and Shifting Optional Exercises: Offset Weaving, Shifting

and Turning on Different Curves

and Weaving,

Stopping Quickly on Command,

Tight U-Turns

and Stop-and-Go, Counterbalancing

in Decreasing-Radius Turns,

Surmounting Obstacles

25 Simulated Traffic Situations Level Two Skills Test:

1. Cone Weave

2.Sharp Turns,

3. Quick Stop,

4. Turning Speed Selection

5. Quick Lane Change (swerving).

26 Passing
27 Turning Speed Adjustment
28 Circuit Training
29 Starting on a Hill
30 Stop and Go
31 Staggered Serpentine
32 One-hand controls
33 Engine Braking
34 Controlling Rear-Wheel Skids
35 Quick Stops
36 Stopping in a Curve
37 Riding on the Pegs
38 Crossing Obstacles
39 Countersteering
40 Quick Lane Change (swerve)
41 Carrying Passengers
42 Pre-Ride Inspection
43 Maintaining Your Motorcycle
Skill Test:


2.Shifting and Stop (in a circle),

3.Operating Controls (in a circle),

4. Straight Line Balance,

5. S-Turn,


7. Stopping,


Did MSF have more to do with Rider’s Edge than we thought?

September 10, 2009

You may have noticed in the prior entry on the simultaneous approval of both the Blast and Rider’s Edge that four of the nine classes (44%) that comprised the Rider’s Edge field-test were conducted in Albuquerque, NM.

The photo of what would be called the Buell Blast showing them in a training exercise was taken in mid-November and posted by The Single Cylinder Gazette on Wednesday, November 24, 1999.

However the range appears to be the very same one that appeared in the infamous marketing video sent out by the Motorcycle Safety Foundation to introduce its new curriculum iteration, the Basic RiderCourse.

That range was part of the New Mexico State Motorcycle Program. And the NMSMP was (and still is) administrated by MSF.

According to Michael Weiss, H-D Director of Business Development in that phone call to state program rider educators, David Smith and Frank Allen were the ones to contact for how Rider’s Edge instructors would “bleed over” in terms of Rider’s Edge and state programs.  Iow, Smith and Allen were the experts on Rider’s Edge instructors.

David Smith and Frank Allen were employees of MSF at the time the spy photo was taken and working on another field-test for MSF in Albuquerque at the same time—more on that in a moment. Smith has been the New Mexico state motorcycle safety program manager for several years even though, at least until a couple years ago, he also taught for Rider’s Edge in Albuquerque.

For many months after the Milwaukee SMSA, Rider’s Edge was a source of controversy and confusion in the rider education community. It might have reassured many in the rider ed community if they knew that  MSF-employed instructors were evaluating the course and the motorcycle.

It appears that MSF was much more deeply involved than suspected with the origins of Rider’s Edge if only because it was administrator of the NMSMP and employed Smith and Allen. Yet it made every effort to distance itself from Harley and Rider’s Edge and would only admit that Harley had permission to use MSF’s curriculum.

For example, Piper posted this on the MSF listserve on Aug 25, 2000: “Harley-Davidson and its Rider’s Edge New Rider Course do not intend to “compete” with existing rider training through the state programs….From the MSF viewpoint, we’re pleased at the effort Harley-Davidson is making to help us address the training capacity issue and to expose non-riders to motorcycling.”

There’s only one place where I discovered any admitted connection (apart from the curriculum use) between MSF and H-D and Rider’s Edge: An e-mail from Albert Thornton, then chair of the SMSA, with the subject head “Chat with Tim Buche on 29 July 1999” to Ron Shepard states “Tim also indicated that the MSF will be assisting Harley-Davidson “with some research this winter” and that HD will be laying [sic— I think he meant playing] a bigger role. (???????)”.  After Milwaukee, Thornton may not have needed the question marks.

The question is: why did TPTB at MSF and/or H-D feel it was important to maintain an illusion of a Chinese Wall when news of a deeper involvement with MSF may have reassured nervous instructors?

Was the Blast and Rider’s Edge testing also passed off as BRC field-testing?

What few readers may realize is that Albuquerque did over 70% of the 1999 BRC field-tests) for the Basic RiderCourse.[i] And those 19 sets of range exercises were conducted between the end of June and December 1999. Thirteen of them (over 68% of all the field tests) were taught by David Smith, his son Mark Smith and Frank Allen from August-December.

Iow, at the same time that Harley-Davidson was conducting the field-test for both Rider’s Edge curriculum and the Buell Blast as a training bike on an MSF range in Albuquerque with MSF employees, David Smith and Frank Allen they were simultaneously conducting field tests on MSF’s Basic RiderCourse.

That overlap of field-tests includes the same weekend, on the same range that the couple snapped the photo of the top-secret Buell used in training in Albuquerque.

David Smith, Mark Smith and Frank Allen in a memo to Ray Ochs dated May 19, 2000 refers to BRC field tests conducted “using normally scheduled classes”. It is assumed that the same policy was followed in the fall since the class sizes were similar.

According to MSF documentation that weekend, November 19-21, there were six students in the class that was held (the photo shows five though one may be out of sight on the right hand portion of the circle).

And, according to MSF, the course that was being conducted was the curriculum that it was just developing—the Basic RiderCourse—and not the MRC:RSS, which allegedly was the curriculum Rider’s Edge first adapted and field-tested—at least that was the impression the rider ed community was given.

Iow, at the exact time MSF claims it was testing the BRC, Harley claimed it was field-testing Rider’s Edge on the same range with the same instructors.

My friend Thor also claims the picture shows Exercise Six from the MRC: RSS. The photo, then, should be of an MRC:RSS class in motion mid-way through the first range session. He based that on the circle on the range and assumed that the students were riding the circle—however, there’s also yellow markings on the range that may or may not conform the to BRC.

But, according to BRC field-testing documentation, no MRC:RSS classes were conducted in Albuquerque in November at all. According to MSF, the only RSS that conducted in Albuquerque in the fall of 1999 was held December 17-19, 1999—almost a month after the pictures of the Blasts were taken and posted on the Internet.[ii]

Iow, if MSF—and Smith and Allen—are to be believed the photo would be of a BRC field test and not an RSS class. But Harley says it was field-testing Rider’s Edge at the same time. So either Harley’s Rider’s Edge was the BRC from the beginning or the instructors who taught the course were confused about which iteration they were teaching that weekend.

Whatever iteration, it was conducted on the Buell Blast.

And remember that Elisabeth Piper said in her February 9, 2000 post that the Blast was currently being tested for use in the New Mexico and Pennsylvania state programs? No training occurs in Pennsylvania in the winter—but it was occurring in NM. And remember, there were only three dealerships that had Buell Blasts at that point—and one of them was in Albuquerque.

But the only MSF training being done in Albuquerque in the winter of 2000 was the BRC field test. Does this mean that at least part of the BRC field test was conducted on Buell Blasts? It’s unknown.

[i] When MSF sought permission from the Oregon Department of Oregon to be approved as an alternate state program it submitted numerous documents including 74 separate supporting appendices. Number 34 is the RESLAB Component Feasibility Testing: Times and Mileage (Sept-Dec ’99) and lists all the field testing done in the latter part of 1999. Other appendices reveal that no field testing had been done in 1999 prior to September and only a very few field-tests had been done in the fall of 1998 (at Eastern Kentucky State University).

[ii] Interestingly, the one RSS conducted as a control group and to compare against the BRC had 33% more students than the average test BRC course.

Of apples and barrels and student errors

August 7, 2009

Recently, a dear friend told me about an incident that happened years ago: Every single one of the students got the exact same question wrong—the one about tailgaters. He realized he had made a joke about how to deal with tailgaters and  assumed the students knew it was a joke when the assigned student gave the correct answer. Clearly, they took his word over the handbook. My friend realized it was his fault and made sure they got the right information.

It is in this context that I saw a correlation with the cellphone story and the (understandably) appalled reaction in the media that 25 percent of all crashes are caused by distracted driving. What struck me was the surprise that those in the media seemed to have that the percentage was that high—and that was several years ago before the massive cellphone penetration we have today.

It struck me, of course, because braking and cornering errors in perception, judgment and skill happen at least 25% of the time in the end-of-course evaluations. Worse yet, some instructors claim that they occur up to 90% of the time.

It troubles me that instructors reported such high percentages were occurring but instructors hadn’t seemed to noticed until they were asked. Nor did any MSF curriculum rider education administer or independent owner I talked to know what the actual percentages were. Worse—to me—they, like the instructors, hadn’t realized they didn’t know. Iow—no one knows what is really going on in the skills evaluation other than how many pass and how many fail. And they aren’t even aware of how many of their students were just one point from failing or if that had changed from one iteration to another.

As someone who has taught for years whether on the university level ( Introduction to Scripture and Freshman Composition) in adult education (screenwriting) and downhill skiing and as a professional evaluator as a first and second round judge for the world’s premier screenwriting contest, that’s alarming. Especially since they are teaching skills that make the difference between life and death.  Because to me and other educators I know, if there’s patterns in student errors those can be evaluative tools we use to improve our performance and the course we’re teaching.

I found it curious that all the instructors presented it as if this was solely due to the students’ ability. Over the past five years many instructors have suggested—some half-joking and other’s deadly serious—that the supply of good students is drying up:

  • Many more complain that the character and personality of the students have changed—and that affects how they do. Iow, characteristics such as being contentious or demanding are related to their innate ability to handle a motorcycle.
  • Or they claim that students are more fearful and nervous as if it’s an enduring characteristic rather than something elicited by the course even though it was specifically designed to be relaxed and fun and low-pressure.
  • And they claim that more students are generally less able to catch on and perform the exercises. A very few postulate that fewer students have experience with any kind of clutch/throttle type machine.

The increasing number of crashes, injuries and lower scores on evaluations, would then be explained away by this alleged increase in inferior students: they didn’t have what it takes; or didn’t learn it; or were too nervous in the evaluation. Iow, it’s caused by a flaw in the student; they are the bad apples in the barrel of rider trainng.

Of course, there’s no hard evidence that the quality of students has diminished and diminished so rapidly. Nor do instructors and administrators think it’s even important to determine if the quality of students has in fact gone done or what should be done about it if it has.

No instructor or rider administrator appeared to consider that the poor performance both in life-critical errors and overall scores was due to the instructor’s ability—or inability—to teach (or coach if you prefer) effectively. This approach would make the instructors the bad apples. And some administrators say that the quality of instructors has gone down because demand has been so high that poorer instructors are passed whereas in the past they would’ve failed.

Of course—and who can blame instructors—they don’t want to consider they aren’t very good at teaching safe motorcycle operations.

Or perhaps the bad apple is the way instructors are trained coupled with the failure of programs to evaluate all instructors regularly (or ever) to ensure they are capable and effective in delivering the course content and at evaluating student progress during the course.

Another alternative is that it’s the curriculum that fails to convey the procedures, processes and so forth effectively and to allow enough time for students to obtain enough competence so that life-critical errors are uncommon. And if instructors are right and a lot more inept people are trying to learn to ride—then an attuned, well-designed curriculum would address the lower level of ability in order to bring them to a basic competency that isn’t one point away from an “F”.

If it’s either the instructor curriculum or the student curriculum that’s implicated, that’s a situational or system error—the barrel itself is producing bad apples. And who wants to look at that—it’s far easier to attribute poor performance to the student apples.

However, I would suggest that instructor and curriculum are almost certainly part of the students’ poor performance since those percentages and errors are found over a number of classes and thousands of students. After all, student quality varies class by class but the instructor and the curriculum remains the same. Consistent errors then are far more likely to be due to what remains consistently the same than what varies.

So let’s say that the instructors are correct and the quality of the students is going down. It’s very possible that fewer students have experience with any machine/vehicle that uses a manual clutch or some kind of power differential system—and a bicycle with gears isn’t even comparable. What if this was true? And it is true that almost all the deaths and critical injuries in rider ed involved misuse of the clutch.

Responsible, ethical and moral rider educators would address this by adjusting the curriculum to address that by increasing the time or exercises—or providing pre-bike exercises to become both familiar and comfortable with a clutch and throttle. They’d fix the barrel to help the apples.

And if it’s instructors in either training or quality, responsible, ethical and moral administrators would work hard to make the instructor apples better.

Braking & Cornering errors increased in MSF’s BRC curriculum

July 24, 2009

Study after study has shown improper braking and cornering errors are the leading rider-based errors in motorcycle crashes. I would maintain that there is no skill error that does not involve a corresponding cognitive error—primarily judgment but to a degree perception and interpretation since going out of control on a corner means a failure to correctly to perceive the corner correctly, interpret it, and correctly judge entry speed, line and lean. The same is true of braking—failure to perceive that braking is needed and when to apply the brakes and how much to apply the brakes.

Given that, the motorcycle licensing test and end-of-course evaluations are supposed to determine the (minimum) level of skill to do those actions in an effective and safe manner in traffic. So it really does matter what errors are frequent in testing and what percentage of students make them.

Many ways to get to some scores, only one way to get to others

If we look at the test scores more closely, we see some scores can only be reached if one or the other of two specific errors are made: stopping distance (1-point per foot over up to 10) or going too slowly around the corner errors are made:

In the RSS, the scores of 1, 2, 7, 11, 14 and 17.

In the BRC: 1, 2, 4, 7, 9, 12, 14, 17 and 19.

Iow, 30% of the possible scores from 1-20 in the RSS and 45% in the BRC are only possible if a critical error in both judgment and skill is involved.

16.14% of the RSS students got those scores.

And 27.04% of the BRC students.

Iow, the percentage of students who pass with guaranteed braking or slow cornering errors has increased in the BRC by 10%.

Of course, this doesn’t include other students who committed stopping distance or cornering errors that are masked by the other score configurations or those made stopping distance errors greater than 2 feet.

This means that without any explanation or excuse possible, more than 27% of all the BRC students who passed the test made stopping distance or braking errors that indicate lack of both  judgment and skill. And, remember, this only counts the scores that cannot be gotten any other way.

This is particularly significant since those errors were made after the students were warned that they will be required to stop suddenly or had practiced that exact same corner repeatedly and made those errors on smooth, debris-free, even pavement on a small motorcycle at very low-speeds. Iow, giving them every possible advantage—few of which they’d have in real traffic conditions—they still couldn’t manage to stop in time or turn at a high enough speed.

Additionally, the scores of 17 and 19 are in the bottom percentile of passing grades. This means these students were among the very same group that more than doubled between the RSS and BRC—very poor riders who still passed.

The data I have does not allow for other errors to be singled out—but this one alone is a critically sever error since it involves misjudgment of when to begin braking and how much pressure can be safely exerted on the brake lever and pedal as well as skill or involves misjudgment in entry speed, line and lean in a simple corner that has no possibility of on-coming traffic, debris, etc.

Setting aside whether the evaluations between the two iterations are equivalent, those percentages represent 357 people who definitely exceeded the stopping distance or cornered too slowly in the RSS and 736 who definitely did in the BRC. That’s more than double the number of people with only one-third more scores to account for those errors. That should alarm rider instructors, administrators—and the riding public.

Regardless of why these errors are committed–it points to a definite and negative change in the proficiency of those students who pass the course.

If students are too scared to go faster or stop faster,  as some instructors claim, that means the very reason MSF claimed the BRC was superior to the RSS–that the students would be more relaxed so they’d learn better–is invalidated by the test data: more students appear to be even more afraid.

It doesn’t matter if the BRC is as good as or worse than the RSS at this point—what matters is the lives of those who are trained and pass the course.

Bottom line is that motorcycle training under the BRC curriculum produces far more students that are seriously–and potentially lethally–deficient in skill, judgment and confidence.

I do suggest, however, it would be interesting to see the percentage of TEAM Oregon BRT students who make stopping distance errors and compare that to the BRC.

Comments on errors

July 18, 2009

Today’s entry comes from an instructor who sent this to me via e-mail:


“I tried to blog this however I kept getting timed out.

“Errors/mistakes in the Skill Evaluation:

“It could be construed that the first error made occurs when the student first tosses a leg over the seat of the bike and everything after that spirals downhill. Jokes aside what occurs in the skill evaluation is a mirror to what ‘skills’ weren’t learned leading up to it or a reversion to the habits that one arrived with at the start of the course.

“I’ll take each of the evaluations one at a time and then give my understanding of what went wrong.

“U-Turn box:

“To properly make the left and then right turn the skills learned in exercise two have to be considered. In that exercise the student is introduced to the throttle, clutch and both brakes. Many students ‘throw away the clutch’ and rely on the throttle in both parts two and three. Posture is also a learned skill and that is introduced in both the class room and again in exercise one and two. Another telling exercise is exercise four when the student is first introduced to the tight turn after the stop. If the student ‘blows’ that off then the slow speed skill of throttle and clutch is not reinforced. Generally the student will get the bike going and then rely on the throttle. A roll on will straighten the bike out and then send the students to parts unknown. Exercise six which is a summary exercise of skills learned to date is the giveaway as to how he student will do in exercise ten and then the first exercise of the skill evaluation.

“The swerve is a crock of sh-t in that the student when he/she exits the u-turn box is already headed in a direction which lessens the impact of a properly executed swerve. It is not evaluated however a roll off is most common. If engine braking is construed to be in fact a form of braking, then every student could be construed to be braking while swerving.

“The common error that I’ve found is ‘anticipation.’ In the Pa. program they took the time to paint an anticipation line before the actual timing line to cut down on the re-try. Again, if the student hasn’t learned how to stop the motorcycle which is taught in the third part of exercise two and then reinforced at every stop thereafter it will be magnified during the skill evaluation. Braking is a summary skill and then it is practiced each time the student comes to a stop regardless of where the stop is to occur. Over braking is common as the student waits until the last possible second then over brakes. this I believe is an overlap to the driving habits they have acquired over years of driving. The student will use the brakes on the bike as they do while driving their car. I believe there is a direct carry over.

“With regard to the last one which is cornering many RCs confuse throttle induced speed with utilizing the proper line through the turn. The premise is if the proper line is utilized which is ‘outside, inside, and then outside’ one will find that its use is the shortest distance through the turn. I’ve timed it with another RC riding and then myself riding. Very little roll on is required if the line is correct. The error stems from many students not using the technique I dare say that the vast majority of students go to the inside of the turn way too early which causes the roll off in mid turn because they literally run out of room. Again the higher speed cornering is learned in exercise five, seven, touched on in eight, eleven, twelve, thirteen, and implied in fourteen, fifteen and again in sixteen. The latter two exercises the implication comes when the student makes the turn to return to end of the line.

“It is rare where a student with decent skills coming into the evaluation will brain fart although it happens. It is also rare for a student to have an ‘ah ha’ moment and do spectacular on the skill evaluation. The skill evaluation magnifies the unlearned skills. The sad part of it all are those unlearned skills will be further honed on the street because the mistakes made on the riding range will surely be made on the street because that is what the student learned to do so they can’t immediately change when their ticket gets punched.

“The whole premise is wrong so the errors start before class. The premise these days is the license begets skill versus skill begetting a license.

“Use what you would like…oh yes feel free to disagree.”

OK-that’s his take. So what do you all think–agree, disagree–have something to add or ???

Question for instructors

July 13, 2009

What errors do you see most often in the end-of-course-evaluations? And, by error, what percentage of students (per class) make that error?

Easy to pass poor riders, difficult to fail them?

July 9, 2009

I have the pass-fail data (though not the raw data) from two different years in one state program. Altogether it gives us a sample size of 5,619 which should be large enough to avoid the errors of a small sample size whether looking at both curriculums together or each separately. And when I looked at it after a couple years, I noticed some oddities that I think the rider ed community needs to consider.

What I noticed is particularly helpful for rider educators who want to see how effective one curriculum may be over another—and especially what kind of errors may be more associated with one curriculum over another.

As you see, it includes both MRC: RSS and BRC skills test results. This is not to compare the efficacy or difficulty of either curriculum but to examine oddities in the data—and to examine what it means to pass or fail the end of course evaluations:

Students Tested



Passed Riding



Failed Riding




Students Tested



Passed Riding



Failed Riding



When we graph the test scores we see an unusual sawtooth appearance—almost like a drunkard’s walk. More on this later. But we also find that only 2.59% more students passed and failed the BRC students than the RSS:

In this way, it appears the courses—and evaluations—are equivalent. Some could say, as a result, that the BRC hasn’t been “dumbed down”. But it’s wise to look more carefully at the results before doing that:

What is clearly different between the two is that the BRC has more students that almost fail it than the RSS did:, 19. 47% of all BRC students and 13.57% of all RSS students got scores of between 18-20.

So, while it appears that the number of students who passed one curriculum or another is equivalent, there’s a statistically significant difference between the iterations in the percentage of those who just barely squeaked by. Iow, the BRC sent  almost 6% more poor riders

This becomes even more significant when we look at the very last possible point that a student can pass—a score of 20:

106 RSS students or 4.13% and 268 BRC students or 8.77% got an even 20.

Iow, more than twice the percentage of BRC students got the equivalent of a D- than  RSS students. Meaning more than twice the number of very poor riders were sent out into an increasingly complex and dangerous traffic environment with a driver’s license-waiver in their hands.

Is this just an oddity or does it indicate something about the two iterations?

The skills tests

For those who aren’t familiar with the end-of-course evaluations in both iterations, read on. (For those who are, skip to “What a difference one point makes”.)

The RSS skill test involved 5 scored evaluations: Sharp (90-degree) Turns, Cone Weave, Quick Stop, Turning Speed Selection (cornering), and Quick Lane Change (swerving).

The BRC skill test involves 4 evaluations: U-turn, Swerve, Quick Stop and Cornering.

The Sharp Turn and Cone Weave were dropped entirely and replaced with the U-Turn, the configuration of the corner was changed and the timing zone and timing chart were changed in the Quick Stop. Additionally, the U-turn is not a life-critical skill, so 40% fewer critical skills are tested in the BRC.

So let’s keep a running tally: 40% fewer critical skills tested in the BRC and 6% more poor riders passed with the last passing score: Iow, it’s possible that the poor riders may be even poorer than suspected.

A passing score is ≤20 and a failing score is ≥21.

In both iterations:

Evaluations are given separately but consecutively and

Each skill was/is scored separately.

The student had/has two opportunities to take each test and

Points are not added until after the second run (therefore, if they make two errors in the first trial and another in the second, only the second error is counted).

In both iterations, points are assessed for a variety of errors but each evaluation had/has a maximum number of points that can be assessed:

In the RSS the maximum was identical to the Alt-MOST used at the DMV—10 points max in each of the 5 evaluations.

In the BRC, the U-Turn has a maximum of 8 points and the other three evaluations have a maximum of 15 points—or a 50% increase in allowable points for each of the tested skills that comprise 80% of the evaluation.

Both tests used a 1, 3, 5, 10 and 15-point scoring system—but they used that system somewhat differently. In the RSS, 3-points could be assessed in 4 of the 5 evaluations—there were 12 errors that scored 3 points. In the BRC, there’s only one skill (U-Turn) that has two possible errors each worth 3 points each (if done once).

The Quick Stop—a critical life-saving skill—also assesses 1 point for every foot beyond the stopping point that the student travels up to a maximum of 10’—but this is not in addition to the maximum points but includes it.

In 3 out of the four evaluations, students must demonstrate the required skill at  between 12 and 18 mph. In the Quick Stop evaluation, for example, if they go under 12 mph, they can collect 5 points—but if they make the stopping distance at speeds higher than 18 mph, no points are assessed. In Cornering, they are supposed to get up to about 20 mph prior to slowing for the corner but the student will not lose points if they don’t go that fast.

The highest score possible in either iteration would be 53 (15+15+15+8=53)—though the actual number of points accrued can be much higher.

What a difference one point makes

One of the obvious things about the above graph is the huge difference between the last possible passing score of 20 and the exceedingly few students that got the first failing score of 21 in both iterations—but this is more far more evident in the BRC.

In fact, while 8.77% of BRC students got 20 points only 1.08% got just one point more and failed the course while 4.13% RSS students got 20 points only 1.32% got 21 points.

This is another significant change between the two iterations. While the percentage of those who barely scrape by more than doubled, the percentage of those who just failed changed from about 4 to 1 to 8 to 1.

And yet, the overall percentage of those who passed didn’t change in a statistically significant manner from one iteration to another.

Iow, while it appears by overall pass scores that the two curriculums were equivalent, they weren’t—40% fewer critical skills were tested and 6% more got the last possible score that enabled them to pass and the number of people who just failed sank like a stone. And that’s significant any way you look at it.

It would be interesting to see how other states measure up if their data was examined and see if this bulge of barely passed and dearth of just failed also exists–and if this patterns is also found in those states that use TEAM Oregon’s BRT.

Tomorrow’s entry reveals that more than 27% of BRC students who pass the test make the exact same potentially lethal error compared to just over 16% of RSS students—and that’s a dangerous change between the two iterations.