Of apples and barrels and student errors

Recently, a dear friend told me about an incident that happened years ago: Every single one of the students got the exact same question wrong—the one about tailgaters. He realized he had made a joke about how to deal with tailgaters and  assumed the students knew it was a joke when the assigned student gave the correct answer. Clearly, they took his word over the handbook. My friend realized it was his fault and made sure they got the right information.

It is in this context that I saw a correlation with the cellphone story and the (understandably) appalled reaction in the media that 25 percent of all crashes are caused by distracted driving. What struck me was the surprise that those in the media seemed to have that the percentage was that high—and that was several years ago before the massive cellphone penetration we have today.

It struck me, of course, because braking and cornering errors in perception, judgment and skill happen at least 25% of the time in the end-of-course evaluations. Worse yet, some instructors claim that they occur up to 90% of the time.

It troubles me that instructors reported such high percentages were occurring but instructors hadn’t seemed to noticed until they were asked. Nor did any MSF curriculum rider education administer or independent owner I talked to know what the actual percentages were. Worse—to me—they, like the instructors, hadn’t realized they didn’t know. Iow—no one knows what is really going on in the skills evaluation other than how many pass and how many fail. And they aren’t even aware of how many of their students were just one point from failing or if that had changed from one iteration to another.

As someone who has taught for years whether on the university level ( Introduction to Scripture and Freshman Composition) in adult education (screenwriting) and downhill skiing and as a professional evaluator as a first and second round judge for the world’s premier screenwriting contest, that’s alarming. Especially since they are teaching skills that make the difference between life and death.  Because to me and other educators I know, if there’s patterns in student errors those can be evaluative tools we use to improve our performance and the course we’re teaching.

I found it curious that all the instructors presented it as if this was solely due to the students’ ability. Over the past five years many instructors have suggested—some half-joking and other’s deadly serious—that the supply of good students is drying up:

  • Many more complain that the character and personality of the students have changed—and that affects how they do. Iow, characteristics such as being contentious or demanding are related to their innate ability to handle a motorcycle.
  • Or they claim that students are more fearful and nervous as if it’s an enduring characteristic rather than something elicited by the course even though it was specifically designed to be relaxed and fun and low-pressure.
  • And they claim that more students are generally less able to catch on and perform the exercises. A very few postulate that fewer students have experience with any kind of clutch/throttle type machine.

The increasing number of crashes, injuries and lower scores on evaluations, would then be explained away by this alleged increase in inferior students: they didn’t have what it takes; or didn’t learn it; or were too nervous in the evaluation. Iow, it’s caused by a flaw in the student; they are the bad apples in the barrel of rider trainng.

Of course, there’s no hard evidence that the quality of students has diminished and diminished so rapidly. Nor do instructors and administrators think it’s even important to determine if the quality of students has in fact gone done or what should be done about it if it has.

No instructor or rider administrator appeared to consider that the poor performance both in life-critical errors and overall scores was due to the instructor’s ability—or inability—to teach (or coach if you prefer) effectively. This approach would make the instructors the bad apples. And some administrators say that the quality of instructors has gone down because demand has been so high that poorer instructors are passed whereas in the past they would’ve failed.

Of course—and who can blame instructors—they don’t want to consider they aren’t very good at teaching safe motorcycle operations.

Or perhaps the bad apple is the way instructors are trained coupled with the failure of programs to evaluate all instructors regularly (or ever) to ensure they are capable and effective in delivering the course content and at evaluating student progress during the course.

Another alternative is that it’s the curriculum that fails to convey the procedures, processes and so forth effectively and to allow enough time for students to obtain enough competence so that life-critical errors are uncommon. And if instructors are right and a lot more inept people are trying to learn to ride—then an attuned, well-designed curriculum would address the lower level of ability in order to bring them to a basic competency that isn’t one point away from an “F”.

If it’s either the instructor curriculum or the student curriculum that’s implicated, that’s a situational or system error—the barrel itself is producing bad apples. And who wants to look at that—it’s far easier to attribute poor performance to the student apples.

However, I would suggest that instructor and curriculum are almost certainly part of the students’ poor performance since those percentages and errors are found over a number of classes and thousands of students. After all, student quality varies class by class but the instructor and the curriculum remains the same. Consistent errors then are far more likely to be due to what remains consistently the same than what varies.

So let’s say that the instructors are correct and the quality of the students is going down. It’s very possible that fewer students have experience with any machine/vehicle that uses a manual clutch or some kind of power differential system—and a bicycle with gears isn’t even comparable. What if this was true? And it is true that almost all the deaths and critical injuries in rider ed involved misuse of the clutch.

Responsible, ethical and moral rider educators would address this by adjusting the curriculum to address that by increasing the time or exercises—or providing pre-bike exercises to become both familiar and comfortable with a clutch and throttle. They’d fix the barrel to help the apples.

And if it’s instructors in either training or quality, responsible, ethical and moral administrators would work hard to make the instructor apples better.

Advertisements
Explore posts in the same categories: Instructors, Motorcycle Safety Foundation, Motorcycle Training, State Motorcycle Safety Programs

20 Comments on “Of apples and barrels and student errors”

  1. aidanspa Says:

    W –

    A very well-written and thought provoking post, as always. You make some interesting points. Not to overstate the obvious, but don’t you agree that in order to effect change in student errors (whatever combination of causes may be responsible) that there must first be agreement by those in a position to make changes that a problem exists? Unless and until that happens this is an exercise. A good one, but an exercise nonetheless.

    Students are NOT responsible for the failure of any educational program, whether it be a BRC or high school algebra or pottery-making. Putting the blame at the students’ feet is lazy and irresponsible thinking.

    If an educational enterprise has an acknowledged & consistent failing it is the responsibility of the administration to determine the cause and correct it. The efficient way IMO to determine causation is start at the student point of contact and work backwards. Acknowledge that the students are not the cause. Begin with the teachers and their methodology, attitude, ethics, etc. Correct any problems that are found and then check to see if the students are still failing in the same area. If so, then examine the teacher instructors. Fix any problems found and check the student results. If the problem still exists then the administrators are next to be examined. Self-correct (?) any problems found and if the student failing still exists then check the curriculum for faults. Be willing and able to change the curriculum if necessary and if that means change it from the ground up then so be it.

    All well and good, but first the problem needs to be acknowledged and there must be a willingness and determination by those in charge to fix it.

  2. wmoon Says:

    And that’s the problem, isn’t it, Aidanspa? There’s a collective unspoken agreement on the part of MSF, program administrators and instructors that there is no problem–or no significant problem exists and a tacit acceptance that not only is it moral and responsible to give driver license-waivers to students who perform life-critical errors but that an increasng number of deaths and critical injuries are acceptable outcomes in training.

    I totally agree with how you suggest such results could be both determined and dealt with–but your last line says it all. IIRC, it was Matthew Henry who said (paraphrased) that there’s none so blind as those who refuse to see and none so deaf as those who refuse to see.

  3. Gunslinger Says:

    Wendy,

    Bad Apples?

    Well as I mentioned to you awhile ago I was summarily told that the curriculum teaches the student to ride and not the Rider Coach/Instructor so michael Jackson lied ‘…one bad apple does spoil the whole bunch -girl…’

  4. CaptCrash Says:

    One of the truly brutal challenges a cirriculum provider faces is getting good cirriculum written and then finding instructors to pass that information along unedited…

    Probably the worst thing an administrator can here is an RC in class saying things like:

    “Well, the BOOK says…BUT here’s the real deal…”

    or

    “The BOOK doesn’t cover that but here’s what to do…”

  5. wmoon Says:

    CaptCrash–do you teach the BRC? The whole premise of the BRC is that the instructor is not limited to what the handbook/range cards say. That they’re supposed to deal with the students in the room/range and translate the material in a way that meets the students’ learning styles. MSF has repeatedly said that having the questions read and the answers read out of the handbook is only one option–the easiest one. That was Och’s definition of a good curriculum–that the instructor could put a cone on his head, do a soft shoe routine or say it in his/her own words–and that’s what you saw on the MSF curricular list: instructors arguing for years about how the exercises are to be taught, whether the cones can be moved or where, etc. etc.

    Your definition of “worst thing” then is the BRC–and the corollary then would be the opposite–where instructors say exactly what they’re supposed to say–and that, dear boy, was the much-maligned MRC: RSS…
    W.

  6. CaptCrash Says:

    One word…or two: TOMS:BRT

  7. Jeff Brenton Says:

    MSF has repeatedly said that having the questions read and the answers read out of the handbook is only one option–the easiest one.

    It’s also the only one that can be taught in the limited time available in RiderCoach Prep, and pretty much the only one that fits in the 15-hour BRC schedule. And a lot of instructors refuse to consider anything else, because the RiderCoach Guide is “THE authority”, and it highlights this technique with words like, “The RiderCoach WILL …”

    Use of techniques that ask the students to discuss and decide what is important in the material (with guidance) take longer. Introducing significant student interaction beyond answering questions is, in my opinion, key to keeping the students engaged and learning. The surprising thing is that, yes, it takes longer, but the students aren’t bored by it, if the instructor is on his or her game. And yes, the ability to go beyond what is in the book is important, but reigning yourself (and sometimes the students) in to keep the things moving is important, too.

    A lot of instructors pride themselves on keeping the boredom to a minimum by getting the students out of the classroom and onto the bikes as soon as possible. “They’re adults, they know this stuff already, get it over and done with ASAP.” “The schedule is for 5 hours in the classroom, but you don’t HAVE to use ALL of it.”

    This leads to students feeling rushed, and they start to discount the “head game” as simply something that has to be covered before the “real learning” begins. A RiderCoach that rushes through the mental portion of the curriculum is effectively dismissing the concept of safe riding being primarily an “eyes and mind” thing.

  8. wmoon Says:

    Ah…well there you go. Interestingly, as more students are dying from medical conditions after they’ve been ejected from their bikes, MSF has begun requiring instructors to not deviate from the cards or books–iow, returning to the RSS scheme. This indicates MSF does know that the deaths are caused by the curriculum–a curriculum that allowed instructors free play.
    W.

  9. wmoon Says:

    I can think of nothing more boring than the Q&A as taught by MSF–I can read and comprehend by myself at home far faster than in the form it’s practiced so no wonder it is boring. But, ironically, MSF’s new online perception “facts” part of the perception test states, “Safe riding is a skill more of the eyes and mind (mental) than the hands and feet (physical).” And yet the BRC handbook contains far less material and information on the mental skills needed for safe riding–and what’s there is fairly garbled.

    The utterly boring way MSF teaches how to teach what it says is a more essential skill to safe riding is exactly the kind of thing that reveals not only MSF’s unethical and irresponsible dominance of rider training but the slavish unthinking behavior of administrators and instructors that will not do what’s best for the students but what’s best for them to keep their jobs.

  10. CaptCrash Says:

    Another issue that’s related is “assessment” or “checking for understanding”. In a Guided Discussion you have to make sure each student ‘gets it’ which can be very tricky. One of the reasons for things like the bullet points used in the BRT is to make sure each concept is presented ‘by the book’ and then you need to make sure everyone is exposed to it.

    Does that mean everyone GETS it? That’s what the end of course assessment is for.

    Does saying it right and then assessing it guarantee everyone RETAINS and APPLIES it? Well, that’s the real challenge isn’t it?

  11. aidanspa Says:

    Capt – Your point is taken regarding retention and application of the curriculum being an unknown, assuming that the presentation and assessment is in order.

    Tell me who is responsible for knowing whether the material begin presented and checked for understanding is being retained and applied? The students? If we were talking about an English Lit class or a course in basket weaving, I would argue that it doesn’t really matter in the long run…but since we are talking about life & death & everything in between, I would submit that it is the responsibility of the course administration.

    Is the curriculum sound, effective, and being retained? The only way to know is by peer-reviewed longitudinal efficacy studies. Never been done to the best of my knowledge. Why not? I would think that if student safety was a priority for those in charge of the program that it would be a big selling point to say that the course has been proven safe and effective by independent research.

    On the other hand, there is no real need for selling points and efficacy studies if there is no competition. Outside of the students or perhaps their next of kin, does it matter (in the big net-net scheme of things) if anything is retained or applied?

  12. wmoon Says:

    Aidanspa,
    There isn’t efficacy studies or retention studies because program adminstrators and MSF have run rider ed as if it wasn’t a matter of life or death–simply a way for the manufacturers to look good and a bunch of men (mostly) who have approached this in nothing that resembles any kind of professionalism. Sorry if that offends some–but it’s the truth.

  13. Gunslinger Says:

    Wendy,

    Okay so the curriculum teaches the students. As a way of checking if the students connected with the classroom portion of the BRC I will ask questions on the range that reach back to what was covered in the classroom. Things like someone tell me the acronim for starting the bike. Time and time again I’ve been met by blank stares. I’ll also receive blank stares when I ask the group the four steps in turning. So to say that the curriculum teaches the students to ride is a great falacy because if a given student can’t retain information garnered some 12-24 hours prior it is quite the sad state of affairs.

  14. wmoon Says:

    Gee, Gunslinger–given what you say, what makes any instructor think that students remember what they learned on the range 12-24 hrs later?
    W.

  15. vstromer Says:

    Regarding conducting the BRC classroom by using the 126 study questions in the back of the BRC workbook…

    I would be ripped to shreds during a QA visit by a mentor or chief instructor if all I did was use the 126 study questions. We are encouraged to facilitate the classroom using learner-centered adult learning techniques. There are many, many ways to do this.

    Using the study questions is just one very poor and boring way to facilitate the classroom. IMHO, any instructor who only uses the study questions is taking the easy and lazy way out. After all, almost every bold study question is covered during the videos.

    At the beginning of the class I tell the students that there are study questions in the back of the book, and it is their responsibility to know the answers to the questions. “After we cover the material, if you don’t know an answer to one of the questions feel free to ask.” After that, I never mention or reference the questions again, unless a student brings it up for discussion, in which case I let one of the other students provide the answer.

    >(using the study questions is) pretty much the only one that fits in the 15-hour BRC schedule

    My classroom typically takes 5.5 to 6 hours (over two days), including the written evaluation.

  16. Jeff Brenton Says:

    My classroom typically takes 5.5 to 6 hours (over two days), including the written evaluation.

    Which is part of my point. The schedule for our classes (Illinois) has been 7-8 hours for classroom, spread over three days. It’s fairly relaxed. We have 10 minute breaks about every hour, not counted in that time.

    The “official” MSF schedule is 5 hours, inclusive of breaks, or about 4:20 of “face time”, as it is referred to around here. Take out the approximate hour that the videos run, paperwork time, written evaluation, etc., and you have around 2 and a half hours of discussion time to cover the book.

    Oh, the students didn’t get the book before class, so they haven’t read it yet? Scratch half of that, because not all students can read quickly, especially if you’re expecting them to read it well enough to find the important concepts.

    Personally, I think the recommended methods for “facilitating” a BRC are well suited for a longer classroom portion, and yes, 5.5-6 hours is a realistic minimum, in my opinion. Running it shorter discourages using learner-centered methods, and retention beyond the completion of the written test suffers. And what good is getting a perfect score on the written test, if you can’t remember how to apply a strategy (ANY strategy, not just SEE or SIPDE) in real time two days later?

    As for the study questions… in my classes, the students may find the questions on their own, but I don’t tell them about them until the end of the second classroom session. “Use them to see how much you’ve learned. If you have a problem answering one, go through that section and review.” They serve the purpose of a final “training aid”, reinforcing what they’ve already learned, rather than being the basis for it.

  17. wmoon Says:

    Jeff, And how does your region test for retention of key concepts? And how does it link what’s learned on the range with what is learned in the classroom?

  18. Jeff Brenton Says:

    I don’t know that the region does any testing on a regular basis. There may be some going on behind the scenes, because there have been a few pilot tests going on in Illinois, as you are no doubt aware. The region I work for was not active in the BRT pilot, but there have been some others. As instructors, we are asked about our experiences during such classes, not what happens after.

    However, when I encounter students weeks and months after class, we have talked about things we covered in class. None have regurgitated what I would call “bad info”. Several have discussed “refined info” they’ve learned subsequent to class, which means they were interested enough to continue learning beyond the basics. It’s especially gratifying when one attributes their ability to avoid a problem to something we discussed in class, though.

    Online, I have encountered students from programs around the country. Some have trouble recalling what they learned in class the previous weekend. Others vividly remember what they heard. I don’t know the classroom schedule used in other programs, but, when asked, many of the “no recall” group went to classes where the 5-hour schedule was used.

    That’s all purely anecdotal, of course.

    Maybe the key is not to stress “data”, but “process”. Teaching via reliance upon answering specific questions does not help when the questions change. When you have the time, you can work with the students on how to reach the answer (what to do in THIS situation), instead of “What is the answer to question 56?”.

    By the way, MSF has a tool at their disposal that could be used to gauge retention, if it were used as such. They have a “sample test”, which covers the same material as the BRC written, using different questions. If you could convince the students to take it say, 6 weeks after their class, and compare their results with what they received on the official test, it could be an additional data point for the discussion…

  19. irondad Says:

    I’m certainly not the ultimate expert in all this. There’s been some great discussion here. Mostly what I wanted to add is the simple way I look at things out here as an instructor.

    The classroom information we teach here is what has been decided is “need to know” stuff. Who’s responsibility is it to learn and retain? Obviously, it’s the student’s responsibility. However, it’s my responsibility as an instructor to help the students take ownership of it. That’s the beauty of guided discussion.

    It’s not “this is my information and I’m going to give it to you”. It’s “This is yours. Taste, discover, explore, and I’ll guide you on the journey. I’ll know how you’re doing because I’m going to ask questions. Each and every one of you will be involved in the process. I’ll make sure of it by asking direct questions if I have to. If I find that my current approach isn’t working, I’ll know to change it up because I’ll be constantly tuned into your journey.”

    I figure that this is the only time the students will probably be in touch with a professional trainer. It’s my responsbility to make it as meaningful and beneficial as possible.

    Do students retain it? I’ve lost count of how many times students come back to me and say they heard my voice in their head when they faced a tricky situation. I tell them they can keep using my voice but they need to work on getting their own voice active.

    It’s the student’s job to learn and it’s my responsibility to facilitate that process.

    Thanks for letting me ramble, Wendy!

  20. aidanspa Says:

    Irondad – Bravo! Team Oregon and its students are fortunate to have you. Your passion and commitment to excellence inspires and compels your students to be the best students they can be. That is what it’s all about. My .02.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: