Tuesday, May 13, 2008

Piling on the exams

A report from the news that voices concern that with all these exams pupils are only being taught to pass them rather then gain an underlying knowledge of the subject. Um duh, what did anyone expect when they started to use the same results to judge teachers and schools by?

Let's say I'm teaching Jimmy Genius and Johnny Dumbo. Jimmy grasps the basics that he needs to pass the exams quickly; Johnny on the other hand is playing catch-up. So a teacher has a choice - teach each section including the underlying knowledge and have to repeat it until Johnny has grasped even the basics with the result that Jimmy gets bored and you might not get to finish all the sections before the exams; or teach only the basics of each section enough to get through the exams, stretch those points until Johnny has grasped them and only after all the sections are complete go back to fill in the rest of the knowledge.

Okay I'm painting it a bit black and white here, but if the latter scenario results in both Jimmy and Johnny getting a passing grade and the former a fail for Johnny, but a better grade for Jimmy; and then you're told your job and school are being judged on the number of passes then what would you do?

I'm not saying this is the way things are done, I hope they're not, but it has to be recognised that if you take a particular measure and use it as a standard of performance then there's a tendency to shift priorities into improving that measure regardless of whether doing so actually increases the actual performance that the measurement was instigated for in the first place.

Case in point - Call centre operators can be judged by the number of calls they deal with, they may not be judged by the number of calls they resolve. So the priority here shifts to the length of the call - can I get this person off the line in the shortest amount of time possible so as to be available to answer another call. Does this help the customer, nope.

In the case of exams who can even pretend to be shocked or amazed that the priority may have shifted to increasing the pass rate rather then educate; show me that person and I'll show you someone who doesn't have a basic grip on reality (so that seems to include most politicians and the media)

3 comments:

Anonymous said...

A radical reform of education of removing ageism from schools would benefit everybody. Jimmy Genius and Johnny Dumbo should progress through the education system based on their ability not their age _ just as every employee throughout every organisation progresses through their corporate system. Johnny Dumbo would have all the time he needed to learn instead of scrapping his current curriculum and starting again just because it's September. Now he is disillusioned and disinterested because he has to start all over again in the new year. Poor Johnny, he only needed a few more months and he would have grasped it!

Mean while Jimmy is becoming disillusioned and disinterested because he has to wait till September when he is a year older for a fresh curriculum while he has already understood and now sick and tired of going over the same old thing again and again. Why not let Jimmy do exams earlier, once he has passed some subjects he could concentrate fully on others (same goes for Alan Average).

Anyway, I must get some sleep. Tomorrow at work I'm going to go straight up to my boss and ask for a raise because this year I will be one year older than last year! :-)

Anonymous said...

This problem of teaching for the average pupil crops up everywhere. Once, ‘the’ solution was selective schools; now that selective schools are elitist, it is setting children based on ability.

Even in Cambridge, the same problem is noticeable. There is much less spread in students' abilities because of selection, and there is a lot of self-study and small-group teaching, so the problem is much less than in schools, but it is still there. Any teaching system where multiple students have to experience the same information rate will have the same problem.

This is all a bit of a tangent to your main point, which was targets. Really, all targets should be defined by lawyers or computer scientists — people with expertise in setting rules that can't be gamed or loopholed — rather than by managers as they usually are. When I worked for Autonomy one of their big sales anecdotes for demonstrating how much money they could save customers was to do with a company whose business involved call centres. Their performance metric was number of calls answered, and took no account at all of the nature of the call. It wasn't until they installed Autonomy's call-centre analysis program that they spotted that one of their employees, who'd recently been awarded a fat, performance-related bonus, had been gaming the system by immediately hanging up on callers.

But then, Asimov spent nearly his whole writing career trying to explain that representing desirable behaviour in a system of simple rules is infeasible: it's not surprising that not everyone has got the point yet.

FlipC said...

I think progress by ability would work provided it was split by subject.

The trouble with having entire 'elite' schools is/was that a person who is a whizz at Maths may be more backward in English or vice versa, unless they transfer back to the 'dumb' school for such lessons then the elite school still has to cater to a wide of abilities; arguably a wider range.

The tiering in schools does help in that you're only restricted to the least able in your class rather then year, but while I was at school I never saw anybody up- or down-graded. While this may be a statistically quirk I can't help but think that doing so would have screwed-up the timetables or placed too many or too few pupils in one class.

Maybe I'm just bitter as I was in the top tier of every subject in Middle School and continued as such into High School except for English where I went down a grade :-P

Anyway, as Dan points out, flaws in the education system wasn't my main point and I'm not surprised to hear about the hang-up guy.

My favourite (hopefully fictional) story was the factory that had a 5% quality failure rate and was told to improve or face closure. They got the failure rate down not by improving the quality of work, but by downgrading the testing process.