Friday, July 6, 2012

Measure for measure

Education departments publish league tables of schools' exam pass rates

The intent: To encourage schools to improve standards of education so more children achieve passing grades.

The reality: Parents start pulling strings to get their kids into the "good" schools. Schools in neighborhoods where their intake is inherently unmotivated suffer, no matter how good the teaching. Schools in a position to do so start being more selective in who they will accept. School teachers only allow children to sit exams who they think will pass. Instead of raising standards of education, many kids are actively denied the opportunity to stretch themselves.

IT help desk is outsourced, and the vendor is paid by the number of incident tickets resolved.

The intent: A simple measure that costs little to implement and results in fair payment for the volume of work performed.

The reality: The vendor is motivated to inflate the number of tickets opened. Every time a client calls to inquire on the status of a ticket already open - bang, you get a new ticket to log the call which is then promptly closed. Also tickets get closed whether or not they are properly resolved, and the when the client complains - you guessed it - a new ticket is opened. Result: frustrated clients and poor customer service.

Government starts measuring hospital waiting lists

The intent: To benefit patients by reducing the wait time for operations.

The reality: Doctors find ingenious ways to avoid putting people onto waiting lists in the first place. Patients miss out on badly-needed surgery as they jump endless hoops of referrals and tests.

People are elected into positions of power based on the number of votes they can garner

The intent: To use the collective wisdom of an educated electorate to select those most fit to govern.

The reality: Gaining votes becomes an end in itself, in a game which favors the most ruthless lying slimeballs imaginable. Truth is an early casualty, and actual fitness to govern doesn't get a look in.

Does anyone see a theme emerging?

In all these examples, and many more that I'm sure you could come up with, someone is measuring something that sounds perfectly reasonable, and with the noblest of intentions, but the outcome is far removed from the intent.

The moral of the story is - beware what you choose to measure, because measurement distorts behavior, often in unpredictable ways.

You think you are pushing people in one direction, but your act of measurement is in fact producing lots of forces in directions you never envisaged - and the system you are measuring will always move in the direction of least resistance. This will rarely be the outcome you expected.

This insight is part of the line of thinking in my "Citizens first" series of posts, which is all about putting citizens back in the frame as majority beneficiaries in our own society.

In order to achieve this goal, we need to find new ways to measure success - other than by the unfettered acquisition of money. Ways that will genuinely motivate individuals, corporations, and governments to think of the ordinary citizen first and foremost, and which will yield the outcomes we want to see.


  1. Isn't it strange that we can see this so clearly but legislators and those in positions of power cannot see it!
    Click here for Bazza’s Blog ‘To Discover Ice’

  2. Hi Bazza! The difficulty is that measurement is more problematic than those folks realize. They may genuinely believe they're doing the right thing, but when it doesn't work out it's very difficult to backtrack. Society does not reward people who admit they got it wrong and want to try something else. Oh, look, another form of measurement :)

  3. Q: Does anyone see a theme emerging?

    A: Short-sighted success. Most of these 'measurments' are only one layer deep and one term long. Real success is measured in the productivity and contributions of lifetimes. Unfortunately, few take the long view and instant gratification is a hard habit to kick.

  4. Sounds very similar to what we see happening here more and more. Especially in Education

  5. Fairchild: Interesting viewpoint. Personally, I think short-sightedness isn't a primary driver so much as a common symptom of our current system. Our measurements, as you point out, have to kick in during the course of a short term, with no reward for long term thinking.

    Mynx: I think much of what I describe is common across what we like to call the developed world. I could cite many more examples, but it gets rather depressing reading :) I'm not trying to depress people, but lead them to the idea that there could be a way out...more to come in future posts.

  6. If only people would take the time to be more invested and involved in decision making that can impact government, education, medical issues, or anything else that can have a major impact on society.

    Regrettably, many don’t want to be bothered and simply depend on the thoughts and decisions of others, whether those decision makers are legislators, politicians, big business, lobbyists with agendas, etc.

    If more people took the time to truly think about intent and ultimate outcomes, things would be much improved. An excellent, thought provoking post. :)

  7. Hi Ian,
    Some powerful and emotive stuff you allude to here. Of course, what you so articulately note here, has many parallels to what happens in Britain and many other places. It's time for people to be more proactive and demand a real say in what the government tries to implement, often by stealth measures.
    My best to you,

  8. Super Earthling, I think many people feel powerless - it takes a massive and disruptive effort to effect any kind of change. And when you look at the revolutions in history, they never achieved anything lasting other than to swap one corrupt leadership for another. My theme is leading up to hints of ways to change the nature of the game itself, but by small measures that don't involve lining people up against walls and shooting them :)

    Gary, demanding a say is one thing, but motivating people to have their say and to make good choices is the real test here.

  9. Our govt. has just begun publishing League Tables in Education and exactly what you say is happening. It should be a good thing, but it has a dubious result. Yet I'm a great believer in freedom of choice in education. Can't blame the parents wanting the 'best'.

  10. Denise, that's what I'm talking about. Good ideas but dubious results. I teasing out thoughts for another post which explains why this is depressingly common.

  11. Hi Ian,

    In the world I work in (industrial automation), I have customers that come in and want to start measuring some parameter of thir factory's process. I always tell them to be sure they *really* want to know the results. They give me a strange look, and I tell them this story:

    A local food manufacturing plant makes pizza crusts for various frozen pizza producers. These crusts are punched out of a sheet of dough, partially cooked, frozen, and then measured.

    They promise their customer that it is round to within 1/4 of an inch. (That is, if it is a 12 inch crust, it is between 11.75 and 12.25 inches diameter, and the horizontal measurement is within .25 inch of the vertical measure.)

    The original method of verification was operators holding rulers. They would tape off the ruler at 12 inches, and do a quick check (like 1 second to check both measurements) to make sure it was "good".

    The company wanted to reduce cost by eliminating these 6 to 8 job positions. They hired me in to install a machine vision system that would look at the crusts on a conveyor and measure them as they went past.

    Guess what? Over 60% of their product was actually out of tolerance. Now between you and me, I don't care if my frozen pizza crust is 3/8 of an inch wider than it is long. None of their customers had ever complained, but now they suddenly had documented proof that what they were supplying their customers didn't meet the promise they had made.

    The vision system was quietly unplugged, put into a closet somewhere, and never mentioned again.



  12. LOL! That is a wonderful story, J. At least they recognized in the end that they didn't really want that measurement after all. Some folks would have spent millions trying to "improve" the product to meet a meaningless quality standard.


I love comments. Please feel free to join in the discussion.

I also try to respond to comments. I usually do so during the early evening (Pacific time) which may be many hours away from now!

So if you leave a comment and return some time later and I haven't responded yet, please don't think I'm ignoring you. I'm not. Honest.

Related Posts Plugin for WordPress, Blogger...