Squaring the circle

My recent posts have been labeled "Citizens first", which was all started by this post: Let's put citizens back in the frame.

The theme in that post was that early human societies developed to benefit their members. Our institutions have since grown so complex that they have lost touch with this purpose. Nowadays, they seem to serve practically everyone and everything but the ordinary citizen, and it's about time to set that right.

But recently, I've been prattling on about the arcane world of measurement - how we try to motivate people into certain behaviors by selective measurement of success. So, why is this important, and what's the connection?

The root problem is that governments and corporations are not motivated to think about ordinary individuals, let alone put their interests first. And why should they? I've talked a lot about measurement in various forms, and how often it leads to unwanted consequences, and here we see it in all its ugliness. Our world is the consequence of powerful measurements that reduce us to papers in ballot boxes and numbers on the bottom line. These organizations cannot possibly think of us as anything other than voters or consumers.

Which is scary, because these organizations dominate out lives.

My sense of hope is that we can take back control. That we can institute new ways of measuring success that will achieve beneficial outcomes for ordinary citizens.

For example, what if leaders and policymakers in education were recognized and rewarded not on exam pass rates in schools, but on some entirely independent measure of how well educated the general population was? What if the leaders and policymakers in healthcare were rewarded not by short waiting lists and how many drugs they can push, but according to the overall health of the general population? After all, that is what we pay them for, isn't it?

And what if the profitability of corporations was intrinsically tied to the net benefits they bring to the society in which they operate? This last example is a distinct shift in focus. What is the purpose of a car manufacturer? Is it to sell as many cars as possible, or is it to offer people the benefit of easy and convenient transport without inflicting undue harm in the process? Is the purpose of a grocery store to push produce, or to feed the local population? Because the profit & loss account is the only measure that matters today, corporate behavior leans towards the former end of the spectrum rather than the (IMO more desirable) latter end.

I spent some time examining measurement in the last few posts in order to show that this is no easy task. It is, after all, what many well-meaning people have been trying to do all along. They just haven't been very successful.

We need measures that are appropriate, which tell a rounded story, and which cannot easily be fudged.

To do that, we need to elevate the dark alchemy that is practiced (somewhat haphazardly) today into a precise science.

There's enough research into human psychology that I reckon we have a good handle on what makes individuals tick, but individuals are not really the problem. The problem is in the emergent phenomena that crop up at all sorts of levels when people get together into larger and larger groups, and the groups themselves start interacting and showing their own unique behaviors.

This is where the world has run out of control.

Pitfalls of measurement

In my last post, I talked about some ways people try to influence behavior by measurement, and some of the disastrous consequences.

The more I think about it, the more convinced I am that these are not simply exceptional cases. These are the rule, not the exception.

Any time you measure performance hoping to achieve an outcome, you are more likely than not to be disappointed.

There are reasons for this, and some possible answers. I'll talk about solutions in another post, but first it's important to understand why measurement is so difficult.

The biggest problem is that people are inherently competitive. The moment you start measuring something, people are extremely good at making sure they "measure up" - whatever that means. Sometimes this shows itself as explicit and conscious behavior. Sometimes the effect is subconscious or more subtle. Sometimes it is indirect, such as through pressures created within the organizations that people belong to, as a result of organizational measures such as profit.

Whatever it is, measurement influences behavior, and often not in the way you want.

Why is that?

Here are a few reasons...

The measure is not a good indication of the outcome

Many of the things we want to achieve are difficult to measure directly, so we end up with lots of indirect measures. Some of these are reasonable, many are not.

For example, we would like to think we are being led by people most fit to govern.

The theory of democratic voting might work well in small groups, where people truly know each other, and assuming that people have the integrity to vote for those they genuinely believe would be the most suitable.

The trouble is that modern democracy is little more than a measure of popularity, and in a large world where voters have no prospect whatsoever of actually knowing the candidates, popularity has in turn become a measure of all sorts of undesirable traits such as the ability to lie convincingly, or to most effectively trash your opponent.

These traits have nothing whatever to do with fitness to govern.

The measure is appropriate, but is not the whole story

People will strive to achieve whatever is being measured. Sometimes this comes at the expense of something else important, but the other important thing suffers because there is little motivation to pay attention to it. It is not being measured.

For example, a focus on saving lives, and on life expectancy, has led to people being kept alive at all costs - whether they want to or not. In focusing exclusively on one (albeit important) measure, other things like quality of life tend to suffer. Also, health services are under pressure to spend disproportionate amounts on expensive procedures to prolong the lives of a few individuals, at the expense of less glamorous treatments that would ease the suffering of millions.

The measure can be cheated or subverted

One of the biggest problems with many measurements - it is often seen as easier to cheat than to comply. Often, the only thing stopping people is honesty - which is another measure that people respond to in varying degrees.

The point of cheating is to improve the measure by some means other than that which was intended. This varies from outright dishonesty - ballot boxes stuffed, records falsified - to more subtle tactics that fall into a grey zone, such as pressuring marginal students into not sitting exams in order to boost pass rates.

And the scariest part is...

Most of the ways in which measurements misfire are not down to dishonesty.

People act as a result of myriad - often conflicting - motivations. Measuring some aspect of performance alters that web of motivation, and it's only natural that people most feel the pressure of the measurement itself rather than the underlying intent. As a result, people tend to follow the line of least resistance, even if the results are clearly undesirable.

And, given how difficult it is to devise a truly good measure in the first place, bad things happen more often than any of us would dare to imagine.

Measure for measure

Education departments publish league tables of schools' exam pass rates

The intent: To encourage schools to improve standards of education so more children achieve passing grades.

The reality: Parents start pulling strings to get their kids into the "good" schools. Schools in neighborhoods where their intake is inherently unmotivated suffer, no matter how good the teaching. Schools in a position to do so start being more selective in who they will accept. School teachers only allow children to sit exams who they think will pass. Instead of raising standards of education, many kids are actively denied the opportunity to stretch themselves.

IT help desk is outsourced, and the vendor is paid by the number of incident tickets resolved.

The intent: A simple measure that costs little to implement and results in fair payment for the volume of work performed.

The reality: The vendor is motivated to inflate the number of tickets opened. Every time a client calls to inquire on the status of a ticket already open - bang, you get a new ticket to log the call which is then promptly closed. Also tickets get closed whether or not they are properly resolved, and the when the client complains - you guessed it - a new ticket is opened. Result: frustrated clients and poor customer service.

Government starts measuring hospital waiting lists

The intent: To benefit patients by reducing the wait time for operations.

The reality: Doctors find ingenious ways to avoid putting people onto waiting lists in the first place. Patients miss out on badly-needed surgery as they jump endless hoops of referrals and tests.

People are elected into positions of power based on the number of votes they can garner

The intent: To use the collective wisdom of an educated electorate to select those most fit to govern.

The reality: Gaining votes becomes an end in itself, in a game which favors the most ruthless lying slimeballs imaginable. Truth is an early casualty, and actual fitness to govern doesn't get a look in.

Does anyone see a theme emerging?

In all these examples, and many more that I'm sure you could come up with, someone is measuring something that sounds perfectly reasonable, and with the noblest of intentions, but the outcome is far removed from the intent.

The moral of the story is - beware what you choose to measure, because measurement distorts behavior, often in unpredictable ways.

You think you are pushing people in one direction, but your act of measurement is in fact producing lots of forces in directions you never envisaged - and the system you are measuring will always move in the direction of least resistance. This will rarely be the outcome you expected.

This insight is part of the line of thinking in my "Citizens first" series of posts, which is all about putting citizens back in the frame as majority beneficiaries in our own society.

In order to achieve this goal, we need to find new ways to measure success - other than by the unfettered acquisition of money. Ways that will genuinely motivate individuals, corporations, and governments to think of the ordinary citizen first and foremost, and which will yield the outcomes we want to see.