Sysadmin Field Notes

August 18, 2006

I just finished Measuring and Managing Performance in Organizations by Robert D. Austin.

I don't know if a management book has ever made me more sad. Here is a pretty decent model that describes everything I have ever observed about using measurement in management. It's backed up with references to extensive studies, lots of data, and lots of interviews and conversations.

He explains and formalizes a thought I've long held anecdotally; namely that there are cases where measuring everything and managing by those numbers makes sense, and there are other cases where trying to manage to the numbers can be poisonous for an organization. With his model, he is able to predict the criteria under which measurement might be made work and where it will fail.

Most impressively, he builds a good case for why, in a complicated industry that meets his criteria for where managing by numbers will fail, the difficult is inherent to the activity. It's not just that the person implementing the measurement scheme messed up the organization or measures so it didn't work, it's that the activity fundamentally resists measurement. So all attempts to apply a "full supervision" model to particular types of activities will fail. That's my favorite part of the book.

So now, armed with all of this information, a lot of it common sense and a lot of it gleaned from data, certainly companies must be moving away from the "measure and reward" model for their complicated activities right? The book was written in 1996, it must have taken the world by storm and swept across the business world right?

Sadly no, it's been completely ignored and companies seem ever more hot than ever to roll out measurement based performance schemes where they are at best expensive and ineffective and at worse are a genuine poison to your high performing teams. That's why it's depressing.

He's got a good explanation for why people continue to ignore the obvious flaws in purely measurement based systems as well.

The only thing lacking in this book is more ideas about what to do about it. Which is fine in the setting of the book. But I would love to see a "guerilla guide," with lots of practical advice about helping convince people that capturing and managing by metrics has a series of tradeoffs that must be considered carefully.

I highly recommend this to just about anyone.