The Scoreboard Index versus The Twelve Principles of Measurements

Those who’ve read my #Workout book are probably familiar with the 12 principles for metrics in the Metrics Ecosystem chapter. Last week, I offered you a new concrete measurement practice, called the Scoreboard Index. Today, I want to have a look at how this practice holds up against the 12 principles.

1. Measure for a Purpose. (No) I make sure that my businesses and teams have a purpose, but the tool itself neither invites nor prevents having such a purpose. It is up to me and the team to define it. The purpose should translate to a few short-term objectives, which in turn translate to hard data with metrics.

2. Shrink the Unknown. (Yes) Each of my objectives must be supported with several metrics because I don’t trust decisions that are made using just one perspective. For example, I measure the success of a newsletter with total number of subscribers and number of opened emails and number of clickthroughs.

3. Seek to Improve. (Yes) There are many things you can measure but only few metrics are worth spending your time on. The Scoreboard Index forces you to focus on a small number of metrics and requires that you use them in an iterative decision-making process. Metrics that aren’t used should be replaced with other more useful ones.

4. Delight all Stakeholders. (No) The Scoreboard Index doesn’t cover this principle directly. Your purpose and objectives can be anything. Indeed, Volkswagen’s purpose was to become the biggest car maker in the world, and they managed their employees accordingly. Now, they face the consequences of having created many unhappy stakeholders.

5. Distrust all Numbers. (No) Again, the Scoreboard Index has no clear answer to this principle. People can easily deceive themselves and others with measurements. “How to lie with statistics” and “Lies, damned lies, and statistics” are well-known expressions. It is up to honest and smart people to use their metrics wisely.

6. Set Imprecise Targets. (Yes) The progress that you measure with a Scoreboard Index is the average across various metrics, using inaccurate scales, and multiple subjective happiness targets. This makes the final index quite a vague thing indeed, which is good because people shouldn’t get all hung up about a silly number.

7. Own Your Metrics. (Yes) I am responsible for my own Scoreboard Index, and each of the teams that I participate in shares ownership of the metrics. I take the initiative to offer an initial version, but team members must decide together which of the metrics should be replaced by better ones, and which targets we should aim for.

8. Don’t Connect Metrics to Rewards. (Yes) In my organization, compliments and bonuses are given in a peer-to-peer fashion. We don’t connect rewards directly to measurements and neither should you. Because changes in metrics directly influence the index, the tool is difficult to use reliably for bonus calculations. And that’s a good thing.

9. Promote Values and Transparency. (No) It was my decision to publish a Scoreboard Index on my blog, but you can just as well keep your results a secret. And what we saw earlier for the purpose of a business applies just as well to your company’s values. The tool does not offer a ready-made answer to misbehaviors.

10. Visualize and Humanize. (No) It is important that metrics are meaningful to people and that they are visualized in such a way that they easily convey what’s happening behind the numbers. Except for a dash of color, the Scoreboard Index doesn’t directly cover this principle. It leaves humanization to those who use the metrics.

11. Measure Early and Often. (Yes) The tool clearly scores positively on this principle, because it is intended to be used on a weekly basis in an iterative fashion. This means that it strongly favors things that can be measured daily or weekly, and that it requires people to use the data every week in their decision-making process.

12. Try Something Else. (Yes) Again, the Scoreboard Index has a positive score on this principle. The board is viewed every week and metrics changes on a regular basis. Obviously, it is up to the teams themselves to actually do this. But the tool doesn’t prevent it in any way, and it actually nudges people to do something when the index remains flat.

Conclusion: The Scoreboard Index satisfies seven of the 12 principles, which is a fine result. No practice is perfect and I’m sure there are ways to either improve this one or to augment it with other complementary good practices.

What do you think? Can we make it better?