The “Optimal” Fallacy

You cannot “optimize the whole”. The best you can do is sub-optimize, cooperate, and iterate.

Allow me to share with you the trouble of speaking at conferences and organizing courses. When conference organizers invite speakers, the optimal thing for them to do is to make speakers commit to their conference as early as possible, while the organizers themselves try to defer their own decisions for as long as possible. At the same time speakers who organize courses want to commit to conferences as late as possible, while trying to get participants for courses to commit as early as possible.

As a speaker and trainer my daily life is a neverending stream of decisions that all tend to be compromises between customers, partners, suppliers, and other stakeholders. People ask me for decisions about speaking engagements, while I’m waiting for courses to be finalized. I attempt to have partners commit to courses, while they defer decisions waiting for participants. We are all selfish bastards, optimizing for ourselves. And therefore we are all sub-optimizing.

There is no other way.

Optimize the Whole

There exists a Lean principle called “Optimize the Whole”, which is too often interpreted as an instruction to only optimize “the whole of the system” and to refrain from optimizing subsystems in an organization. Again and again I come across suggestions to “consider the whole value stream”, to “measure results end-to-end”, and to “focus on customer value”. The assumption is that the customer’s view on a business is one that includes the whole system and therefore a relentless focus on customer value delivery will “optimize the whole”.

This is not true.

A business is a complex system involving customers, employees, shareholders, suppliers, partners, local communities, and natural resources. (And a number of very confused managers.) They all try to optimize for themselves. No observer of a system can claim to have an objective view on the whole, wrote Jerry Weinberg in An Introduction to General Systems Thinking. And therefore, the customer’s view differs from the shareholder’ view. The shareholder’s view differs from the view of the local community, and so on, and so on. Movements like stakeholder management and sustainable development attempt to address the need for reconciliation of multiple views on businesses, from different perspectives.

The Optimizing Whole

We cannot “optimize the whole” by focusing only on the customer. The best we can do is sub-optimize, negotiate, cooperate, and re-optimize, in a neverending stream of compromises. All the parts try to optimize for themselves, and through interdependencies between the parts the whole system tends to evolve to an optimal situation, wrote Stuart Kauffman in At Home in the Universe. That’s how complex systems optimize a whole. Not as a principle to be followed, but as the outcome of interdependent parts, cooperation, and iteration.

The optimizing whole is emergent, not directed.

I call this kind of reasoning complexity thinking, which is slightly different from systems thinking. Some systems thinkers and Lean gurus seem to optimize their own consultancy firms instead of teaching people the nature of complex adaptive systems. But I get paid for helping people think, instead of selling solutions. Lean tools are nice, but counterproductive when applied without proper thinking, wrote John Seddon in Freedom from Command & Control.

Feel free to contact me and add an invitation to my very sub-optimal calendar.