The Danger of Lean: Ignoring Social Complexity

After reading three books about Lean software development I feel slightly uncomfortable with the heavy emphasis the Lean community seems to place on process and economy.

(Transparency: I read Implementing Lean Software Development, Managing the Design Factory, and Lean Software Strategies. They were advised to me by several lean experts.)

The only things these Lean books seem to focus on is reducing queue size, reducing cycle time, reducing variability, reducing work in progress, and reducing batch size. It is all inventory this, and control systems that. No wonder the Lean brand appeals more to higher management than Agile. Somehow the word “people” got lost somewhere. (Note: I’ve been told the people-issue is an important topic in the works by Deming and Jeffrey Liker. But I specifically wanted to investigate software development books.)

Try it yourself! Just pick your average Lean software development book, go to the index, and try to find references to concepts like motivation, creativity, innovation, personality, or fun. You probably won’t succeed. And if the content is there at all, it is drowned out by all the stuff about inventory, processes, control, economics, value streams, efficiency, queues, variability, and waste. (I’ll be honest: the Poppendieck book has a chapter called “People”. But this little gem of information pales in comparison to the volume of economical process jargon in the other books, and the many blog posts about Lean software development.)

And I don’t have a problem with this. All that stuff is indeed very valuable!

However, there is a problem when people see Lean as The Only Way.
Because it isn’t.

Alan Shalloway published a blog post The Dangers of Complex Adaptive Systems, where he tried to analyze where complex adaptive systems (CAS) work and where they don’t. I respect Alan a lot, but his blog post completely misses the point about complexity theory. Most systems in the universe are complex systems, and many of them are adaptive. Debating whether or not CAS works is like debating whether or not biology works. It makes no sense. The system just exists. Whether it is a school of fish, a flock of geese, or a team of software developers, they are all complex adaptive systems. You can debate whether certain survival strategies of these systems work. If the strategies don’t work, the systems will need to adapt, or they go extinct. If they don’t go extinct, then obviously the strategies work! That is all part of the theory of complex adaptive systems.

Complexity theory is just a scientific description of the world. It has no “good” or “bad” qualifiers.

But Lean is a prescriptive model. It tells you what you should do when you try to manage the complex system that we call a software development team. It tells you there are good and bad choices.

And that’s where things can go wrong…

The Incompressibility problem of complex systems tells us that every model (including Lean) is inaccurate:

There is no accurate representation of the system which is simpler than the system itself. In building representations of open systems, we are forced to leave things out, and since the effects of these omissions are nonlinear, we cannot predict their magnitude.
– P. Cilliers (2005), Knowing Complex Systems

When you create a model of a system (like a software project), you have to leave things out. Unfortunately, because of the famous Butterfly Effect, in complex systems every little thing that you leave out can still have huge unexpected consequences. And because complex systems are non-linear, you cannot predict which parts of the real system you can safely ignore.

This leads to a simple, but very important conclusion:

All models are wrong, but some are useful.
– Box and Draper (1969), Evolutionary Operation

Lean software development describes a software project as a pull system with a value stream. That means it is a model of the real system, which is more complex. It is a very, very useful model. Unfortunately, it is also wrong because it is inaccurate. But so are all the other models. And some models are worse than others. (Actually, I think Lean probably scores quite well among all those bad models.)

The Lean model ignores the social part of complex systems. I just reviewed all the diagrams in the three books mentioned earlier. I couldn’t find a single diagram or picture with people in it! What are the consequences of ignoring the people-issue in the model? What happens to people’s motivation when you reduce variability? What happens to people’s creativity when you treat the system as a production flow instead of an emerging design effort? How is the process aligned with people’s personalities? And where is the fun? I can tell you, those three books were about as much fun as my last visit to the dentist. Useful, yes. Healthy, sure. Painful, oh God, yes! They certainly didn’t make me look forward to being part of a lean software development team. (I fondly remember the first books about Agile software development. They were about people. And they were fun. There’s certainly more humor on the Agile side…)

Again and again I see emphasized that Lean software development is based on systems thinking. That is correct. And that is exactly the problem. Just have a look at this interesting diagram created by David Snowden, a complexity expert (the red text in the diagram is mine):

Systems thinking is a great tool, but system thinking ignores the unordered parts of the world. The messy bits. The unpredictable stuff that always comes as a surprise, no matter how much time you’ve spent drawing process diagrams and analyzing value streams. In software projects, the proper foundation for management is not systems thinking but social complexity. Social complexity is complexity theory, but applied to people instead of fish and geese.

The danger of Lean is that it overemphasizes process and economy. This makes it deceptively easy for manage
rs to embrace Lean (instead of Agile) and to forget all about people-issues. Again. And the people-over-process principle will be ignored. Again.

Social complexity is the only proper foundation for management of software teams. Systems thinking is outdated. Ralph Stacey, another complexity expert, threw systems thinking in the trash bin. And rightfully so. Goodbye, and good riddance. Systems thinking has served its purpose, but science has moved on.

So yes, Lean has some fantastic ideas, as there are great ideas in Agile too. But neither Lean nor Agile is a complete model of a complex system. The only accurate model of a complex system is the system itself. Which means you must focus on the entire system, including the people. And not only on value streams and waste reduction. It takes a complex mind to recognize the dangers of incomplete models.

I’m writing a book about this stuff. Did I tell you that? I think some managers are going to need it. The book will be useful, and healthy. And fun.

Note: I could just as well have named this post “The Danger of Agile”, focusing on the things that Agile ignores. But there are already so many posts bashing Agile, Scrum, and XP. It wouldn’t have been interesting to write. I’m human too. I want my work to be fun. I think increasing variability is more fun than reducing it.

(picture by Foxtongue)

 Twitter –  Subscribe – Newsletter – Bookmarks