The Danger of Lean: Ignoring Social Complexity

After reading three books about Lean software development I feel slightly uncomfortable with the heavy emphasis the Lean community seems to place on process and economy.

(Transparency: I read Implementing Lean Software Development, Managing the Design Factory, and Lean Software Strategies. They were advised to me by several lean experts.)

The only things these Lean books seem to focus on is reducing queue size, reducing cycle time, reducing variability, reducing work in progress, and reducing batch size. It is all inventory this, and control systems that. No wonder the Lean brand appeals more to higher management than Agile. Somehow the word “people” got lost somewhere. (Note: I’ve been told the people-issue is an important topic in the works by Deming and Jeffrey Liker. But I specifically wanted to investigate software development books.)

Try it yourself! Just pick your average Lean software development book, go to the index, and try to find references to concepts like motivation, creativity, innovation, personality, or fun. You probably won’t succeed. And if the content is there at all, it is drowned out by all the stuff about inventory, processes, control, economics, value streams, efficiency, queues, variability, and waste. (I’ll be honest: the Poppendieck book has a chapter called “People”. But this little gem of information pales in comparison to the volume of economical process jargon in the other books, and the many blog posts about Lean software development.)

And I don’t have a problem with this. All that stuff is indeed very valuable!

However, there is a problem when people see Lean as The Only Way.
Because it isn’t.

Alan Shalloway published a blog post The Dangers of Complex Adaptive Systems, where he tried to analyze where complex adaptive systems (CAS) work and where they don’t. I respect Alan a lot, but his blog post completely misses the point about complexity theory. Most systems in the universe are complex systems, and many of them are adaptive. Debating whether or not CAS works is like debating whether or not biology works. It makes no sense. The system just exists. Whether it is a school of fish, a flock of geese, or a team of software developers, they are all complex adaptive systems. You can debate whether certain survival strategies of these systems work. If the strategies don’t work, the systems will need to adapt, or they go extinct. If they don’t go extinct, then obviously the strategies work! That is all part of the theory of complex adaptive systems.

Complexity theory is just a scientific description of the world. It has no “good” or “bad” qualifiers.

But Lean is a prescriptive model. It tells you what you should do when you try to manage the complex system that we call a software development team. It tells you there are good and bad choices.

And that’s where things can go wrong…

The Incompressibility problem of complex systems tells us that every model (including Lean) is inaccurate:

There is no accurate representation of the system which is simpler than the system itself. In building representations of open systems, we are forced to leave things out, and since the effects of these omissions are nonlinear, we cannot predict their magnitude.
– P. Cilliers (2005), Knowing Complex Systems

When you create a model of a system (like a software project), you have to leave things out. Unfortunately, because of the famous Butterfly Effect, in complex systems every little thing that you leave out can still have huge unexpected consequences. And because complex systems are non-linear, you cannot predict which parts of the real system you can safely ignore.

This leads to a simple, but very important conclusion:

All models are wrong, but some are useful.
– Box and Draper (1969), Evolutionary Operation

Lean software development describes a software project as a pull system with a value stream. That means it is a model of the real system, which is more complex. It is a very, very useful model. Unfortunately, it is also wrong because it is inaccurate. But so are all the other models. And some models are worse than others. (Actually, I think Lean probably scores quite well among all those bad models.)

The Lean model ignores the social part of complex systems. I just reviewed all the diagrams in the three books mentioned earlier. I couldn’t find a single diagram or picture with people in it! What are the consequences of ignoring the people-issue in the model? What happens to people’s motivation when you reduce variability? What happens to people’s creativity when you treat the system as a production flow instead of an emerging design effort? How is the process aligned with people’s personalities? And where is the fun? I can tell you, those three books were about as much fun as my last visit to the dentist. Useful, yes. Healthy, sure. Painful, oh God, yes! They certainly didn’t make me look forward to being part of a lean software development team. (I fondly remember the first books about Agile software development. They were about people. And they were fun. There’s certainly more humor on the Agile side…)

Again and again I see emphasized that Lean software development is based on systems thinking. That is correct. And that is exactly the problem. Just have a look at this interesting diagram created by David Snowden, a complexity expert (the red text in the diagram is mine):

socialcomplexity

Systems thinking is a great tool, but system thinking ignores the unordered parts of the world. The messy bits. The unpredictable stuff that always comes as a surprise, no matter how much time you’ve spent drawing process diagrams and analyzing value streams. In software projects, the proper foundation for management is not systems thinking but social complexity. Social complexity is complexity theory, but applied to people instead of fish and geese.

The danger of Lean is that it overemphasizes process and economy. This makes it deceptively easy for manage
rs to embrace Lean (instead of Agile) and to forget all about people-issues. Again. And the people-over-process principle will be ignored. Again.

Social complexity is the only proper foundation for management of software teams. Systems thinking is outdated. Ralph Stacey, another complexity expert, threw systems thinking in the trash bin. And rightfully so. Goodbye, and good riddance. Systems thinking has served its purpose, but science has moved on.

So yes, Lean has some fantastic ideas, as there are great ideas in Agile too. But neither Lean nor Agile is a complete model of a complex system. The only accurate model of a complex system is the system itself. Which means you must focus on the entire system, including the people. And not only on value streams and waste reduction. It takes a complex mind to recognize the dangers of incomplete models.

I’m writing a book about this stuff. Did I tell you that? I think some managers are going to need it. The book will be useful, and healthy. And fun.

Note: I could just as well have named this post “The Danger of Agile”, focusing on the things that Agile ignores. But there are already so many posts bashing Agile, Scrum, and XP. It wouldn’t have been interesting to write. I’m human too. I want my work to be fun. I think increasing variability is more fun than reducing it.

(picture by Foxtongue)

Twitter TwitterRss SubscribeEmail NewsletterDelicious Bookmarks

Latest, greatest and favoritest posts:
ScrumButs Are the Best Part of Scrum
The Complex Manifesto for Software Development
Simple vs. Complicated vs. Complex vs. Chaotic
  • The Do-It-Yourself Team Values Kit
  • The Open Space Policy for Managers
Related Posts
free book
GET MY FREE BOOK!
“How to Change the World”
  • http://martin.cron.com Martin Cron

    I was all ready to argue with you over this one. Ready to bust out the “respect for people is one of the seven main principles of Lean” but you’re dead right on this one.
    Too much discussion and thought around Lean is too prescriptive and leaves humans out of the picture. I’m starting to think that different people have varying degrees of compatibility with different models of software development, and that there’s nothing wrong with adapting the model to fit the people doing the actual work.

  • Brian Mead

    Lean is great for making more widgets with less waste. It is not great for improving software development, especially if you are looking down from 10 ft. Software development is more like art than widgets, and Lean not only forces you to pick painting, it’s like paint-by-number.
    Lean works for software development if you are looking down from 10,000 ft. It’s good at finding waste in the overall process, but the waste usually comes from corporate culture than from actual software development. If you want to clean up your software development, one retrospective will do more than ten kaizens.

  • http://www.chwlund.com Carl

    just read a roundup-article of the Lean And Kanban conference which explains this quite well I think:
    http://lizkeogh.com/2009/10/02/lean-and-kanban-conference-roundup/
    “That was when I realised that this community – even more so than most of the Agile communities I’ve been involved with – are hugely respectful, communicative, collaborative and fun-loving. They don’t talk about people much because it’s all about the people.”
    And you mentioned that you are reading “Implementing Lean Software Development”. That book is based upon the original book “Lean software development” by the same authors which has a whole chapter called “Motivation”. The book you are reading also has this whole chapter called “People”.
    I also think its very important to distinguish between maintenance projects and new development projects when discussing Lean vs Agile and so on. I really think Kanban and Lean introduce very useful mindsets and techniques for organizing the maintenance of an it-system.

  • Paul Boos

    Great post; having done some Kanban work to streamline some software maintenance work and also some on the “floor” work at a Navy Air Depot, it is all about the people. But you’re right, it seems many of the postings/books emphasize problem solving techniques to make a process lean. I believe (because these all originate from Deming thinking) that t is not intended to leave people out, but given the average manager doesn’t think about people much and more about process I can see the worry.
    So you’re spot on, even though it probably isn’t the various author’s intent. It’s unfortunately what management will read into it. It would be best if these books emphasized the human aspect.
    Paul
    (Who depending on your POV is fortunately/unfortunately a management type…)

  • http://profile.typepad.com/6p0120a5c306bd970b Machiel Groeneveld

    I don’t see Lean as a complete model that applies to the whole process. It’s more constructive to view the rise of Lean from a historic perspective. In the end it’s all about finding the optimum mix of tools to apply to your project or organization. From a historic perspective, Agile repaired the imbalance introduced by the big RUP processes, which made people forget about teamwork.
    When I read the Lean books, I assume we’re past the Agile point. I’ll never forget it’s about people, but you can also view the process from a more technical point of view (Lean) and see new ways to optimize the process on top of making sure there is good team work. For some reason every method promotes itself as ‘complete’ and ‘universal’, but they’re only fighting for a place in the sun. If you look at history, I predict people will first start following Lean like zealots, only to find out after 2 years it’s too one dimensional (like you said) and restore balance yet again.
    Agile is now at the balancing stage (it’s starting to pay off!), which is why a few Agilists are furious that people are already talking about introducing Lean “just when we finally got this Agile thing right!”

  • http://www.mendeltsiebenga.com Mendelt Siebenga

    Lean doesnt ignore the social parts, It reads more like you ignored the social parts of lean in your reading. Lean is more than the pull system. Just like agile is more than just doing iterations. Most of what lean has to say about quality insurance actually works bottom op, enabling the workforce to improve the process to improve quality.
    Lean is just as incomplete when it comes to actually managing groups of humans as agile. I tried your book-index-test with ken-schwaber’s books for example and came up negative too. Actually the Poppendieck books scored a bit better there.
    Agile as well as lean are about processes, not HR. I agree there is room for improvement there on the lean as well as the agile front.

  • http://www.bobtuse.blogspot.com/ Bob MacNeal

    Well considered argument and well-written post. Lean manufacturing has lessons, but gives scant “warm fuzzies” to the people doing the work. I usually bristle when I’m on the receiving end of a prescriptive approach, and thanks to your post, I can now express why.

  • http://profile.typepad.com/jurgenappelo Jurgen Appelo

    In reply to some of the comments:
    I am not against Lean, and I don’t favor Agile over Lean. (Like I said, I could write another such post about Agile.)
    But I am against the lack of information about people-issues in Lean literature. It it a _fact_ that (some) managers read about Lean and then start culling people from their workforce.
    The lack of communication about people in Lean literature is the danger. Not the lean practices themselves.

  • http://www.targetprocess.com Michael Dubakov

    Jurgen noticed an interesting point, however these books are irrelevant on my opinion. It is better to start from some fundamental books about Toyota and, for example, “The Goal” by E. Goldratt. The Goal is a very interesting book, it is written as a novel. And while it focuses on processes, it focuses on people even more.
    So on my opinion you may consider SD Lean books as a Level II. Maybe better books about Lean in SD are coming, but if managers will start massive Lean adoption based on just 2-3 books reads – they are in danger.

  • http://profile.typepad.com/6p0120a5af1467970c Ryan Martens

    Jurgen – great post on this topic. If you read Kent Beck’s comments on my post “Lean and Agile are an Oxymoron?” you see a similar thread on the people emphasis in agile. (http://www.rallydev.com/agileblog/2009/06/agile-and-lean-software-development-an-oxymoron/)
    I know from working on the Agile Alliance board, we held on to “humanity” as part of the core purpose of agile. I really feel the force of people in agile more so than Lean. I agree with the other comments that Lean talks about people, but when I talk to most people about lean – they all talk about eliminating waste.
    Clearly the best future builds on a combination of”Agile and Lean,” not Agile or Lean. I am so tired of Lean VS Agile and Scrum VS Kanban. Make sure to read’s Jean post on Escalation – http://www.rallydev.com/agileblog/2009/10/escalation-is-killing-agile-can-we-please-stop-it/
    Really appreciate the post.
    Ryan

  • http://agileanarchy.wordpress.com/ Tobias Mayer

    Excellent post Jurgen. Good to hear someone else acknowledging the fierce process-focus of Lean as it is applied in the software world. It is too bad Lean is being adopted by so many consultants as an alternative to Agile. It is a step backwards.
    Additionally, I especially liked your thoughts on systems thinking and complexity theory… “Social complexity is the only proper foundation for management of software teams”. Indeed.

  • http://blog.brodzinski.com Pawel Brodzinski

    Jurgen,
    Actually you could stop after:
    there is a problem when people see Lean as The Only Way. Because it isn’t.
    You could also replace Lean with pretty much anything and it would be equally true.
    Actually people are way too much bonded with names or methodologies to see the simple truth: there’s no panacea. If there were we’d all be using it soon. Those who’d resist would also extinct.

  • http://www.changingorganisations.com Stephen Billing

    I agree with you that complex adaptive systems (CAS)theory is a very useful way of explaining the behaviour and patterns of complex adaptive systems and there is not much point in trying to debate whether or not they work.
    The point is whether or not human social endeavours are complex adaptive systems and hence subject to what we have learnt about CAS. It is commonly taken for granted amongst CAS thinkers that they are.
    I think not. Societies, organisations, groups of humans are not systems of any kind – not simple systems, not soft systems, not living systems and not complex adaptive systems. In systems, the parts have a purpose which is to make up the whole and are defined in relation to other parts.
    The parts of systems do not have choice, consciousness, and will. Humans do, and therefore our social worlds, like societies and organisations are not systems, and so any kind of systems thinking is inappropriate when applied to human interaction. You could think of the human body as a system, but not human interaction.

  • mawi

    “When you create a model of a system (like a software project), you have to leave things out.”
    “The Lean model ignores the social part of complex systems.”
    “So yes, Lean has some fantastic ideas, as there are great ideas in Agile too. But neither Lean nor Agile is a complete model of a complex system.”
    I’m sorry, what are we really learning here?
    Systems theory ignores messy bits. And:
    The danger of leaving an perspective out of a model is that users of the model forget that perspective?

  • http://profile.typepad.com/gregorpetri Gregor Petri

    Jurgen,
    Think your last line sums it up, “I think increasing variability is more fun than reducing it” . Although Lean (manufacturing) was developed to be able to produce more different variants and models of similar products , variability was never a goal more an enemy. For people who consider themselves artists, Lean is not the way to go. Unless maybe if you work at the Place du Tetre in Paris and sell hundreds of paintings of the same bridge in variable sizes and frames day after day.
    Lean is about maximizing value for the customer and minimizing waste. Not something someone like Rembrandt really cared about (remember why his customers did not want to pay for the Nachtwacht, that was not just because they were Dutch). Even in the golden age there was only one Rembrandt and several thousands of people painting for a living. Lean is about production , that is also why – unlike Agile – Lean is taking such a high flight in IT Operations. I am convinced working in a Lean car shop beats the working at assembly line for T-Fords, but let’s not compare it to people who design and handcraft Morgan and Spykers (because let’s face it, most of us don’t).
    Gregor Petri
    http://Twitter.com/leanitmanager

  • http://davidvivash.wordpress.com David Vivash

    I agree completely with what you’re saying – “lean software development” is not a process – it’s about making processes efficient. Pretty much all processes leave something out, it’s just important to recognise what’s missing when adopting a process, and to introduce some extra practices to fill in the gaps.

  • http://www.netobjectives.com Alan Shalloway

    Jurgen:
    Sorry, I think you are totally off the mark here. Lean is based on a foundation of respecting people and creating an environment where their intrinsic motivation and creativity will come out.
    This is a common misrepresentation of Lean – that because it has a science underneath it it must therefore ignore the social complexity. Culture is incredibly important in lean. Only problem is, you can’t change it directly. Culture is a conversation people have that results from the management experience people have. To change culture you have to change the management.

  • http://lisacrispin.com Lisa Crispin

    I’m no expert on Lean, but I really like the Lean books by Bas Vodde and Craig Larman, because they focus on people. I recommend those, if you haven’t read them. A different perspective from the other Lean Development books I’ve read.

  • Jim Sutton

    Jurgen,
    You are spot-on about the need to put people first. But you might want to go back and reread the details in the works of the authors you’ve referred to.
    One of the four fundamental principles in Deming’s system of Profound Knowledge is “Psychology,” which is “the study of thw human mind, including how people act and interact in different situations.” Deming anticipated applied complex adaptive systems theory by many years. Deming said that all four principles were essential, so he clearly understood the importance of people.
    You correctly point out that the Poppendiecks refer to the human factor, though I think you underemphasize their commitment to it. I’ve read them too, and they struck me as very much people-oriented.
    Al Shalloway’s writings are all about people; how could that not have come through? Ditto for Don Reinertsen (“Design Factory”): For instance, he tells anecdotes about how the modern military is all about applied complexity, and that old-style command and control is a thing of the distant past. Of course, he is a flaming genius at turning qualitative thinking into quantitative thinking (e.g., via queuing theory), and math is frequently less human-feeling than, say, history…but you need both. Even Turing, in coming up with the roots of complexity theory, worked up math for morphogenesis…the first major example of the “butterfly principle” to which you appeal as an example of the need for more human emphasis.
    One of the authors of “Lean Software Strategies,” I am also a certified Cynefin (complexity theory) practitioner. This means I actively work with the human part of systems. In our book there are multiple references to the importance of people, including both text and many diagrams (it’s at the heart of the Kano chart and model figures in the book, for instance).
    My co-author Peter Middleton is a university sociologist as well as software professor, and the last fifth of the book is a series of case studies he did on the human effects of applying Lean in real-life (i.e., complex) situations. There are numerous human-related figures in his chapters as well.
    So, I agree with your premise that we must make people central to the systems that develop and use software systems. I don’t see how provocatively painting a large portion of the existing Lean literature as if it didn’t do this at all (“I couldn’t find a single diagram or picture…”)—when it clearly does so even if it is less up-front than you would like—and using self-congratulatory wording (“It takes a complex mind to recognize…”) is strengthening your case.
    Better to find common cause and build upon it to strengthen the community, than to reach for differences that don’t stand up to closer examination and use them to differentiate oneself from others. Insult is not my intent. Just suggesting being a spot more careful with references in the future, and showing a little more restraint about the “win/lose” approach of appearing to promote oneself at the expense of others.
    Regards,
    Jim Sutton

  • http://profile.typepad.com/jurgenappelo Jurgen Appelo

    Everyone: thanks for the feedback!
    I’ve learned that some Lean thinkers and experts are actively addressing social complexity, and others don’t. And some claim they do, in a few paragraphs, but then completely lose themselves in queuing theory and non-human systems.
    It depends, of course, on which books and blogs you read.
    Let me say I’ll continue to read about Lean (and Agile, and complexity) in order to understand the full picture, and to arrive at the nuanced view that, perhaps, I sometimes lack. 😉

How to Change the World - free Workout - free
CLOSE