Iterative and Incremental Development: New Definitions

I came to the conclusion that iterative and incremental development both need a new definition. Here is a piece of text I have been working on. It's not finished yet, so it's likely to contain some serious and embarrassing errors. But I keep rewriting it until it is of a Pulitzer Prize level.

<ArticleAlmostWorthOfPulitzerPrizeButNotYet>

Many authors have elaborated on the importance of iterations and increments, particularly in the last ten years. Unfortunately, only few of them have made serious attempts to explain what the differences between iterations and increments really are. Even worse, methodologies like RUP and Scrum have muddled the waters by making iterations and increments equivalent to each other, which could make it difficult for project managers and developers to decide how to handle temporal cycles in their software projects. And communication among agile practitioners doesn’t get any easier when authors of agile methods like XP and DSDM don’t agree on definitions and terminology.

Incremental and iterative: old definitions

The 16-year old definitions of increments and iterations, provided by Alistair Cockburn, are as follows:

• Incremental development is a staging and scheduling strategy in which various parts of the system are developed at different rates, and integrated as they are completed.
• Iterative development is a rework scheduling strategy in which time is set aside to revise and improve parts of the system.

While these definitions have been extremely useful for a long time, I now think that they have begun to break down under the environmental changes of the last decade. The definitions, in particular the first one, reflect the old way of looking at the construction of big systems: break them down into manageable parts, schedule them all to be built and then assemble the system from those parts.

Some agile methods can be blamed for confusing the incremental and iterative strategies, but it seems that there was a good reason behind it: We have learned to build better products by putting them into the hands of our users. Only the people who actually use our products are the ones who are in the best position to tell us what we did wrong. This was the whole idea behind evolutionary and adaptive software development, both of them popular terms in our field until agile development became the norm.

Before the whole evolutionary-adaptive-agile wave iterative development was often seen as a rework cycle within the cycle of big increments. But with the new approach incremental delivery itself, just like iterative development, became just another rework strategy. It appeared that users preferred working with a system that was only half correct and half finished, over having to wait endlessly for a system that was being tuned to perfection. I believe this is the main reason why so many experts simply decided to lump iterative and incremental development together: both of them are now associated with learning and adaptation.

Weekly demo+planning sessions are examples of incremental practices.

Incremental and iterative: new definitions

Therefore, I think we need to adjust our views on increments and iterations, or we will never be able to understand which approach is best for our projects, and how popular methodologies and best practices compare with each other. Allow me to explain my first attempt at cleaning up this mess that we got ourselves into.

For development of complex software systems in the 21st century the definition of incremental development might look like this:

• Incremental development is a staging strategy in which various parts of the system are developed and delivered to the user at different rates, with the intention for the software project to adapt to external feedback.

The goal of an increment is to expose a growing system to the environment it will need to survive in. The feedback drawn from real users, real hardware, real infrastructure and real interfaces, acts as a feedback loop that enables learning and adaptation. A software project continuously needs to seeks the optimum value in a fitness landscape that is defined by a performance system comprised of numerous quality criteria. These criteria are imposed by the environment and are always subject to change.

After rewriting the definition of incremental development the definition of iterative development might look like this:

• Iterative development is a rework strategy in which various parts of the system are revised and improved at different rates, with the intention for the software project to understand the system it is building.

The goal of having iterations is to learn by doing. The feedback in this case is from a person’s own knowledge and experience, from other members in the team, and from people representing the customer and the intended users. In reworking requirements, the user interface, the data model, the architecture and any other parts of the product, the project learns to deal with itself as a complex system. Note that feedback from users may be involved here too. But they will be users who are part of the project itself, acting as people who might have a better chance at predicting how the product is going to survive once it is released in its real environment.

Refactoring and daily stand-ups are examples of iterative practices.

</ArticleAlmostWorthOfPulitzerPrizeButNotYet>

So, what do you think? Am I on the right track for a Pulitzer Prize?

images: Sukanto Debnath, Shayan

Jurgen Appelo is an award-winning speaker, trainer, and author of Management 3.0, a management book for software development. You can hire him as a speaker or trainer, to add some spice to your discussions, workshops, or conferences.