I think I had some sort of a breakthrough idea today! It happened just over an hour ago. I had given a presentation about the Prince2 project management method and one thing that I had discussed with my collegues is that Prince2 in itself can never be sufficient to manage the complexity of our daily jobs. In my car, on the way back home, I was trying to figure out why this is the case, and why it's such a stubborn problem that people complain about it whenever they are confronted with formal process descriptions (regardless of the particular method). I think I now have (part of) an answer.
It takes complexity to manage complexity.
If you split any complex system in two subsystems, where one is the managing part and the other is the producing part, then both subsystems will need to be complex or they will fail. In many organizations the producing parts of the projects are usually complex (containing people, tools, deliverables, etc.) In order to manage that complex part the managing part (which contains processes, artifacts, reports, etc.) needs to be complex too.
The problem with Prince2, and with most other management methods, is that it is a static/ordered system. It is complicated, but the processes don't change and adapt, so the method in itself is not complex. Therefore you cannot succesfully manage software projects with just the Prince2 method. Of course, you can switch to complex management by adding people to the equation who are capable of continuously adapting the standard method to the environment. To be fair, I think many people knew that already. However, I think I now better understand the theoretical concepts behind it.
Let the records note that I have simply described here what came to my mind in the past hour or so. Be aware that I am just thinking out loud now! (I keep deleting all the dirty background thoughts that are also trying to find their way into this text…) You will understand that this idea still needs to be fleshed out before I can use it in either my book or my articles. But I will start working on that very soon.