Attitudes Regarding Iterative Software Development Planning and Measuring

Attitudes Regarding Iterative Software Development Planning and MeasuringNowhere is the change in values of iterative development planning and measuring more important than in the management team. If the project is to iterate successfully, it must be managed iteratively.

The project manager must believe that an iterative approach is the best way to manage the project and must be prepared to set aside any inflexible, predictive, waterfall management practices that have been used before. This doesn’t mean that you should throw away all the good management practices and experiences you have built up over the years; good, disciplined project management is essential to the effective application of iterative development techniques. It is just that you must put aside some of the conventional wisdom about planning to give yourself the freedom to fully exploit the flexibility and power of the iterative approach.

The conventional approach to planning is prescriptive, based on the assumptions that the work which needs to be done can be predicted with great precision and that the unusual rarely occurs. This is true for many things—building a bridge over a highway or a standard family dwelling or a prefabricated commercial building, for example. These engineering efforts are technically predictable, and planning this kind of work is based on hundreds of years of experience. This experience has given rise to the generic project lifecycle in which the different types of project activities are aligned to the single phase that bears their name (i.e. Requirements, Analysis, Design, Code, Test and Deploy).

Waterfall planning is founded on a complex system of interrelated beliefs:

  • Products can be completely developed in one pass
  • Requirements can be completed and frozen early
  • All requirements are of equal importance
  • Designs can be verified without building and testing something
  • The entire project can be planned in detail with a high degree of precision at the start of the project
  • Everything important can be known at the beginning of the project
  • The project can “earn” value by only doing one discipline at a time
  • Most importantly, if we follow the prescriptive steps of our process, then all the project risks will be addressed, and therefore one should measure teams on their ability to follow a plan and managers on their ability to create a plan

The focus of all waterfall methods is on optimizing the steps in the process to eliminate repetition and rework and to detect problems through detecting deviations from the plan. In short, the plan is usually regarded as perfect, and deviations from the plan are considered undesirable.

Why Conventional Planning Wisdom is Wrong When Applied to Software

The short explanation is that almost every one of these assumptions is wrong when applied to software. Software development is an immature discipline, typically requiring extensive innovation and invention, rather than a mature discipline guided by generations of experience. Software projects are also typically unique in that we do not build the same things in the same way over and over again.

We are usually building something new or rebuilding something in a new way - software is typically a response to the needs of a changing business. In addition, innovation in software development technologies proceeds at such a blistering pace that in addition to changing business conditions, the implementation technologies are constantly shifting.

In opposition to the assumptions underlying the waterfall methods, we find that:

  • Problems are discovered too late to do anything about them.
  • Predictive plans do not provide the flexibility and adaptability required to handle extensive unknowns.
  • Early commitment to detailed requirements is counter-productive.
  • Undertaking only one activity at a time separates teams, reduces effectiveness, and creates adversarial team relationships.
  • Only obtaining feedback from the next sequential activity delays the detection of errors and problems.
  • The plan does not adapt to the risks facing the project.
  • Integration and testing are always left until the end where, more often than not, they are cut back in an attempt to meet the original delivery date.
  • Design feedback is obtained late, usually leading to late design breakage, when it is too late to fix problems in the architecture.
  • Because the objective feedback provided by demonstration and testing is only obtained late in the project, risk resolution typically occurs too late to be effective.

The adoption of waterfall practices means that the problems are typically concentrated at the end of project, which is unfortunate because the majority of the budget has already been consumed and it is often too late to fix the problems.

The failure of the conventional wisdom about planning, especially for high-value, high-risk projects, has led to the evolution of a more progressive school of thought and the foundation of iterative development.