On at least two occasions we've run into a Product Owner that was, to be honest, an agile team's worst nightmare.The Product Owner is supposed to represent the interests of the business, providing information on needs, requirements, priorities, and the like.  This one, however, was a former IT guy, steeped in the traditional approach.  Instead of merely establishing priorities and representing the needs of the business, he also wanted the team to have a detailed task plan that he felt he needed to approve.  He didn't understand or accept the idea of a self-organizing team.  In short, the guy was a control freak who felt that being the product owner put him in charge of everything.

While we want the Product Owner to be engaged and feel a sense of commitment, this guy needed to be committed, or at least get some therapy!  The twin ideas that there is no detailed iteration plan, and by this I mean a plan with formal tasks and dependencies, and that the team is self-directing, flew in the face of everything this former IT manager held dear. 

The agile team needs to act as a team, based on bonds of professionalism and mutual accountability.  There is no room for micromanagement, and being appointed the Product Owner does not come with the right or responsibility to tell the team how to do their jobs.  The Product Owner has clear responsibility for the what and the why, but not the how or when.

When you find yourself faced with such a Product Owner, the only suitable reaction is to stage a coup; you need a different Product Owner.  If that won't work, you need to accept that your project is not going to be agile.  You might even want to think about finding a new project.

In a prior blog entry I noted the traditional approach to funding new development efforts:

1. the business gets an idea and writes a few sentences of description,
2. IT tries to figure out what the business really meant and comes up with an estimate based on their best guess, and
3. this amount gets locked in as a "committed" estimate to which IT is held accountable. 

The only thing that should be committed is the people who think this is a sane approach to funding projects.

A better model works like this:

• The business identifies a set of desired business outcomes that they would like to achieve, along with the net present value of achieving those desired outcomes.
• Based on this value, the business decides how much it is willing to spend to achieve those desired outcomes.  Usually this is based on the required rate of return of investments in the business.  This amount establishes an upper limit on the project that could deliver those outcomes.
• The IT planning process then focuses on determining whether the desired outcomes can be achieved within the constraints established by the business planning process.  If they cannot, further funding for the project is stopped and more worthy projects are funded.  Several alternative approaches may be tried and rejected, and deciding project feasibility may require some prototyping and evaluation to determine viability.

This is how research and development funding works and, let's face it, software development has a lot in common with R&D: both deal with a lot of unknowns at the start of a project, so much so that it is usually not possible to precisely define the exact work that will be needed to achieve a particular end.  Both R&D and software are essentially exploratory: at the beginning of the effort the exact result is not known with a high degree of assurance so we have to evolve toward a solution.  The current funding model's assumptions that the solution is easily definable and therefore should be easy to estimate just doesn't work for most new software development efforts.

Part of the problem with funding projects is that, from a funding perspective, we treat software development as a one-time event.  When we create a new application I think we all expect that maintenance will need to be made over time, but this is not reflected in the funding model.  Each round of maintenance usually requires a project charter and funding approval.  In obtaining funding approval the maintenance project now has to compete with all other projects for funding approval.  This sounds sensible until you think of the implications.

If we have a building with a leaky roof, we know that we have to fix it or more damage will occur.  In software it is just the same: if the existing application supports the wrong business process, damage to the business occurs as long as the problem goes unaddressed. Forcing application maintenance into the project approval process can delay, sometimes indefinitely, the funding of needed maintenance, resulting in higher costs later on. Read More

It is that time of year again, when organizations are putting the final touches on their project funding plans for tomorrow.  Over the past several months, a strange tale has played out across the globe, a farcical tale that if you tried to tell anyone outside IT about they would scarcely believe you.  It plays out like this:

Someone in the business gets something resembling an idea - not something fully formed, but at least an inkling that suggests that there might be some benefit achieved in developing something.  A few sentences are written about it and here is where the silliness begins.  All these little fragments of ideas are gathered together and then IT is asked to estimate the cost and schedule - based on little more than the few sentences of description.  If they are lucky they might get to ask some questions.

Everyone knows this is ludicrous, they will say, but they say they have no choice.  They put all manner of disclaimers on the estimates but these are forgotten over time.  What is recorded and remembered is a time and dollar cost estimate for each project - based on, remember, a few sentences of description.  This sounds so absurd that I know you would be laughing if you did not see it every year in your own organization.  But wait, it gets worse. Read More

The agile ideal is for a team to be composed of equals, peers who neither direct each other nor oversee each other, but who work collectively toward a common goal. Team members are self-directing in the sense that they, seeing the goals of the team, choose to work on things for which they are most capable. Since many teams start from a traditionally managed, externally directed world, the journey toward this idea takes time and adjustment for everyone involved. A team typically cannot become instantaneously self-directing.

Typically the journey is assisted and facilitated by a coach who helps the team learn new techniques and overcome barriers. The very existence of the coach suggests that the team is not wholly self-directing, at least not yet; the coach will naturally provide some direction to team members during the learning process. The degree to which the coach will "direct" depends on the progress the team has made toward becoming truly agile.

A typical problem is reflected in a recent question raised by a client. We recommended use of a particular newer framework (SOAJ) in place of the Hibernate/Spring framework familiar to the team, based on our understanding of the goals of the project and the skills of the team; in short, the new framework would reduce coding and increase code quality while improving performance. Some team members felt more comfortable with their familiar framework, however, and the team was divided on the use of the new framework. If the team is self-directing and self-organizing, who makes the decision in the event of a split decision?

It often falls to the agile coach to make these decisions. The coach generally has greater experience and is better equipped to make the choice when the team lacks experience. Coaches must be careful not to become dictators, however benevolent, and teams must take care not to become over-reliant on the coach. Over time, the incidence of coach intervention should decrease as the team gains experience and confidence.

Coaching is more than mere facilitating new practices: there is always an element of leadership in successful coaching engagements. Teams learn best by doing, and the coach must lead by doing as well. Over time, team members step forward and the coach steps back until, one day, the team is leading and directing itself. At first, however, the coach takes more of a directing role.

Understanding the potential barriers to change will help you make the right choices of timing, project, and approach. Some of the more important questions to ask include in the following:

• How supportive is senior management of the change? The measurements and milestones
  they establish can easily derail an iterative approach, as is the case when they ask even seemingly
  innocent questions such as “When will the design be completed?” or “When will requirements be
  signed-off?”
• What is the scope of your authority to make changes? How much of the development lifecycle
  are you responsible for? For example, the requirements might have already been specified in a
  format and to a level of detail that would make it more difficult to adopt an iterative approach.
• What are the team’s feelings about the changes? How enthusiastic is the team about iterating?
  To achieve the transition to iterative development, you will need the support of the team,
  especially the other members of the leadership team.
• What else does the team have to do? How many other projects and initiatives is the team involved
  in? If the team is not focused on and dedicated to the project, the transition to iterative
  practices will probably be slower and take more time and energy to complete.
• What capability do you need to improve? It is important to understand the capability of the
  team and how well the current capability supports the proposed iterative approach. For example,
  is there any testing capability in the team? Testing will be needed from the first iteration, which is
  often a problem in companies organized around the phases of a waterfall approach. Read More

The progressive, adaptive, iterative approach is founded on the following principles:

• Proceed in small steps, continuing steadily by increments.
• The plans must adapt to the project’s changing circumstances.
• The characteristics that distinguish the product or service must be progressively elaborated.
• They should be broadly defined early in the project and made more explicit and detailed as a better understanding of the project emerges.
• Large projects must be broken up into smaller projects.
• Requirements errors can be 100 times more costly to fix after deployment than at requirements time.
• Initial estimates have a large margin of error—most waterfall projects cost two to three times what was initially predicted.
• The first one never works; always develop things at least twice.
• Initial designs will be flawed.
• Requirements must evolve alongside demonstrable versions of the product.
• Teams need to own and be accountable for their achievements and plans.
• You should believe the project results, not the project plan.
• Detailed planning should be limited to the next milestone.
• Projects should adopt an active and aggressive approach to addressing project risk.

Progressive, iterative processes were developed in response to the problems inherent in the waterfall approach. Instead of developing the whole system in one go, an increment is selected and developed, then another increment, and so on. The selection of the contents of the first increment is based on risk, addressing the highest-priority risks first. To address the selected risk(s), select a subset of the requirements to be implemented and tested. Develop the minimal set of functionality that enables objective verification (through a set of executable tests) that the risks have been successfully addressed. Then select the next highest risks to be addressed in the next iteration, and so on.
One of the great ironies of modern software development is that in the paper that led to the popularity of the waterfall approach to software development, Winston Royce argued that projects should always go through at least two development cycles because the first version never works. He was actually arguing that projects should take a more incremental approach, not that they should adopt a prescriptive waterfall development approach. Progressive, iterative approaches take this principle one step further by splitting the lifecycle up into many smaller increments, each explicitly addressing the project’s major risks.

Nowhere is the change in values of iterative development planning and measuring more important than in the management team. If the project is to iterate successfully, it must be managed iteratively.

The project manager must believe that an iterative approach is the best way to manage the project and must be prepared to set aside any inflexible, predictive, waterfall management practices that have been used before. This doesn’t mean that you should throw away all the good management practices and experiences you have built up over the years; good, disciplined project management is essential to the effective application of iterative development techniques. It is just that you must put aside some of the conventional wisdom about planning to give yourself the freedom to fully exploit the flexibility and power of the iterative approach.

The conventional approach to planning is prescriptive, based on the assumptions that the work which needs to be done can be predicted with great precision and that the unusual rarely occurs. This is true for many things—building a bridge over a highway or a standard family dwelling or a prefabricated commercial building, for example. These engineering efforts are technically predictable, and planning this kind of work is based on hundreds of years of experience. This experience has given rise to the generic project lifecycle in which the different types of project activities are aligned to the single phase that bears their name (i.e. Requirements, Analysis, Design, Code, Test and Deploy). Read More

Iterative projects often require fundamental changes to the way team members work on a project. On a traditional project, it is easy to avoid really working together as a team. Everyone tends to have their own specialty and deliverables, and it is easy for a person to work in isolation and only interact with other people by passing documents around.

Successful iteration is very different, and requires a very interactive, collaborative way of working, involving:

• Commitment - The whole team (including business representative participants) needs to be committed to making the project a success. This doesn’t have to mean working long hours or promising more than can be delivered; it means just doing whatever it takes to work as a team and meet the iteration’s objectives.
• Focus - The team needs to keep focused on the iteration’s short-term objectives and not get sidetracked by other issues or activities. This is particularly true of the management team members, who need to stick to the iteration’s set of objectives after they have been set and agreed to with the team. The time to change the project’s direction is between the iterations, not during them. Read More

One of the frequent questions we are asked is "how can I get the time to explore needs when the business says that it is already done - "it's in the business case", even when it is obviously not complete?  This is actually a pretty common occurrence - nobody wants to take time to understand needs because they think it has already been done. Read More