Measurement

Measuring Outcomes not Causes – Making Measures Inspirational not Punitive

As discussed in an earlier blog post, you need to measure the desired results.  If Agility and Agile Transformation is going to make you better, then look at the things it is going to make better and set some targets.  If it is going to make you faster, then look at the things it is going to make faster and again set some targets and track against them.  If it is going to make you cheaper and happier, then focus on those things.  This approach begins to make your measurement framework much more inspirational.

Historically, many organisations struggle with their Organisational Measurement getting any traction.  I have seen many organisations where every year they introduce some key performance indicators and by the end of they year, they have abandoned them.  I have seen many organisations with large dashboards with graphs no one understands and no one looks at but a lot of money is spent on them.

You need to change this whole approach.  Measurement needs to change at the organisational level to become more agile – valuing people over processes – collaboration over contracts etc. All the great things from the Agile Manifesto should be visible within your measurement program as well. 

A balanced scorecard organized around the four goals of better, faster, cheaper and happier is a great place to start.  Recently, we worked with KPN and implemented a organizational measurement program using the better, faster cheaper, happier framework. It was an excellent framework for their organisation. It allowed them to look at what is important in terms of getting better – Were they becoming more predictable and were they delivering a higher level of quality?, faster – cheaper - , and happier -    The framework allowed them to look at the problem areas and phrase questions that business and IT could understand. It was so successful that everyone in the whole organization, including the business, embraced the measures with both enthusiasm and commitment. See the case study / webinar for more details.

A better, faster, cheaper, happier balanced scorecard provides a way to communicate the effectiveness of IT in a way that everybody in the organisation can embrace and understand. It also allows you to measure success in a way that is independent of the processes used by the projects and teams being measured – essential if you want to demonstrate that an agile approach is better than more traditional approaches or build a platform to enable you to continually improve and adapt your agile way-of-working.

Better, Faster, Cheaper, Happier: An Agile Balanced Scorecard

Better, Faster, Cheaper, Happier: An Agile Balanced ScorecardAdopting an agile approach has a lot of benefits, not just when developing software and improving team working but for all aspects of our business. When it comes to measurement, I recommend that we all take an agile approach and keep things simple, open and adaptive to change. 

A simple, intuitive score-card is essential for any team or organization embarking on any form of agile adoption or transformation. The Better, Faster, Cheaper, Happier framework provides a great starting point for motivating and measuring your changes.

Agile: What’s the Point?

Let’s stop and think about why businesses get involved in agility.  What is the promise of agility that is so attractive to businesses?  What we at IJI have found when talking to business people is that agility is appealing because of the belief that better products will be built.  Not only will they be better software products but they will be built faster and cheaper than before. The promise is certainly there  and this is what people want to see as they make investments in agile adoption and, in particular, any kind of large scale agile transformation. 

The other significant result claimed for agile ways-of-working is that there will be an overall increase in everyone’s happiness - happier customers; happier users; happier sponsors; happier purchasers, and happier developers. Read More

Let’s measure the things that really matter

Let’s measure the things that really matterAs technologists, we tend to collect very detailed information and present it in a very technical way.  We talk about things like process maturity levels, and productivity indexes. I have even seen some measurement data about Agile maturity models that attempt to show how agile we are.  The problem with this kind of approach is it doesn’t provide anything that the business and the customers can relate to. 

The business does not really care how agile we are, they want to see results.  They want to be able to set targets, see improvements, and understand the benefits and the quality of the results being generated by IT. Only then can they begin to put the value of Agility into context – the context of the value and improvement generated by the move to agility. Read More

What is an Iteration?

What is an Iteration?

Iteration: A self-contained mini-project, with a well-defined outcome: a stable, integrated, and tested “release”. Let’s look at the three aspects of this definition in more detail.

A software development project produces a new release of a software product by transforming a set of users’ requirements into a new or changed software product. With an iterative and incremental approach, this process is completed little by little, step by step, by splitting the overall project into several mini-projects, each of which is called an iteration.

From the perspective of the development team, each iteration can be considered to be a self-contained
project. This approach is very powerful because it enables the development team members to focus on meeting their immediate objectives and ensures that the results generated are frequently and objectively measured. The management team needs to ensure that the iteration objectives form
a credible part of the larger overall project.

The management team needs to reinforce this way of working by ensuring that each iteration has the following:

Read More

"Earning" earned value by Ivar Jacobson

Traditional project management approaches focus on planning in detail, assigning the resulting tasks to people and then tracking "progress" as measured by completed tasks. The problem with measuring progress this way is that completing a task, while important, is hard to correlate with progress against the overall goal - just because you've completed 20% of the tasks does not mean that you're 20% done - and for tasks that take a long time to complete the self-reported estimates of "percent complete" is often merely "wishful thinking".

My preference is to measure progress in a concrete and measurable way - in the form of tested scenarios, following an iterative project management approach.  In other words, planning works iteration by iteration, with each iteration developing and testing one or more scenarios.  At the end of each iteration, you have a set of developed and tested scenarios, making progress easier to measure: knowing that you've developed and tested 20 out of 100 scenarios is a lot more meaningful than knowing that you've completed 20% of the tasks - especially if those tasks are focused on creating documentation rather than running and tested code.  Scenarios correlate nicely with business value - each scenario should be useful to at least some subset of the stakeholders.  In my view, only when you've successfully tested a scenario can you claim to have "earned value".

Measuring Project Success and Managing Expectations by Ivar Jacobson

There are a number of studies that cite poor performance of software projects - The Standish Group being the authors of one of the more often cited, their Chaos Report (and old version from 1995 is posted here, and although the data is old the conclusions are not dramatically changed).  The gist of these studies is that the majority of projects (as high as 70%) fail when measured against original schedule, budget, and expected features. I would be the last to argue the general conclusion: that it is very hard to manage a project to success. 

Most projects lack clear direction and purpose, and many are rife with disagreements about what success looks like.  There is, however, something in the assumptions behind these studies that rings hollow: that the initial schedule, budget and expectations for projects is a reliable milepost against which to measure. Most projects are vaguely conceived at best - they often lack a clear understanding of why they should exist and what problems they need to solve.  At their initiation they are usually poorly scoped and vaguely purposed, and the funding associated with them is often assigned based on an  allocation of an arbitrarily assigned budget.  Their schedules, at least those produced at the start of the project, are largely speculative endeavors, a mixture of gut and guesswork, that bears little basis in reality.  Measuring project performance against the initial schedule, scope and budget is of little value except to illustrate the point that there is a large disconnect between the expectations of business sponsors and the ability of teams to deliver against those expectations.  There are, to be sure, rampant problems with performance, but there are also widespread woes of expectations that are just as important to address.

Where should we start?  The first place is probably with project funding and measurement. The real thing of importance to measure is whether the project produced (or exceeded) the business value  expected of it.  If a delay in the project caused a market window of opportunity to be missed, that is significant, but it is the decline in value delivered that needs to be measured, not a schedule variance that cannot be correlated with economic activity.  Forcing a focus on business value produced would also put the right attention on the role of the business in following-through on their assertions of the value that will accrue from having the solution.  Requesting projects based on business needs has an opportunity cost - choosing one project over another should affect the value delivered to shareholders - and accountability for assertions by the line of business is just as important as accountability for project delivery.

If we shift our attention to value delivered rather than meeting schedule and budget, we may free the development team to find better ways to deliver the value, which may or may not include the initial set of features envisioned by the business sponsor.  Initial feature lists are usually vaguely conceived and don't provide a very good target for delivery.  Work is usually required to ascertain the real needs from this initial list of "features", some of which contribute to satisfying real needs but many of which are simply good initial starting points for discussion about real needs.  It may very well take longer than expected to solve the real problems (it usually does, as we all tend to be more optimistic than we should about how long things will take).

The problem is that most teams are set up to fail from the start.  By measuring them against budgets and schedules based on arbitrary assumptions and often a poor understanding of the real business value that needs to be produced, we find them constantly struggling against a plan that cannot possibly succeed.   Measuring against initial schedule, budget and expected features is not merely meaningless, it's actually part of the problem.  We need to shift our focus to better articulating problems to be solved and needs to be satisfied, and measuring business value produced.  Once we start to do that, we can focus on the plans and milestones needed to ensure the delivery of business value.