software development

Iterative Sprint Planning

I’m encountering a recurring theme in my agile coaching. Some organisations have a sharp divide between their business and IT functions. This always spins off a twin-track coaching path for me: on the one path let’s try and address the fundamental root cause of the dysfunction/divide and on the other path what can we do to ameliorate and get around the problem in the short term?

So far my experience is that most of the symptoms of this issue are revealed via the Product Owner role. Sometimes business folks just don’t understand the basics of either software engineering or software economics. It’s just a simple inevitability of the organisation and experiences/skills their career paths have taken them on. So they can be experts of the business domain, but appreciate none of the pain and realities of software development. That is not a trivial problem to solve and will likely take a lot of effort and time. So in the meantime, what should we do to minimise the impact? Well, for sure, this is going to depend on all sorts of factors surrounding what exactly the problems are being created by having Product Owners with no software experience.

I’ll describe what I have found works quite well at one particular client in regard of one slice of this overall challenge: Sprint Planning.

The difficulty faced by the agile teams was that the allotted time Sprint Planning was always reached before any kind of commitment to the content of the Sprint was close. On closer examination, there were (as so often is the case) a number of contributing causes for this and all need attention. As well as the common issue of developers being uncomfortable with the relative sizing of backlog items without knowing almost every line of code impacted (they may have got bitten in the past), the thing that seemed most acute was the compounding problem of unclear requirement. With the lack of an empowered and embedded full-time Product Owner, it was perhaps inevitable that even well-formed User Stories were insufficient in getting a shared understanding of the requirement to the agile team(s). In this circumstance, we have used a Business Analyst as a proxy Product Owner with regard to backlog generation.

Scrum and similar methods describe the Sprint Planning event in some detail. They suggest a planning meeting of around four hours per two-week Sprint. There is also a recommended ≤10% allowance in the Scrum Guide for backlog grooming and refactoring by the whole team. By converting that ≤10% grooming into more structured events, we are finding it much easier to contain the planning to the suggested time boxes. The structuring consists of three iterative parts to Sprint Planning:

  • GET-AHEAD sessions. The proxy Product Owner meets regularly with business (and other stakeholders) to discuss upcoming requirements a Sprint or two ahead of when the requirement needs to be delivered as a backlog item/ User Story. By starting these meetings ahead of the Sprint execution, it gives the diverse stakeholder community the opportunity to address issues and questions raised concerning the requirement. Typically, this was left until the Sprint Planning meeting previously, giving no time for issues to be analysed and answers to be gathered. This not only gummed up the Sprint Planning, it also blocked the subsequent Sprint execution. The agile team need not be involved in the GET-AHEAD sessions (it isn’t an effective use of their time) – this helps reduce the overall amount of time that the team spends in planning sessions.
  • CONFIRM sessions. Once the proxy Product Owner has established the requirement to a comfortable degree, the backlog refactoring would continue by discussing the requirements with the team. By this stage the requirement is largely understood by the proxy Product Owner so whilst business stakeholder attendance could be useful, it wouldn’t be mandatory. Discussions in the CONFIRM session are generally more technical and where the Product Owner is not of a software background, this is of limited interest to them and not a great use of their time. Whilst the proxy Product Owner would be able to answer most questions that the team would have, sometimes more questions may be raised and these are taken back to the next GET AHEAD session (hence the process is iterative). Typically relative sizing of the stories (through planning poker) starts in CONFIRM sessions but not task decomposition.
  • COMMIT sessions i.e. the Iteration Planning Meeting itself. Once the major issues have all been resolved in good time, the team can confirm the story content and sizes, do task decomposition and estimation, then commit to the delivering the work in the impending Sprint.

By following this process in an iterative way (just enough planning in just the right amount of detail), we have been able to get much more meaningful Sprint Plans produced just in time. There is still some way to go to get this working in an optimum way, but we have reduced one major impediment and can now move onto tackle others…

Funding New Projects

In a prior blog entry I noted the traditional approach to funding new development efforts:

  1. the business gets an idea and writes a few sentences of description,
  2. IT tries to figure out what the business really meant and comes up with an estimate based on their best guess, and
  3. this amount gets locked in as a "committed" estimate to which IT is held accountable. 

The only thing that should be committed is the people who think this is a sane approach to funding projects.

A better model works like this:

  • The business identifies a set of desired business outcomes that they would like to achieve, along with the net present value of achieving those desired outcomes.
  • Based on this value, the business decides how much it is willing to spend to achieve those desired outcomes.  Usually this is based on the required rate of return of investments in the business.  This amount establishes an upper limit on the project that could deliver those outcomes.
  • The IT planning process then focuses on determining whether the desired outcomes can be achieved within the constraints established by the business planning process.  If they cannot, further funding for the project is stopped and more worthy projects are funded.  Several alternative approaches may be tried and rejected, and deciding project feasibility may require some prototyping and evaluation to determine viability.

This is how research and development funding works and, let's face it, software development has a lot in common with R&D: both deal with a lot of unknowns at the start of a project, so much so that it is usually not possible to precisely define the exact work that will be needed to achieve a particular end.  Both R&D and software are essentially exploratory: at the beginning of the effort the exact result is not known with a high degree of assurance so we have to evolve toward a solution.  The current funding model's assumptions that the solution is easily definable and therefore should be easy to estimate just doesn't work for most new software development efforts.

Use-cases – why successful and popular? by Ivar Jacobson

I am pleased, honored and gratified that use cases are still a popular way of working with requirements.  Googling “use case” yields 6 times more hits than Googling “user story”, but software development should not be driven by popularity.  Instead we should use the most practical way of working.  And, of course we have learnt something from other techniques.  For instance, as I will discuss in my next blog, user stories and aspect-orientation have inspired us to make use cases even better while maintaining their core values.

The popularity of use cases has led to some misunderstandings and some distortions of the original technique.  This is natural, and while it is encouraging to see authors take the original concept and adapt it to solve new problems, some of the misconceptions and distortions have clouded the original vision.

Common misunderstandings

Before further discussing the improved use cases, let’s first discuss common misunderstandings about what we have had since their inception (1986-1992).  Many people believe that:

1) Use cases are for requirements only, which is not true. In fact, from the very beginning, they have also been used to analyze behavior, design software, and drive test identification, just to name a few uses.

2) Use cases are heavyweight; that you need to write fat specifications of each use case, which is also not true.  In fact, use cases can be exceptionally lightweight (a brief description only), to lightweight (just an outline of the flows), to comprehensive (full descriptions of all behavior), and every variation in between.  For most systems an outline can be very valuable and yet still be very lightweight.  Today, we express this in a better way: when describing use cases, focus on the essentials, what can serve as placeholders for conversations.

3) Use cases are a technique for decomposing the behavior of a system, which is also not true.  Some authors have introduced levels of decomposition, and others try to show use cases “calling” other use cases as if they were subroutines.  Neither of these is right.  A use case shows how a system delivers something of value to a stakeholder. Use cases that need to be “composed” in order to provide value are not real use cases.

4) Use cases are complicated.  In fact, using use cases, if done right, makes a system easier to understand.

o It is impossible to understand what a system does from looking at many hundreds of user stories; the equivalent use-case model might express the system’s behavior in a few handfuls of use cases. 

o A user is represented by a stick figure and a use case is represented by an oval.  Their interconnection by a simple line.

o The relationship between a use case and its scenarios are likewise very easy to represent. 

o To solve this problem with user stories, people have started to invent concepts such as themes and epics, making a case that the user story by itself is an incomplete concept. 

o The use-case approach can accommodate a wide range of levels of detail without introducing new and potentially confusing concepts.

5. Use cases are only seen as being good for green field development, which of course is not true.  They are great to explain large legacy systems as with such systems there is often little or no documentation left.  Use case modeling is a technique that is cheap and easy to get started with to capture the usages of the system.

What people like about use cases

The reason use cases have become so widely accepted is that since their introduction they are useful in so many ways in software development. 

1) A use-case model (a picture) already mentioned, which thus allows you to describe even a complex system in an easy to understand way, and which tells in simple terms what the system is going to do for its users. 

2) Use cases give value to a particular user, not to an unidentifiable user community.

3) Use cases are test cases, so when you have specified your use cases, you have also after complementing with test data, specified your test scenarios,

4) Use cases are the starting point to design effective user experiences, for instance for a web site.

5) Use cases ‘drive’ the development through design and code.  Each use case is a number of scenarios; each scenario is implemented and tested separately.

Moving forward
As we refine and improve use cases we are careful to make sure that we don't impact any of these things that are key to their popularity and success.  In my next blog I will describe how we adapted use cases to backlog driven development and managing cross-cutting concerns.

-- Ivar

Light Weight Software Development Process #2

The Software Development Process Challenge

*The first post in this series can be read here

Before I describe the deck of cards, I’d like to set the stage for using the cards. We can view software development from three time-scales (see Figure 1). Successful software development process is in essence, being able to coordinate the three time-scales effectively.

  • The broadest time scale covers from the beginning to the end of a software development cycle which is marked by several key business decision making milestones. This is of great interest to stakeholders and decision makers on whether development can proceed or whether the software is suitable for release.
  • The next time scale breaks the lifecycle into time-periods – months, or weeks, or what is known as an iteration. It is a fixed time-box where a number of target objectives are to be met by a development team. If there are multiple teams, then each team would be assigned their specific set of objectives. This is where team leaders operate.
  • The lowest time scale is what a team member does. The work done here can be in terms of hours or days depending on the complexity of the work.

Light Weight Software Development Process #2

Figure 1:The Three Perspectives to Software Development.

A major problem I see in many development organizations is the serious disconnect between the three levels. The objectives set at the higher level do not easily translate to work at lower levels. Lower levels are unable to see their contribution to higher level objectives. There is a  miscommunication between the levels, and poor coordination within the same level, which leads to blockages which should not even happen. We need to solve this.

Light Weight Software Development Process using State Cards

Light Weight Software Development Process using State CardsIt is a well known fact that all software is different; all software development teams are also different. So, why should we expect software development processes be fixed? There is no such thing as “one size fits all.” Yet, it is also common sense that there must be something in common, as otherwise there is absolutely no way to learn from experience and mistakes. The challenge is then, to find a middle ground that is easy to communicate to the development team and stakeholders. In the next few blog postings, I will present a pragmatic approach using a deck of A8 (5 1/4" x 7 7/8") sized state-cards that is small enough to fit into your pocket. I will demonstrate how you can use the state-cards to understand the state of software development, how to define your lifecycle model and how you can use it to define your value streams. It is important to get your team to define and own their software development process and state-cards provides the building blocks to do this.

The posts to follow on this topic include:

1.       The Software Development Process Challenge

2.       A Solution through a Deck of Cards

3.       Dynamic Software Development Lifecycle with State-Cards

4.       Lean Iterative Management with State-Cards

5.       Organizing Design of Lean and Agile Teams

6.       Effective Work Assignment Using State Cards

7.       Applying State-cards to Manage Software Development

Attitude Regarding Teamwork and Collaboration

Iterative projects often require fundamental changes to the way team members work on a project. On a traditional project, it is easy to avoid really working together as a team. Everyone tends to have their own specialty and deliverables, and it is easy for a person to work in isolation and only interact with other people by passing documents around.

Successful iteration is very different, and requires a very interactive, collaborative way of working, involving:

  • Commitment - The whole team (including business representative participants) needs to be committed to making the project a success. This doesn’t have to mean working long hours or promising more than can be delivered; it means just doing whatever it takes to work as a team and meet the iteration’s objectives.
  • Focus - The team needs to keep focused on the iteration’s short-term objectives and not get sidetracked by other issues or activities. This is particularly true of the management team members, who need to stick to the iteration’s set of objectives after they have been set and agreed to with the team. The time to change the project’s direction is between the iterations, not during them. Read More

Attitudes regarding risk and change

Attitudes regarding risk and changeMost project management approaches address risk in a fairly superficial way: risk mitigation is viewed as an important activity, but one that is largely a distraction from the main effort of the project. Risks are identified at the start of the project, they are usually assiged to someone to address the risks, but these risk-mitigation activities are largely viewed as things that take away from the project work. In practice the risks are only identified once and then they are largely forgotten in the large amount of work to be done.

On a traditional project, change is viewed as a bad thing, and great efforts are undertaken to prevent it. Change disrupts the original plan, and it is the plan (and not value delivery) that is sacrosanct on a Traditional project.

Similarly, change must be embraced. Change results from new information, from feedback that something in the original plan will not result in the desired end result. Change is something that will occur as people get new information, but change is a good thing: it means convergence on the right solution. Read More

Let’s measure the things that really matter

Let’s measure the things that really matterAs technologists, we tend to collect very detailed information and present it in a very technical way.  We talk about things like process maturity levels, and productivity indexes. I have even seen some measurement data about Agile maturity models that attempt to show how agile we are.  The problem with this kind of approach is it doesn’t provide anything that the business and the customers can relate to. 

The business does not really care how agile we are, they want to see results.  They want to be able to set targets, see improvements, and understand the benefits and the quality of the results being generated by IT. Only then can they begin to put the value of Agility into context – the context of the value and improvement generated by the move to agility. Read More

Attitudes Toward Delivering Business Value

Traditional software development projects are executed in a value-neutral setting in which:

  • Every requirement is treated as equally important
  • The delivery of technical components is seen as of equal importance to the delivery of usable systems
  • "Earned value" systems track project cost and schedule, not stakeholder or business value
  • A "separation of concerns" is practiced, in which the responsibility of the development team is confined to turning software requirements into verified code rather than delivering business value
  • The actual desired outcomes of the project are ignored in favor of implementing the largest number of requirements possible

No wonder so many projects fail to deliver the desired business results! Unfortunately this includes many iterative software development projects where the developers iteratively implement the requirements rather than delivering business value. Any management system that rewards things that are easily measured (like implementing requirements) without a clear and direct tie to business value delivered is headed down the wrong path. Read More

More accurate requirements: when process is lost

More accurate requirements: when process is lostOver the last few days a number of people have asked me the same question: “How can we get more accurate requirements?” While I agree it is not nice to answer a question with a question, in this case you might bend etiquette a bit because in reality this question can be hard to answer. For example, why is it necessary and which problem does it solve? Or, what exactly do you mean with accurate?  And do you have accurate requirements and you want more of those? And in that case, more than what? Or do you have inaccurate requirements which you want to become more accurate? And if so, how much more?

You might say: “Hey, that’s just playing with words”. Well, that’s right and so is writing and communicating requirements. In order to get accurate requirements you need a number of things. However, an often overlooked element to writing accurate requirements is understanding the structure of language and how language is perceived. Read More

Page 2 of 3123