I often get the question:  “RUP was your baby, but how do you look upon RUP today?”  In an interview a couple of years ago I responded jokingly: “Yes, RUP is my baby, but you know babies grow up and some of them need correction.”  RUP was created in Sweden under the name Objectory.  This was in 1987. 

Objectory was really new since its base could be used to describe all aspects of software development.  Our focus was to identify all of the rules that we use when we develop good software: what are good use cases, good components, good test cases, etc.  The literature in software engineering was totally empty in this space at the time. In order to do this we used object modeling to describe Objectory.  Now we could let the base grow forever – technically speaking.    Any company could make their own new process using Objectory.

Objectory became very successful.  It survived the merge with Rational in 1995.  The next version was named Rational Objectory Process, but after the success of UML, the name was changed to the Rational Unified Process or RUP as we all know it.  In the work of merging the two companies, the process content grew significantly.  However, hidden behind the size, our innovation of how to create a process base and the rules for goodness survived.  The crown jewels of Objectory had survived!

Thus, RUP is one of my babies.  The challenges with Objectory and consequently with RUP were serious. 

1. Adoption
   Although our way of working in creating a process base was new and great, adoption became quite expensive.  To truly be successful with the adoption and consistent usage was to involve customization, mentoring and consulting which most customers couldn’t afford.  Thus the success rate was too low.
2. Growth
   RUP was developed inside one company and all new ideas from the outside world had to be filtered by our own people and added to RUP.  It goes without saying that ideas come from anywhere in the world and by having a process be company owned makes it difficult to assimilate these great ideas.  The use of community much like what has been done with open source models would allow for continuous input and improvement.  This also allows for scrutiny. 
3. Execution 
   All processes are paper-ware and often managed by people who don’t use the process.  This quickly becomes a problem in most organizations because the process becomes a religious document and not a realistic one.  This then has the effect that what teams do and what the process tells them to do is often not in synch.  And since the team is in charge (doing the work) the value of process becomes small.

These challenges paved the way for the agile movement.  The different agile methodologists have one thing in common: they hate RUP.  But that is NOT very smart. There are lots of good ideas in RUP.  Many companies have made the effort in adopting RUP and done so quite successfully.  This has provided them with a very competent staff and something to build on for the future.

However, “the baby needs correction”, and I and my company have done so.  The adoption problem is attacked by focus on the essentials and being light instead of trying to be complete.  We use cards and game boards to describe the process.  Moreover, you don’t need to adopt a complete process all at once, just adopt a practice at the time.  The growth problem is not a problem anymore, since we downplay big process and we make instead practices first class citizens.  Practices can come from any source including the RUP and much of the practice work we have done is in support of RUP adoption. Practices can be composed to create your own way of working.  Finally, the execution problem which is the hardest to achieve, is for the time being our little secret, but you are welcome to learn more about it by studying EssWork and EssUP. 

Trust me, this is very smart.

During a recent trip to China and Australia I observed that everyone wants to be agile.   It may be that Northern Europe and the USA are a bit ahead but the trend is clear all over the world.  In a round table meeting with CIO’s, I usually ask what people are particularly interested in right now.  Five years ago a common answer was we are trying to adopt the Unified Process.  Now, the same question returns the answer we are trying to move to agile.  Thus you would assume that people know what agile is.     

The last month I gave four public presentations with around 100-200 people each.  I met with about twelve companies.  At every occasion, I asked what really is new with agile.  Here are typical unfiltered answers: “rapid iterations”, “working software”, “coping with change”, “communication”, “flexible”, “adaptable”, “eliminate waste”, “accepting changes”, “small iterations”, “feature-driven”, “continued integrations”, “test driven development”, “no documentation”, “people before process”, “adapt to change”, “the name”, “the team sets their own priority”, “early stakeholder involvement”.

An absolute majority, around 60%, said that agile is about iterations (or sprints to use the Scrum terminology).  It is a bit disappointing that people don’t know that iterative development was introduced more than 25 years ago by Barry Boehm.  He called it spiral development.

It is even more disappointing to hear that people think that RUP is not iterative, but based on the waterfall model.  In fact, if you wanted to use RUP for waterfall development you would have to make a real effort to restructure RUP.  We clearly said that everyone should move to iterations for the same reasons that people now like about agile: rapid, working software, change, flexible, risks, etc.

Given that around 60% think that agile is about iterations, and RUP was designed to support iterations, is RUP agile?  My answer is that RUP can be applied in an agile way but RUP itself is not agile.  Thus there need to be something more. 

20% of the answers were about technical ideas such as feature-driven, test-driven, user stories, etc.  However, none of these ideas would have created a revolution on their own.

10% of the answers were about light process – light to understand, light to use and light on documentation.  Now we start to come to the core of agile.  I truly believe that in the past we have been too ambitious in describing process, in adopting too much process and in documentation.  The reality is that even if people write a lot, very few people will ever read it.  Thus the trend towards light will sustain.  However, it is easy to be light.  The trick is to be as light as possible but not lighter.  I believe you will find our work on EssUP and EssWork new and fresh.

The last 10% were about how to work together daily, weekly, monthly, etc.  It is about communication, people and teams, about how to organize teams, how to take decisions, how to protect the team from the outside.  This is what we call social engineering.  Agile has put the finger on the fact that we need highly motivated and competent people to be successful with software development.  No process has ever developed software.  It has always been done by people.  We have of course always known this, but we have not pushed it as much.  The focus on people is really what makes agile unique, and this is why agile originally broke through.

Now, it doesn’t really matter what people think agile is.  Agile has become more of a philosophy.  It appears that everything good is now agile.  Thus it is not really easy to tell what agile is.  However, one thing we know.  Everyone will subscribe to being agile (as they should) so one day agile will go without saying. 

Let me though make a cautious reservation.  There is an obvious danger that as it continues, agile will be discredited because the concept is sometimes used as an excuse for doing shoddy work, for having no requirements, for developing whatever the developers feel like doing.  This is not in the spirit of "true agility" but if it continues it will give agility a bad name.

Whatever happens we will one day get a new fashion.  I can’t tell you what it will be but be sure of one thing: it will be smart, very smart.

The style in which we develop software is as fashion influenced as the clothing style, maybe even more.  We don’t dress ourselves dramatic different more than every five to ten years, but when it comes to software we change our development style dramatically every five years. Just a few years ago the Unified Process was a la mode and every software organization was adopting UP one way or the other.  Just 2-3 years ago the hottest style was Extreme Programming XP, but now I hear very little about XP.  Now, everyone is running after the next “silver bullet” Scrum.

But what about Scrum, isn’t it fantastic?  When I for the first time met Ken Schwaber, a father of Scrum, I said: “What I like about Scrum is that it describes how really good project managers motivate people and run an iterative project.”  I continued: “…and its beauty comes from being agnostic to how you actually do things…you can pick your own way to do requirements, design, code and test.” I summarized: “Can I label Scrum ‘a common sense project management approach’?”  “Yes”, said Ken, “that label works”.  After this introduction we had a lot to talk about.

I can assure you that the founders of Scrum didn’t view their baby as a silver bullet, but with all the pressure from people who love fashion and want to create buzz, they may very well have to accept that they have created a silver bullet.

I think Scrum is a great project management practice.  It also includes some good social engineering work patterns which make it agile. 

However, that is all it is. 

• You still need all the other practices that a good master of software needs to excel in, such as requirements, tests, architecture, components, modeling, configurations, and tools, etc.  This is all quite basic but important stuff. 
• You also need to know how to build complex architectures such as SOA, PLA, EDA, and EA, architectures that are not just paper-ware but executable. 
• Finally, Scrum is not enough if you want to know how to scale to large distributed organizations which may have outsourced projects.

What would make sense to do?

• If you already are a Scrum fan you can say that you need Scrum++ where the Scrum stands for the tip of the iceberg and the ++ stands for the bottom of the iceberg which constitute all the other good practices that you will need.
• If you are not a Scrum fan, you could consider replacing your current project management practice with a Scrum practice.  You can still work as you did before so you don’t need to throw out the baby with the bathwater.  However, you would use Scrum for project management for smaller projects.  For larger organizations you will need more.
• In both cases you need to use modern social engineering work patterns as you need to be agile.

The ability to mix and match practices from different camps is what I and my company have been working with over the last four years, and now we can do it.  That will allow us to move forward and stay away from the silver bullets, the hype, the buzz and the fashion and treat software with the respect it deserves.

This would be smart.  Smart by the industry. 

Management styles have a huge effect on software development teams.  A significant shift required in  the movement toward a more agile approach is a change in the way that teams are managed and measured.Management styles can be described in  a number of ways - Theory X and Theory Y are popular approaches dating back to the 1960's but still applicable today.   Basically, Theory X assumes that people are basically untrustworthy slackers who need to be constantly monitored and told what to do and are only working because they need the money.  Theory Y assumes that people want to do a good job and are motivated by more than money, and that people produce their best results when they are working in a supportive environment that frees them to be creative and productive.It should come as no surprise that a necessary condition for a movement toward agile teaming is a "Theory Y" management culture - a team trying to adopt an agile approach in an organization with a "Theory X" management culture is doomed to failure and frustration; it will constantly be fighting the management system and the surrounding culture. In these organizations, the management culture (and its supporting measurement system) must change along with the team approach in order for both to be successful. Before you can change it, however, you need to understand it.

I often get called into companies who are thinking of going "Agile".  They have invested many thousands of dollars on a very complex SDLC taking the best ideas from the industry, but that process is not being followed and their teams are effectively not working together, and now think that Agile will solve those problems.  They come to IJC with a simple question "how do I adopt agile ?".  But when you dig, you find that this question is not as easy as it would appear.  When you question their motivation and their constraints, you find a whole list of issues and problems that Agile by itself can not solve.  Issues can range from off-shoring, to quality and performance.  Issues that no one approach can solve.

Agile software development seems to be a way of describing everything that is good in software process today.  It combines techniques and team practices with ways of changing organizations.  Thus, when people talk about agile they are talking about so many different things it is really hard to get a handle on them all - it is like saying I like European food and everyone knows what that means.  Do I mean Spanish, Swedish, French...?  Of course I can not mean English.  There is a lot of great stuff that has the label Agile, but the area that I will discuss today are the team techniques that Agile has brought to the table.  In particular SCRUM. 

First let's define a structure that SCRUM fits into.  SCRUM provides a fantastic set of very simple project management processes that help teams better function, but it does not provide great guidance in the areas of actually building software or making sure that project fits into something much larger.  Thus, I always position SCRUM in the context of three other processes.  Firstly, on top is an organizational process.  These are typically described in SDLC milestones, gates or phases.  They represent the key stages a project MUST go through from a funding and control point of view.  The second view of process is the team.  How does the team function to deliver software in support of this lifecycle?  This I label as team.  The third is the techniques that an individual must employ to actually build software.  Techniques such as OOAD or Test Driven Development - Also this is where techniques such as Use Case Driven development fit. SCRUM fits nicely into the middle layer - It is a set of team techniques that really help the team become a team.  But without the top and bottom views of process being in place SCRUM by itself would not work. 

Firstly SCRUM provides a very simple set of roles.  Product Owner (the go to person who owns the problem you are solving), the SCRUM Master (the person who runs the SCRUM meetings and protects the team) and the Development Team (the people that do the work).  I also often add a fourth role that of the Technical Owner, the person who owns the system from a technical point of view and is the go to person about all things about the architecture.  Secondly it provides three meetings.  A kick off meeting at the start of a sprint.  Oh, I had better introduce the idea of a sprint - A sprint is like an iteration.  A small chunk of time when stuff happens - has a clear set of goals and delivers stuff.  In terms of what it delivers they are classically stories, but can also be features or other units of stuff.  What is important about a sprint is that real work is done, that means working software is produced, tested and deployed (maybe to a pseudo production environment).  The second type of meeting, and the one that gets all the press is the daily Scrum - this meeting is short, say 15 minutes, and focused on three questions.  How did you do, what are you doing and what is stopping you?  These three questions enable the project to move along at a rapid, focused rate.  The third meeting is all about retrospective and proving you did what you were promising in your sprint.   So, SCRUM provides a great way of organizing the team, but without the lifecycle the team can not decide on what sprints they will do and the management can not appreciate the value of this sprint in the context of something bigger.  Without some techniques for building software individuals do not know what to do. 

So SCRUM must always be part of something bigger to really make an impact.  I would therefore argue that agile on its own is like a single food item; really to enjoy it, it needs to be put into the context of a meal.

August 15, 2003

I am currently in Lima, Peru where I have participated in an international conference and given a series of workshops.

This week started in New Orleans, where I gave a keynote at the XP Agile conference. I arrived late Sunday night but my XP friends were kindly waiting for me. Ward Cunningham, Bob Martin and Ron Jeffries were there. I had a great time with these people whom I respect very much. I have known Ward since 1987 and he is one of the few people who has moved technology into the masses with ideas such as CRC Cards, XP and also Wiki. Ron Jeffrey and I have been on a panel discussing light versus rich processes and we will be on another panel in October. We were missing Kent Beck (who arrived later) and my old personal friend Dave Thomas (who is on the advisory board of Jaczone).

I sympathize a lot with XP and agile methods. I have different opinions on some XP ideas, but in general I think XP is a good approach when exploring a new product. However, I think that XP doesn’t help you grow an organization. Therefore I want a process that can be as light as XP when you start a new initiative, but a process that can grow with the organization as you become successful. Well, I could talk a lot about that but not in this short postcard. My keynote was about the "Future of Process". The core message is that we talk too much about process instead of focusing on getting the job done. Instead we should rely on best practices of many kinds: technical, human, project, organizational etc. These best practices need to be well-defined, they need to be kept separate, but composable, and need to change as time goes by. Ideally the end-result should come across as an "Invisible Process". Here techniques like Waypointer developed by Jaczone will play a very important role.

Peru was very friendly, but the conference could have been better organized. I gave three presentations, I had about 200-400 people each time despite my talks being announced just 4 hours in advance of the actual talks. One challenge in Lima is that people in general don't speak English that well. Luckily, they found an excellent translator for me -- Elizabeth Ramirez Perasso, who has been studying my work for many years. I found that I have many fans in Peru. Thus I need to go back!