I often get called into companies who are thinking of going "Agile".  They have invested many thousands of dollars on a very complex SDLC taking the best ideas from the industry, but that process is not being followed and their teams are effectively not working together, and now think that Agile will solve those problems.  They come to IJC with a simple question "how do I adopt agile ?".  But when you dig, you find that this question is not as easy as it would appear.  When you question their motivation and their constraints, you find a whole list of issues and problems that Agile by itself can not solve.  Issues can range from off-shoring, to quality and performance.  Issues that no one approach can solve.

Agile software development seems to be a way of describing everything that is good in software process today.  It combines techniques and team practices with ways of changing organizations.  Thus, when people talk about agile they are talking about so many different things it is really hard to get a handle on them all - it is like saying I like European food and everyone knows what that means.  Do I mean Spanish, Swedish, French...?  Of course I can not mean English.  There is a lot of great stuff that has the label Agile, but the area that I will discuss today are the team techniques that Agile has brought to the table.  In particular SCRUM. 

First let's define a structure that SCRUM fits into.  SCRUM provides a fantastic set of very simple project management processes that help teams better function, but it does not provide great guidance in the areas of actually building software or making sure that project fits into something much larger.  Thus, I always position SCRUM in the context of three other processes.  Firstly, on top is an organizational process.  These are typically described in SDLC milestones, gates or phases.  They represent the key stages a project MUST go through from a funding and control point of view.  The second view of process is the team.  How does the team function to deliver software in support of this lifecycle?  This I label as team.  The third is the techniques that an individual must employ to actually build software.  Techniques such as OOAD or Test Driven Development - Also this is where techniques such as Use Case Driven development fit. SCRUM fits nicely into the middle layer - It is a set of team techniques that really help the team become a team.  But without the top and bottom views of process being in place SCRUM by itself would not work. 

Firstly SCRUM provides a very simple set of roles.  Product Owner (the go to person who owns the problem you are solving), the SCRUM Master (the person who runs the SCRUM meetings and protects the team) and the Development Team (the people that do the work).  I also often add a fourth role that of the Technical Owner, the person who owns the system from a technical point of view and is the go to person about all things about the architecture.  Secondly it provides three meetings.  A kick off meeting at the start of a sprint.  Oh, I had better introduce the idea of a sprint - A sprint is like an iteration.  A small chunk of time when stuff happens - has a clear set of goals and delivers stuff.  In terms of what it delivers they are classically stories, but can also be features or other units of stuff.  What is important about a sprint is that real work is done, that means working software is produced, tested and deployed (maybe to a pseudo production environment).  The second type of meeting, and the one that gets all the press is the daily Scrum - this meeting is short, say 15 minutes, and focused on three questions.  How did you do, what are you doing and what is stopping you?  These three questions enable the project to move along at a rapid, focused rate.  The third meeting is all about retrospective and proving you did what you were promising in your sprint.   So, SCRUM provides a great way of organizing the team, but without the lifecycle the team can not decide on what sprints they will do and the management can not appreciate the value of this sprint in the context of something bigger.  Without some techniques for building software individuals do not know what to do. 

So SCRUM must always be part of something bigger to really make an impact.  I would therefore argue that agile on its own is like a single food item; really to enjoy it, it needs to be put into the context of a meal.

Right now I am in my home in Switzerland to take a break and ski. I ski during the days and I work mornings and nights. Sleep takes too much time so I do as little as possible of that.:) One of the questions I often get nowadays is “Does Process come from the top or from the bottom?” Before answering I will start with two questions that I usually ask during my presentations. “How many of you love process?” In a group of 750 people, I usually see 3-5 raised hands. These are of course the people who work with process implementation, and they don’t have to follow a process themselves. Then I ask provocatively “How many of you hate process”. Now I get a very different answer. Usually 60-70% of the audience raise their hands!

Why is it like that? I believe that in many larger companies process has been pushed down on the people who know how to develop software. This won’t work anymore. The competent will speak up to get the process they want, which is good. What worries me though is that the pendulum will swing over to the other extreme. Instead of process push-down we will get process pop–up.

People do as they want. If every team gets their own process, they will not be able to learn from other teams, they will not get common set of tools and they will not harvest reusable assets. What we really need is a balance between push-down and pop-up.

For a very long time (since 1987) I have told my clients that one of the most important success factors for process adoption is to identify both a sponsor and a champion for the process improvement. The champion is a developer with many years of experience, great competence and with high credibility among his comrades. The sponsor is a higher level manager who wants to improve the way his organization develops software to support the company’s business. Without support from both don’t bother to adopt any new process.

Another of success factor is to use experienced people to coach the team adopting the new process. However in many cases management wants process but doesn’t want to invest in coaching. They just hope that people will adopt the process offered to them without proper coaching. As a consequence a lot of organizations have failed in process adoption. Management tried a shortcut, but it became a detour. The money an organization spends on product development is a factor of ten higher than the money it spends on proper coaching. For twenty years I have told my clients that without adequate training and coaching don’t bother to adopt any new process.

A third success factor is to balance the need for process from the top (the management) and from the bottom (the developers). This is true for any process whether it is a software development process or a business process. Only working from the bottom is a very slow process, and working only from the top creates a lot of resistance. Unfortunately, in many organizations management pushed down process on people counter to our recommendations. The developers were not enthusiastic but they tried to adopt the new process. That was not easy. Management believed their people were using the process but the developers did what they wanted to do, usually what they always had done. We got a gap, sometimes a huge gap, between what the process said and what people actually did. We call it the process-project gap. Thus we got failures and people started to hate process.

We need to change this. My recipe contains four ingredients:

1. Reduce complexity. Instead of adopting a big process consisting of many practices, adopt a practice at a time. This will significantly reduce the complexity of process improvement.
2. Balance the push-down from the top and the pop-up from the bottom, instead of just doing one or the other.
3. Allow teams to change the practices so they work in their projects. The way you really work must be fed back as a change to the practices. We want a process that is alive and not just a dead book. This allows us to close the gap. One challenge here is of course that traditional process hard-heads will demand that all deviations from the described practices must be documented and perhaps even be given formal exemptions. If that is necessary then people will feel that it is easier to do what is demanded than to do what is right. What we need is a utilitarian perspective on process rather than dogma. :)
4. Activate the practices and bring them alive in a way traditional process has never been able to do. While activated, a practice can help with all kinds of things. It can review what the developers are doing and suggest corrections. It can automate mundane tasks transparent to the developers. It can train the developers on the job reducing the need for classroom training and traditional human mentors. The selected practices should give context-sensitive help to the developers. This means that they should only give the developers help when they want it and only the help they really want and no more. The goal is to give the developers more time to think and create and to remove a substantial part of the no-brain work that developers suffer from today.

We have had a period of process push-down. The pendulum is now swinging to the other extreme – process pop-up. After a few oscillations the pendulum will hopefully swing to a new position where we will get processes that balance both the need for governance and for the need for creativity.  

There is a conflict between people who support Software Engineering approaches (such as Rational Unified Process RUP) and people who support Agile methods (such as SCRUM or XP). This is understandable since the two approaches are described in an incompatible way. However that is unnecessary, since the ideas behind the two are in all essentials complementary. The trick is how to combine them in a way fair to both of them.

In the software world we like swinging. We are swinging from one way of working to another way of working -- from one extreme to the other. Some people have been swinging for their whole life and have had enough of it.

Once, about five years ago, when I spoke in Singapore for a small group of 50 people, a senior guy stepped up, very frustrated, and told me that UML and RUP would be dead in five years and I would be gone as well. I like provocations so I asked him calmly what he based that upon. He told us that he had worked with software his whole life. “In the 60s we all worked with assemblers, then we got Cobol and Fortran, then we got database design” and he continued describing a zigzag path with new methodologies in every step: “structured programming, structured analysis and design, object-orientation, components and so on.” He felt it was awkward hearing about a new standard modeling language. I told him that there was another path: Assembler, components with assembler, component with a programming language, components with structured programming, components with objects and now components with objects and aspects. And we have all the time used a modeling language that can be described as UML.

Well, I am still here and I intend to be here for many more years…knock on wood. The guy who was so unhappy became even unhappier next day when he was fired by his boss. However, it is true that UML and RUP have lost some momentum, but they are far from dead. They are just taking a break while the world is swinging.

We are swinging from the extreme of doing everything with UML following what many believe is a prescriptive Software Engineering process. And we are swinging to another extreme called Agile. Very young people are having their first swing and now they are Agile. Middle-aged people, what a cute label, are right now swinging from the extreme of working with Software Engineering to another extreme called Agile. We swing and our companies swing with us.

Today everyone is agile. Of course. Everything else would be silly. Let me say it loud and clearly: I am a big fan of agile. My teams at Ericsson were extremely agile.

However, most people I talk to have a fuzzy understanding of what agile really is about. I participated in a panel discussion with agile evangelists in the UK last March. The organizers had hoped I would be against agile. They were surprised. The audience asked us to define agile. The panelists claiming they were agilists started to ramble. They repeatedly said that the most important property of agile was that it was iterative. That is dead wrong. Iterative development is one of the key practices in RUP and it has been around long before RUP. It was previously called spiral development and created by Barry Boehm in the late 70s. So I had to help explain what agile is:

Agile is about three things:

1. Most important, agile is about social engineering. This is what actually made agile different. It is about how to work as a team, how to make people motivated, how to collaborate, etc.
2. Agile is about lightweight. Instead of relying on explicit knowledge like in RUP, agile relies on tacit knowledge. In RUP we tried to write down what we know are good practices. However, since people don’t read process books, it doesn’t make sense to have them. Instead agile assumes that people have knowledge in their heads that is enough to develop good software. This can of course be debated but that is what it is.
3. Agile contributes some technical practices. This is the weakest part of agile. Very little is really new. Iterative and incremental development is as I already have said an old idea. User stories are a special kind of simplified use cases. Most interesting new idea is test-driven development. I am not claiming the agile technical practices are uninteresting, just saying that if it were just for these ideas we wouldn’t have been excited about agile.

As you see Software Engineering and Agile tackle different aspects of software development. Software Engineering’s strengths are about technical practices and Agile’s strength is about social engineering. Thus here the two are very complementary.

That Software Engineering is straightjacket and Agile is light is harder to tackle. The question is can we get the best from both worlds here. Yes, we can!

Finally, the Software Engineering camp has a set of technical practices, the different agile methods have other sets of partly overlapping practices. Can we find a way to live with both? Yes, we can!

To do that we have had to invent a new concept Practices. We don’t talk so much about process anymore; instead practices are first class citizens. Process is just a composition of practices. Instead of talking about second generation processes that are passive and monolithic (big process), we talk about a third generation process which is active and practice-centric. If you already now want to have a peek at a third generation process, please go to my web site www.ivarjacobson.com.  

The weekend July 8-9, Dave Thomas visited me at my country house in the Stockholm archipelago. Some of you have already met Dave. He is a very dynamic and colourful Canadian. He is very popular in some camps and he is feared in others.

His great strength is really finding innovative, competitive solutions to people's business problems which can be solved by software; and assembling and motivating people to move towards a goal. He looks many years ahead at the business/competitive climate and navigates a path through to a goal. He uses the best things he can find to solve the problems and doesn't generally constrain his solutions to the practices or technologies that are popular or widely available. If everyone uses the same process, technology and tools then he doesn't see where one gets competitive advantage since everyone can hire smart people. Hence he seeks strategies which others don't see and competitive opportunities to use them. This is why so many companies have problems today including MS, IBM but also companies like Fujitsu, Samsung etc. Unlike Google for example they did not think out of the box, they just run hard. MS has a hard time to understand enterprise clients, and IBM doesn't really understand how to build and deliver software as a service. Smalltalk was successful because Dave made IBM do it (The IBM Smalltalk), and Eclipse is successful because Dave created a strategy to disrupt the tool market. This is his real strength, managing software is something he does for fun. :)

The first time he visited my country house was 1988. At that time I had founded Objectory a year earlier and we discussed what I should do. He strongly believed in my vision and technology. He gave me some advice which included the following:

You should write a book about use cases and objects – no more that 150 pages. Then that book will become a market leader. He saw my hesitation. I said something like software development is serious work; you can’t describe that in 150 pages. He responded, of course you can’t, but people want simple solutions, they don’t want complex books. And, then you can come out with an improved version every year, and people who bought the first version want to by every other version. :) This is how you create market leadership. Of course, I should have done as he suggested.

Dave and I had a great time together at my country house as you can see from the photo below.

We did some good work together on EssUP and NGP as this picture shows: 

He likes our work and he will help us communicate it in the software community. I will write again about the specific results of the meeting which will come in the next few months.

Rational’s Software Development Conference 2005 is over. It has been a very exciting week. I must admit I am a bit exhausted. The days were packed meeting customers or giving presentations. The nights became very late having fun meeting friends – both old and new.

Jaczone and Ivar Jacobson Consulting shared a booth and it is no exaggeration to say that we got a lot of attention. “I was really taken with the crowd at the spotlight theatre presentation by Dr. Ivar Jacobson and the demo of WayPointer. Standing room only and it wasn't even lunch time!" said Buell Duncan (General Manager, IBM ISV & Developer Relations). At this theatre presentation Tata Consultancy Services announced their work with WayPointer to make their own process active. TCS reported a more than 20% productivity gain (!), increased quality and ease of learning using WayPointer.

Jaczone and IJ Consulting gave a total of 5 presentations, all very well received. My own presentation was titled “Beyond Agile: Smart” – a thought-provoking title. This is a talk that has been cooking for more than a year. I still was not sure about how it would be received.

- o -

Today everyone wants to be agile. It is pathetic to see how basically every speaker has added the word agile to the title of their talks. I haven’t done it. I haven’t done it, for two fundamental reasons.

All my work over the years has, as I soon will explain, strived to make software development more agile.

1. Agile is not enough, we need more. I want agile+++.
2. What I actually want is to get smart software development. That is why the title of my talk was “Beyond Agile: Smart”.

Thus we all agree that we need agile software processes. We all agree with the four statements in the agile manifesto. We agree on many agile principles such as iterative development, continuous integration & testing, use only what you need, etc.

However, we have different ways to get there. The so called “agile methods” primarily rely on tacit knowledge. Tacit means implicit knowledge (achieved ad hoc and undocumented). The Unified Process relies primarily on explicit, structured knowledge. This is a big difference that has not come through in the debate. In previous postcards I have discussed this difference, see for instance my postcard from 2004, September 30, Minneapolis.

This postcard is to some extent a repetition of what I wrote then but I will also talk about some new stuff. I will clarify the difference between being just agile and being smart with four new manifesto-principles. I will extend the concepts of pair programming, as in XP, to many other kinds of pairs such as pair architect. However these pairs will be virtual pairs assisting you with knowledge and making you smarter. We will get there soon.

A process or a method being agile to many means that its definition (description, book, web site, etc.) is light, i.e., it is sketchy. Thus when working according to it in a project, you have to use tacit knowledge and therefore the project is agile. This may be true in many cases, but in larger organizations you usually have a lot of people with different tacit knowledge. This creates a lot of difficulties working together. People need guidelines in the form of explicit knowledge to work consistently and create good software. Thus, a very important insight is that: 

   A light process may make a project heavy.

We also know that too much explicit knowledge such as in the Unified Process (UP), can make projects heavy, if they have to select what to learn, learn it, apply it and update it with new explicit knowledge as they learn more.

However, if we in some “magical” or say smart way dramatically can reduce the work to select, learn, apply and update the knowledge in UP, the situation will be different. If we can deliver the knowledge you need, and only that knowledge, and exactly when you need it and not before, then the size of the process doesn’t matter. Whether the process “book” is 100 pages, 1,000 pages or for that matter 100,000 pages will be irrelevant for ease of use. Thus, the more the better! You will only get just what you need and when you need it. And the bigger book, the more real on-line mentoring you will get.

This is what we can do with intelligent agents. Every developer has an online agent, or as we say, a virtual mentor. The virtual mentor is able to select what you need know, teach you exactly that, help you apply what you learnt and learn itself from your experience. With WayPointer we have proven that this technology really works. Much more will happen in the years to come, but already today, as experienced by TCS, substantial increases in productivity, quality, user experience, etc. can be made.

XP talks about pair programming. With these virtual mentors we talk about:

• Virtual Pair Programmers
• Virtual Pair Analysts
• Virtual Pair Designer
• Virtual Pair Tester
• Virtual Pair Project Managers
• Etc.

A process supported by intelligent agents, such as the Unified Process with WayPointer, is much more than an agile process. It is a next generation process. It is beyond both agile methods and traditional processes such as UP. I call such a process a smart process.

A smart process is agile (as defined by the agile manifesto) but it is also more. I have formulated four manifesto-principles to define a smart process and I call them Manifesto for Smart Software Development:

• Make well-known knowledge explicit over keeping knowledge tacit
• Active process over passive process
• Let models be code over letting code be models
• Team capability over dependency on individuals

I am not sure that these four principles will be the most important, but they work for now. I can’t see that methods or processes that primarily rely on tacit knowledge ever can compete with smart processes. Together I hope we can make the software world smarter and not just agile.

If you want to keep up with how our work on smart methods evolves, sign up on www.ivarjacobson.com and we will keep you informed. 

September 25, 2004

Coming back after many years to a place where you previously have lived is a special feeling. I spent a year at MIT 1983-84 as a visiting scientist. I was awarded a grant from the Marcus Wallenberg Foundation in Sweden to study abroad for one year, and I spent this year with my whole family in Boston. This was one of my most important years from a career point of view. I learnt to know a lot of people, some of them were professors at MIT. They were important references for me when I started my company Objectory in 1987.

Thus, this week when I attended the Software Development Best Practices conference in Boston, I also decided to visit MIT and see if I would find any of my old colleagues. I went to the new computer science labs which are located in a new building with a very impressive and advanced architecture. I will not attempt to describe it. I looked for names from my year at MIT. And I went to their offices to say hello, but most of them were out. I found Hal Abelson, who was one of the two teachers of the famous class: Structure and Interpretation of Computer Programs. That was and still is the best class I ever have taken. It was so great that I actually brought it back to Sweden as VCR tapes and it was taught at two universities in Sweden for some years. We were chatting for a while, he recognized me very well. “You were working with usability, right”, probably thinking about use cases. Anyway, I didn’t dig deeper into that.

At the SD Best Practices conference, I gave a keynote on Active Software. The last couple of years I have promoted this new idea as one of the biggest mega-trends on the horizon. A book is underway, and I have also devoted some previous postcards to this idea.

However, I am not going to talk about my keynote in this postcard. Instead I will devote this postcard to agile methods. These have become so popular for interesting reasons which I can’t resist reflect on.

Many talks had the term agile in the heading. Many old talks have been modified to motivate the term. Every author and presenter needs to make clear they are agile. So they join the club. I have joined the club too. I wrote a paper at the end of 2001 with the title: A resounding Yes to agile processes – but also to more. Of course we should be agile, there is absolutely no conflict. For me this has always been the goal of everything I have been fighting for. However, we have different ways of getting there.

Before getting there, let us look at the values of the Agile Alliance:

1. Individuals and interactions over process
2. Working software over comprehensive documentation
3. Customer Collaboration over contract negotiation
4. Responding to change over following a plan

Who can disagree with that?

1. Of course, people are more important than any book, even if it is a big book on the web. Books don’t produce software. However, knowledge is important to make people successful.
2. Of course, with working code your customer is happy, even if the design and architecture is expressed in UML. However, working code needs to be understood and maintained after the initial team developed it.
3. Of course, software requirements can’t be specified upfront. They grow as people work on growing and experiencing the software iteration by iteration.
4. Of course, detailed planning at the outset of the project is cheating the customer. Software development is a change process and the project should be smart about accommodating changes.

Of course, we all agree with that. So what is then the difference? There is a huge difference, but before answering that question I will give my opinion on why agile methods have become so popular? There are several reasons:

1. UP (in particular RUP which is the IBM Rational version of Unified Process) has become too large and complex. This is true, but it also leads to a misunderstanding. It is true that UP is large and comprehensive, but the intention was never that everyone should learn everything about UP. Instead, a small subset of UP relevant for a project is selected, and this is what the project members will be trained in. Thanks to a new class of tools based on intelligent agents, the problem of UP being overwhelming will soon be history. WayPointer is such a tool.
2. No really good tools from major vendors. We tend to forget that the modern software tool industry is less than ten years old. We have not really yet seen a tool environment with seamlessly integrated tools. My old Objectory tool from 1994 had a lot of features and great integration between different models not yet seen in any other tools. The problem was that the tool was developed in Smalltalk and that never became a commercial platform. However, we will very soon see a new generation of tools coming to our market place.

This was of course a very natural basis to grow discontent and there was room for something new. That new had to be something else than technology, because UP had that pretty well covered. The truth is that Agile methods have not contributed anything exciting and original from a technology point of view. Test first design is maybe the most interesting new idea, but it is similar to use case driven tests. From a technology perspective most ideas in agile methods are 20-30 years old.

So again, the new had to be about something else than technology for it to be missing in RUP (or rather perceived as missing as much of the debate is a matter of perception).

1. Agile methods have added something new.
   They have added best practices on the so called soft factors in software development, such as pair programming, on-site customer, 40 hours work week and so on. We programmers have been drowning in technology and our work environment has had low priority. We like this new interest for the soft factors.

However, this in it self wouldn’t have made agile methods popular. I believe that the most important reasons for the popularity of agile methods are two:

1. Many experienced developers felt side-stepped by the success of UP. They felt that their hard earned experience became less valued by management. This is of course a management mistake as a successful process adoption requires both knowledge (UP) and experience (people). Successful adoption typically happens when the most experienced architects in an organization are part of the change effort. Leaving them on the side-line is begging for problems.
2. Another reason that these methods have become so popular is that they don’t require any specific or defined knowledge. Agile methods stand for "tacit knowledge". For experienced people this is of course exactly what they want, no one can tell them what to do. They can accept some general principles but no detailed suggestions are popular. For the inexperienced people, this is of course wonderful. They can also do it with whatever knowledge they have. One uses what one knows, quite simple.

In reality agile methods require very competent programmers to become successful. This is clearly expressed by its originators. In fact they require more knowledge than what a project using the UP requires. The problem is of course that there are not enough good programmers around the world. And those who are that good are very attractive, so they move around quite extensively. This is very different with UP. Usually once people have used UP for a project, the project is less dependent on any individuals (still individuals are important). The UP provides a lot of explicit knowledge, and it doesn’t rely on tacit knowledge only. We can educate in UP to a deep degree, but you can’t educate in tacit knowledge.

In a way I think these methods that now have introduced a concept for tacit, unknown, unstructured or undefined knowledge, takes us back 25-30 years. Let me explain what I mean more explicitly.

1. In the beginning there was chaos.
2. UP was meant to solve/control 1. Unfortunately it has had some negative side-effects as we discussed earlier.
3. Agile methods are removing the side effects by moving back towards 1.
4. We think that we instead should remove the side-effects by moving forward to third generation processes/active processes., i.e., instead of going back to generation 1.5.

Thus so far the software industry can be described as based on tacit knowledge. As a result we have got a poor reputation for quality of software. This distinguishes the software industry from other modern engineering disciplines. And those are not based on tacit knowledge! We too can’t continue base our future on tacit knowledge.

By now you know why I think agile methods have become popular. It has happened because we didn’t do a good job with UP and tools. This created discontent and was a good ground for making excellent programmers raise there voice and for young people to feel that they could do it without a lot of knowledge. Thus I view agile methods as populist methods. In my next postcard, I will describe how these approaches if not changed dramatically will again loose in popularity because a new class of tools will make them anti-agile in comparison with what will come. What will come will be agility in many more dimensions than today. I will talk about what will come next. Beyond Agile!  

July 23, 2004

Another Rational User Conference has just been concluded. Since 1997 I have attended every one of them, now eight conferences. There were many good talks, but I had no time to attend more than the opening keynote by Mike Devlin. Instead my days were filled by meeting people. Many had heard about my recent adventures in Singapore and Korea and wanted to understand where I am heading. Basically, I see a huge need to help people around the world to apply best practices of software development. Even if we have developed the RUP, we have not been successful in getting people to adopt it and apply it. One of the consequences of this “failure” is that agile methods like XP have become popular. I want to change that, and since RUP not yet has penetrated East Asia, this is a good place to start.

My keynote this year was titled “What you didn’t know about RUP”. So this talk was about the secrets of RUP. Actually a lot of people, including RUP evangelists, don’t know about these secrets. One of the secrets is that RUP or the Unified Process in general, is agile and is in fact becoming increasingly agile as time goes by. I feel that we have not succeeded in explaining why it is so. I took it as my mission to do so.

First I developed some vocabulary to explain why RUP already is agile. When people talk about a process being light or heavy, they actually mean that the process definition (the “book”) is light or heavy. They don’t talk about the process in the meaning of what actually is done and created when you run a project. So we actually have two concepts to discuss, the process definition (PD) and the executing process (P). What we really care about is that the P is agile and not really that the PD is agile.

Thus we can have a light PD (a book) but since it doesn’t explain much it is not of much support to the team that runs a project. They will have to invent the particulars of the process as they go. This may slow down the project substantially, something I have experienced in many projects. Thus a light PD frequently result in a heavy P.

Also, although a rich PD may require that people spend time upfront to learn what to do; it may result in that the P works very smoothly. Everyone in the team knows what to do, when to do it and how to do it. Thus a rich PD frequently results in an agile P. I have seen this happen many times in serious commercial software development.

So far most people agree with me. So let us now talk about the executing P and how to make it agile. Today, to get a PD like RUP to be used in a project, it takes three things:

1. A subset of the PD has to be chosen and maybe some additional process steps will be added. There are good tools to do so, but still this slows down the project. To be able to select what to use you need to know all that can be used. Most companies don’t have the luxury of time and competence to do that. Makes the P heavy!
2. Once a proper PD has been created, it has to be adopted. This means that people have to learn it, they need to get it into their heads. This requires training which takes time. Makes the P heavy.
3. Finally, while running a project, the PD must be applied consistently and effectively. This means that people need to know what to do, when and how. Usually, people can’t do this by themselves, it is a far too big step to go directly from the classroom to your office and do useful things in a productive manner. Thus projects need mentors. Makes the P heavy.

Why would people take on all this work (to define and learn a PD) to develop software? There is only one good reason. We know that people will develop better software, better software from all perspectives.

What have we done to make RUP implementations overcome these difficulties?

1. Now and even more in the years to come, I believe we will see a number of predefined variants of RUP become available. We will have ready-made light PDs, and we will have heavy PDs for product-line developments. This makes RUP more agile. You start using the PD that works for you.
2. With RUP comes a lot of training material. RUP comes with a set of very systematic models, enforcing traceability and consistency between them. This makes it very systematic compared to the light methods, and consequently we can train developers in a systematic way to understand good software development practices. We can train people to become competent. Still, yet training needs to be done in a more agile way.
3. Good software and software developed in an agile way can only be created by smart people. You can only be smart if you are competent. Since RUP helps you to become competent, you also have the preconditions to become smart which helps you to become more agile in applying best practices.

However, we won’t stop here. Already when I started the development of what became RUP back in 1987, I knew we need to take one step more. We need to make the process active and executable, not just a dead book. We need to support the developer with “intelligent” tools. This is why we started Jaczone and developed WayPointer. WayPointer will help us make a big step forward in agility:

1. One day I believe WayPointer will help configure the PD while you are working. WayPointer will recognize what the team is doing and present a PD that is just enough of process. For a small project it will be a light version of RUP. For a larger project a richer version of RUP will be provided. This will happen as you go, without having a heavy upfront configuration activity. Isn’t this agile?
2. With WayPointer, training doesn’t happen in classrooms but while you work. Thus no heavy training before starting, instead “train as you work”. Very agile, right?
3. Finally, WayPointer will help you to also apply the PD. It is very context sensitive, so it can give proposals of what to do in every micro step as you work. It can facilitate the doing of many activities which today have to be done manually, but which should follow one another in a natural way. It speeds up the whole project in a substantial way. This is what I would call agile, and it far exceeds what you can do with what is called agile methods.

To summarize, agility can only be delivered by people, people that are smart. People can only be smart if they have knowledge. Knowledge needs to be shared between the members of the team; otherwise it will slow down the team. Most of the knowledge they need are rules about best practices and implementation technologies. Most of these rules can be captured in a “tool” like RUP, but to achieve substantial agility, these rules should be manifest in another more intelligent tool, making the process active. In that way there is no limit to how agile we will be developing software in the years to come.