Complexity and Unified Process by Ivar Jacobson

September 30, 2004 

Minneapolis was the very first city on the US main land that I visited. In 1981 a delegation from Ericsson went here to evaluate some technology from Sperry-Univac. I don’t remember more than that I got very bad flu on the flight over, and that I stayed in bed for most of my visit. Well, not entirely, because one evening our host took us out to a dancing-place. For a couple of hours my flu was gone, and I had fun.

This time I came to Minneapolis to meet a customer that is deploying UP broadly. I was also invited to speak at the Rational User Group in the Twin Cities. This was a nice event with quite a large number of people. After my presentation, some people took me out for dinner. During the dinner I told them about my dancing experience 23 years ago. I asked: “How is it today, do you still have good dancing places?” “Sure, let’s show you.”

Thus we left the restaurant, and spent the next hour walking from place to place to check out the dancing scene. Place after place was either closed or there was a big guy in front of the entrance, saying: “You are not going to like it!” We were not young enough.

However, eventually we found a place. The dancing hadn’t started yet, so we watched a show with pretty girls. A tall, big woman with a mike came up to us and interviewed us. At the end she asked “Are you all straight? Or, do you want to try something new?” We were at a gay bar. The pretty girls were boys. The only place open for dancing was a gay bar. Well, we didn’t feel uncomfortable. People had fun, and everyone respected each other so we enjoyed the music and the drinks. However, it is no overstatement that we didn’t dance much.

- o -

In my latest postcard from Boston, I explained why I think agile methods have become popular. It has happened because we didn’t do a good job with UP (or other defined processes) and tools. This created discontent and we got agile methods -- or populistic methods. Agile methods don’t provide any defined knowledge. Instead they rely on "tacit knowledge". You can do it with whatever knowledge you have. This was of course also very popular.

However, the basis for this discontent will soon be gone. It is just a question of time, at the most two years, before we will have substantially better tools from leading tool vendors, and they will be better integrated. On top of them we will get intelligent tools (WayPointer is such a tool) that remove a lot of the no-brain work that programmers spend their time on today. These tools will make it impossible for ‘tacit knowledge’-based methods (read agile methods) to compete in terms of agility. They will be even less competitive when it comes to quality in measures such as productivity, quality, time-to-market. Quality includes good architecture and design.

Moreover, the complexity of the UP will be eliminated through these intelligent tools. Let me take you through this in smaller steps:

  • Why is UP complex today?
    One reason is that UP is a rich knowledgebase. This knowledgebase defines a set of best practices, and their interrelation. The goal is to have all the best practices that a team may need available easily.
  • What is the problem with this knowledgebase?
    The problem is that it can be overwhelming. This was never a surprise to me. I have known for many years that this is how people perceive the UP. I have been crystal clear for all these years that the only way to really implement UP is through substantial training and mentoring. My assumption was that only a small fraction of the software developers would be able to adopt UP properly. I have been clear when talking to people interested in UP that if they are NOT willing to do these investments, they shouldn’t even bother to think about UP.

Another problem is that even if UP is rich it is not rich enough. This may sound as a paradox, but it is not. I believe that one of the major problems with UP is that it is generic. Since UP is a book, it can’t be specific to a particular application or domain. Special cases or exceptions have to be treated very superficially. Otherwise no one would read it. This means that the process is explained generically and that examples and such probably come from a problem space of no interest to a particular reader. Examples and advice must be particular and context sensitive to be useful. However, this problem can also be solved.

  • Why did we then develop this rich knowledgebase?
    The goal has all the time been to achieve agility + more. My belief is that the more people know about software best practices, the more agile they will be able to work. However, I also wanted more as I will tell in a minute.
  • How can we help people adopt and apply this rich knowledgebase easily?
    Through a completely new class of tools, based on intelligent agents, the knowledgebase in UP can be served in very small chunks. The minimum knowledge needed in a particular detailed context will be served. Not only so, it is presented in context of the actual system that the user works with, instead of generically. The rules in UP may trigger and defects may be found, reuse may be suggested, next steps proposed, solutions carried out. The tools will serve as a motor for each individual and for the team as a whole. WayPointer is for the time being the only such tool. With such tools, the programmer can use his or her time to what is creative instead of doing "no-brain work". Today 80% of the programmer’s time is spent on doing no-brain work. This number is based on projects using agile-like (or code centric) methods. With WayPointer this time is reduced dramatically.

We all want agility. The goal is the same for all of us. However we have different ways of getting there. The biggest difference (and maybe the only really important reason) is that agile methods assume tacit knowledge, whereas UP assumes explicit (defined) knowledge served through intelligent tools to make the software process active.

Why is explicit knowledge so important?
Moreover, I strongly believe that good software is developed by people being smart, and that you can only be smart by having critical knowledge – and of course having this knowledge turned into experience and skills. In our world the critical knowledge is on best practices in software development. This requires explicit knowledge available to the whole team; tacit, unstructured knowledge which only is in the heads of individuals is not sufficient. This is why UP serves a very important role; it includes much of the critical knowledge you need and intelligent tools can serve this knowledge when you need it.

I believe that it is not enough to be agile. The term agile doesn’t imply anything about whether the software is good or not. I have always worked to get good software (good in terms of all measures including good architecture, good test cases, good interfaces, good components, etc.). Furthermore, it has all the time been my belief that if you know how to make good software, you can always figure out how to make it fast or agile. This is what we have been doing with UP + intelligent tools. Thus instead of talking about being agile or about agile methods, I would rather talk about being smart and smart methods. Smart means agile, but also much more. It means all the things we need to get good software. UP with intelligent tools helps you to become smart.

Next year, the world will be looking for a new buzzword. Maybe I should launch the term Smart Methods. Maybe I should give a keynote with the subject: "Beyond Agile: Smart". Trust that Smart Methods are building on top of defined knowledge with smart agents.

What do you think?