For many years I have written postcards to those of you who have subscribed to them on our web sites.  I will discontinue writing the postcards, but instead talk about the many different aspects on software development on this blog.  And I am not the only one who will write for you.  I am so lucky because I am surrounded by world-class experts coming from around the world.  These people will publish notes here.  Thus, you will here from Kurt Bittner, Ian Spence, Svante Lidman, Pan Wei Ng and several other people.  I will introduce them as they come forward.We will use this blog to talk about our visions and our results.

I like to dream about the future, but I also would like to reach that future at some point in time.  I believe that when it comes to software, dreams only come true if the dreamer drives the realization of the dream.  Too many people have dreams.  Very few actually have the guts to make their dreams a reality.

In 1992 I wrote an editorial in IEEE Software that started like this:  “We all have dreams. Just a little over 25 years ago, I had a dream. I dreamed that the technology known today as "object- oriented" could be used to design telecommunications systems. I dreamed that this technology would be generalized and given a wealth of wholly different applications. I dreamed that the computer scientists who - in those days - ridiculed this technology and saw it as a dead-end street would one day accept it. I dreamed that this technology would in time become an indispensible element in the mainstream of computer science.”

In my summary I wrote: “Our long-term goal (which I hope we can achieve within another 25-year period) is to become as industrial in our handling of computer techniques as engineers are within their various disciplines. This requires, among other things, development of the architecture in large- scale object-oriented systems; it requires that the development process support project management as well as product management; and it requires that we find other, more sophisticated languages - specification languages as well as programming languages - compared with those we have today.”

Now 15 years after I wrote this article, it is interesting to see how far we have gotten.  We have got UML which is a standard modeling language for software development.  Before UML we had lots of different notations with almost no semantics.  UML basically took all of these notations out of the picture and replaced them with one single notation with reasonable well-defined semantics.  It is not perfect but it is practical.  It is being applied around the world for all kinds of software and for all kinds of models including business, requirements, architecture, analysis and design. 

When it comes to process and methodology, we have also made substantial progress.  When I wrote the IEEE article there were more than 26 published methods identified by the OMG (Object Management Group).  The most successful one of these methods (and tragically basically the only one that really survived) was Objectory which became the Unified Process. 

In parallel with the success of the Unified Process, new kinds of methods have come into play.  The most successful of these other methods are labeled agile.  However, the meaning of agile has been confused by its proponents.  What really is unique about agile is that it cares about people.  Thus it is about social engineering.  No process has ever developed software.  Only people develop software.  They may get help by tools.  They may get advices from process or methodology, but eventually we must make our people motivated and competent.  Nothing is as effective in getting software quickly and at low cost as having motivated and competent people.

Given that no methodology has everything you need, and even if it has most of what you need, it won’t be the methodology you want to use in ten years time.  New ideas and new technologies will come into play.  These will come from individuals and companies throughout the world.  If you try to extend your methodology with these ideas, it will grow until it collapses under its own weight. Instead of publishing branded complete processes or methodologies we should focus our skills on single practices.  Usually a methodologist is good on one or a few practices, but not on all practices that a team needs when developing software. 

Now it is time to formulate one of my new dreams.  My dream is that great people around the world will contribute their best practices to the world maybe in the form of a community.  These practices can be of different kinds.  They can be technical such as those from the Unified Process camp, or they can be social engineering practices from the Agile camp.  A team can then select from these practices and compose the selected practices into a way of working that helps the team develop good software, quickly and at low cost.  This is my dream.  Now, we are not just dreaming.  We have come a long way in realizing this dream.  We have developed a practice platform EssWork where you can find a whole set of practices, you can author your own practices and you can select the practices you like and compose them.  There is still a lot to be done before we are finished, but with your help the dream will come true.

Cheers

Ivar 

One of the most popular movements in software development in recent years is the move toward agility. Today everyone wants to be agile. That is good! However, the essence of being agile is being smart. I have for several years expressed that the most important character you need to have to be a great software developer is to be smart. In several of my columns I have summarized what you need to be successful by saying: You need to be smart! What does that mean? Most people know intuitively what “being smart” means in everyday language, but what does it mean for software.      

It is not smart to model everything in UML for instance when building software. It is not smart to model nothing and go straight to code. It is however smart to find exactly that something that is of importance to model and code.  

Advice: What is that something? It is about the most essential use cases and in particular about the most essential scenarios through these use cases. It is about the components and in particular the parts of those components that realize these essential scenarios. Thus, it is about the essentials. Now you may ask what makes a scenario essential. An essential scenario is the response to the question: “what are the most important scenarios the system is supposed to do”. Which scenarios exercise the critical parts of the architecture? Which scenarios need to work in order for us to say that the highest technical risks have been eliminated? It is not smart to write requirements without caring that these requirements are testable. It is smart to make sure the requirements also are test cases. 

Advice: try use cases since they are also great test cases. It is not smart to work in a waterfall manner, first specify all requirements, then do the design and the code, and finally test it all. If you do, you will discover serious problems with performance, architecture, usability too late. It is smart to first build a small skinny system and then build a little bit more, and a little bit more, before you release the system to customers. Each time you build something, you must be able to run, validate and test it. 

Advice: use a controlled iterative approach such as the iteration practice in the Unified Process or sprints in scrum.It is not smart to run off and build a lot of “stuff” before first assessing if you can source the whole or parts of the application (Open Source or commercial offerings). It is not smart to develop software with a process that cannot scale if your system is successful and customers want much more. It is smart to use a way of working that is no more than what you really need, but that can grow as you succeed with the product. 

Advice: make sure your process has a dial for each interesting process parameter so you can turn the dial to the proper position for your project.  It is not smart to throw out your existing process and adopt a completely new process. That is doomed to fail in most cases. Not everything you did in the past was wrong so why should you start all over. It is smart to improve your process in small manageable steps.  

Advice: add one practice at the time. It is not smart to find a shortcut that in reality becomes a detour, such as skipping requirements and going straight to code.  It is smart to do enough requirements to find what to use to build your first increment, your skinny system. In general it is not smart to be extreme in what you do such as: model everything or model nothing, follow a strict waterfall process or an unstructured iterative approach, throw out what you have and start all over. It is smart to be balanced to do what is needed right now but with an eye to the future.  Above I have given a number of examples. In each case, there are some ideas on how to think about being smart. And each case can be expanded further. You will become smart with experience, but experience on the other hand is not a guarantee for being smart.  Of course eventually, it comes back to you. Smart is not the same thing as being intelligent. You can be intelligent without being smart. And you can be very smart without being very intelligent. Smart is not the same as having common sense. You can have common sense without being smart, but if you are smart you must have common sense.  

Smart is to do exactly right, not to find a broad solution that is just about right 

Today everyone talks about Service-Oriented Architecture. And as with all buzz words there are many interpretations. As the name suggests SOA was originally about architecture, but as time goes by its proponents put more and more meaning into it in the same way as proponents of EA put more and more into it. This is why some people have nicknamed SOA to Service-Oriented Ambiguity.      

What is SOA at its core? It is a kind of software architecture which allows you to connect course granular components called services through well-defined interfaces. These components can reside on any kind of platform (mainframe, client/server, etc.). They can be web applications, java or .NET applications, ERP systems or your own legacy systems.  

What is the business case for SOA? Since, services are essentially reusable components, you can efficiently support new business processes by using old services and interconnecting them. Since, they have well-defined interfaces they can be replaced by any other component complying to the same interface. 

Is this really new? Yes and no. No, because SOA has existed almost forever in the telecom space. The most successful commercial product ever built in Sweden was the Ericsson AXE switch. AXE’s success came to some extent from that its architecture was service-oriented. However, Ericsson had to build its own infrastructure - a unique computer architecture and operating system. My Software Reuse book (Software Reuse: Architecture, Process and Organization for Business Success, ACM Press) from 1997 is essentially a book on SOA. Yes, it is new because big vendors like IBM provide an infrastructure that allows services to reside basically anywhere in a network. 

How to succeed with SOA? It goes without saying that you need good people – leaders as well as champions. And they need to be smart – have common sense, practical, do what is needed not more, and of course be competent. Having said what is obvious let’s move to a more techie perspective:  

Design top down, bottom up and then find the balance as a trade-off between value and cost. Top down means you start from some business process ideas and deduct a solution as a set of interconnected components. If you just go top down it will usually be unreasonably expensive. Bottom up means that you start from what is available/reusable, such as legacy systems, web services, ERP systems, whatever, and build something close to what you need by interconnecting these components. This may not be exactly what your business modeling people were dreaming about, but something you can build quickly with low costs. Finally, you balance the top down and the bottom up and get what you may call a good enough solution. Your solution reuses a lot, but you may also have to build something new. This is smart! However, it is not enough. 

Do this very light (don’t major in modeling or post-its), develop quickly a road-map, build in small executable steps (even if the enterprise system is huge). And beware of the service companies that want to sell you big solutions – lots of billable hours without being accountable for delivering executable software!Don’t fail with SOA. It is so incredible expensive, and there is no reason you should fail. You just have to be smart ;-)  

Enterprise Architecture failed big way!

Around the world introducing an Enterprise Architecture EA has been an initiative for most financial institutions (banks, insurance companies, government, etc.) for the last five years or so, and it is not over. I have been working with such companies and helped some of them to avoid making the worst mistakes. Most EA initiatives failed. My guess is that more than 90% never really resulted in anything useful.

Why did people fail? There are many reasons, but they can all be summarized by the word smart. They were not smart when they selected solution. They were not smart when they selected way of working. They were not smart when they organized their business and IT resources. Building an EA is not rocket science. 

There are two common reasons specific to EA failure:

1. Focus on paper-ware instead of executable software.  When enterprise architects work in an ivory tower without caring about what can be implemented, they produce too much models and documentation without executable solutions.  Enterprise architectures should be implemented incrementally, starting as early as possible. We call such architectures for executable EAs. 
2. Big gaps between layers instead of seamless relationships.  Usually there are several layers such as a business layer, an application layer, a data layer and a technical layer. There are huge gaps between these layers which results in very brittle architectures. It is like trying to stand on a skateboard which is on top of another skateboard which in its turn is on top of yet another skateboard, etc. To have a chance these skateboards need to behave like one which is hard enough.  Thus the relationship between the business layer end the application and data layers are not straightforward and the relationships between the application layer and the data layer is as hard to manage as it was 20-30 years when we used methods like functional decomposition, or structured analysis and design. It is amazing that people haven’t learnt anything from component based development (with or without objects).

There are many other mistakes that people have made, many of which are related to organizational change in general. Examples include lack of business support for EA, not communicating the scope and purpose of EA, no strong IT leadership etc. In addition to these common challenges, all it takes to succeed at EA is to use best practices for modern software development, avoid upfront academic modeling, build both top down and bottom up, look upon the whole enterprise system as a system of interconnected systems.

Many of the companies that failed are now looking for the next silver bullet – Service Oriented Architecture SOA. To me SOA is what EA should have become. SOA can be described as EA++ -- it is Enterprise Architecture made better. SOA is clearly on the right path, but again adopting it requires that you work smart!  

Right now I am in my home in Switzerland to take a break and ski. I ski during the days and I work mornings and nights. Sleep takes too much time so I do as little as possible of that.:) One of the questions I often get nowadays is “Does Process come from the top or from the bottom?” Before answering I will start with two questions that I usually ask during my presentations. “How many of you love process?” In a group of 750 people, I usually see 3-5 raised hands. These are of course the people who work with process implementation, and they don’t have to follow a process themselves. Then I ask provocatively “How many of you hate process”. Now I get a very different answer. Usually 60-70% of the audience raise their hands!

Why is it like that? I believe that in many larger companies process has been pushed down on the people who know how to develop software. This won’t work anymore. The competent will speak up to get the process they want, which is good. What worries me though is that the pendulum will swing over to the other extreme. Instead of process push-down we will get process pop–up.

People do as they want. If every team gets their own process, they will not be able to learn from other teams, they will not get common set of tools and they will not harvest reusable assets. What we really need is a balance between push-down and pop-up.

For a very long time (since 1987) I have told my clients that one of the most important success factors for process adoption is to identify both a sponsor and a champion for the process improvement. The champion is a developer with many years of experience, great competence and with high credibility among his comrades. The sponsor is a higher level manager who wants to improve the way his organization develops software to support the company’s business. Without support from both don’t bother to adopt any new process.

Another of success factor is to use experienced people to coach the team adopting the new process. However in many cases management wants process but doesn’t want to invest in coaching. They just hope that people will adopt the process offered to them without proper coaching. As a consequence a lot of organizations have failed in process adoption. Management tried a shortcut, but it became a detour. The money an organization spends on product development is a factor of ten higher than the money it spends on proper coaching. For twenty years I have told my clients that without adequate training and coaching don’t bother to adopt any new process.

A third success factor is to balance the need for process from the top (the management) and from the bottom (the developers). This is true for any process whether it is a software development process or a business process. Only working from the bottom is a very slow process, and working only from the top creates a lot of resistance. Unfortunately, in many organizations management pushed down process on people counter to our recommendations. The developers were not enthusiastic but they tried to adopt the new process. That was not easy. Management believed their people were using the process but the developers did what they wanted to do, usually what they always had done. We got a gap, sometimes a huge gap, between what the process said and what people actually did. We call it the process-project gap. Thus we got failures and people started to hate process.

We need to change this. My recipe contains four ingredients:

1. Reduce complexity. Instead of adopting a big process consisting of many practices, adopt a practice at a time. This will significantly reduce the complexity of process improvement.
2. Balance the push-down from the top and the pop-up from the bottom, instead of just doing one or the other.
3. Allow teams to change the practices so they work in their projects. The way you really work must be fed back as a change to the practices. We want a process that is alive and not just a dead book. This allows us to close the gap. One challenge here is of course that traditional process hard-heads will demand that all deviations from the described practices must be documented and perhaps even be given formal exemptions. If that is necessary then people will feel that it is easier to do what is demanded than to do what is right. What we need is a utilitarian perspective on process rather than dogma. :)
4. Activate the practices and bring them alive in a way traditional process has never been able to do. While activated, a practice can help with all kinds of things. It can review what the developers are doing and suggest corrections. It can automate mundane tasks transparent to the developers. It can train the developers on the job reducing the need for classroom training and traditional human mentors. The selected practices should give context-sensitive help to the developers. This means that they should only give the developers help when they want it and only the help they really want and no more. The goal is to give the developers more time to think and create and to remove a substantial part of the no-brain work that developers suffer from today.

We have had a period of process push-down. The pendulum is now swinging to the other extreme – process pop-up. After a few oscillations the pendulum will hopefully swing to a new position where we will get processes that balance both the need for governance and for the need for creativity.  

Since my last postcard I have been traveling around the world talking about the Next Generation Process -- which actually is not about process but about practices.

I have been meeting customers and given seminars in San Francisco, Seoul, Beijing, Taipei, Tokyo, Boston, New York, Jacksonville, London and now I am in Stockholm for a few days. When I see that people need a break from technical stuff in my sessions, I usually give them a break.

Sometimes I ask the audience if they know where object-orientation and components were born. Which was the very first object-oriented programming language ever designed? Usually, I get some mumbling as an answer so then I go on saying: OK, I will teach you some geography and history :-).

Up there in the north (I point in some direction which is not very precise), there is a large, beautiful country that looks like a rectangle called Sweden. To the west of Sweden is a tiny, totally uninteresting country called Norway :-). It is unbelievable but the first OO language was born in Norway (created by Ole-Johan Dahl and Kristen Nygaard). It was in 1967 and the language was Simula. Simula had everything important about OO that you find today in for example Java.

At the very same time, in Sweden, components were born. Everything important about components and component-based development was used in the Ericsson AXE system. The software component idea was my first baby. Thus it is not true that Microsoft and Bill Gates invented software components.

The problem was that we in Sweden didn’t know what our neighbors were doing. Even if the geographical distance between Sweden and Norway is small, the intellectual distance is huge :-). In the minds of Swedes, nothing of intellectual value can come out of Norway. Swedes are even surprised that Norwegians have the capacity to have the reverse opinion :-).

South of Norway is another tiny, totally uninteresting country called … and I hope for an answer…yes, Denmark. Do you know what came out of Denmark? Do you know who the father of C++ was? Yes, Bjarne Stroustrop. Bjarne had studied Simula and when he joined ATT in the US he made C look like Simula and called it C++ :-).

Then, we have another country to the east of Sweden called … yes, Finland. Do you know what came out of Finland? Nokia? More? Linux, yes. Do you know who the father of Linux was. No? Linus Torvalds. In Finland about 5% of the habitants have Swedish as their mother-tongue. Linus is Finnish-Swedish :-).

Anyway, you may wonder why I tell you all this? Well, I want to point out that everything important about software was not born in the US :-). However, if you want to make money from it, you have to make the leap over the pond to the US! :-) 

There is a conflict between people who support Software Engineering approaches (such as Rational Unified Process RUP) and people who support Agile methods (such as SCRUM or XP). This is understandable since the two approaches are described in an incompatible way. However that is unnecessary, since the ideas behind the two are in all essentials complementary. The trick is how to combine them in a way fair to both of them.

In the software world we like swinging. We are swinging from one way of working to another way of working -- from one extreme to the other. Some people have been swinging for their whole life and have had enough of it.

Once, about five years ago, when I spoke in Singapore for a small group of 50 people, a senior guy stepped up, very frustrated, and told me that UML and RUP would be dead in five years and I would be gone as well. I like provocations so I asked him calmly what he based that upon. He told us that he had worked with software his whole life. “In the 60s we all worked with assemblers, then we got Cobol and Fortran, then we got database design” and he continued describing a zigzag path with new methodologies in every step: “structured programming, structured analysis and design, object-orientation, components and so on.” He felt it was awkward hearing about a new standard modeling language. I told him that there was another path: Assembler, components with assembler, component with a programming language, components with structured programming, components with objects and now components with objects and aspects. And we have all the time used a modeling language that can be described as UML.

Well, I am still here and I intend to be here for many more years…knock on wood. The guy who was so unhappy became even unhappier next day when he was fired by his boss. However, it is true that UML and RUP have lost some momentum, but they are far from dead. They are just taking a break while the world is swinging.

We are swinging from the extreme of doing everything with UML following what many believe is a prescriptive Software Engineering process. And we are swinging to another extreme called Agile. Very young people are having their first swing and now they are Agile. Middle-aged people, what a cute label, are right now swinging from the extreme of working with Software Engineering to another extreme called Agile. We swing and our companies swing with us.

Today everyone is agile. Of course. Everything else would be silly. Let me say it loud and clearly: I am a big fan of agile. My teams at Ericsson were extremely agile.

However, most people I talk to have a fuzzy understanding of what agile really is about. I participated in a panel discussion with agile evangelists in the UK last March. The organizers had hoped I would be against agile. They were surprised. The audience asked us to define agile. The panelists claiming they were agilists started to ramble. They repeatedly said that the most important property of agile was that it was iterative. That is dead wrong. Iterative development is one of the key practices in RUP and it has been around long before RUP. It was previously called spiral development and created by Barry Boehm in the late 70s. So I had to help explain what agile is:

Agile is about three things:

1. Most important, agile is about social engineering. This is what actually made agile different. It is about how to work as a team, how to make people motivated, how to collaborate, etc.
2. Agile is about lightweight. Instead of relying on explicit knowledge like in RUP, agile relies on tacit knowledge. In RUP we tried to write down what we know are good practices. However, since people don’t read process books, it doesn’t make sense to have them. Instead agile assumes that people have knowledge in their heads that is enough to develop good software. This can of course be debated but that is what it is.
3. Agile contributes some technical practices. This is the weakest part of agile. Very little is really new. Iterative and incremental development is as I already have said an old idea. User stories are a special kind of simplified use cases. Most interesting new idea is test-driven development. I am not claiming the agile technical practices are uninteresting, just saying that if it were just for these ideas we wouldn’t have been excited about agile.

As you see Software Engineering and Agile tackle different aspects of software development. Software Engineering’s strengths are about technical practices and Agile’s strength is about social engineering. Thus here the two are very complementary.

That Software Engineering is straightjacket and Agile is light is harder to tackle. The question is can we get the best from both worlds here. Yes, we can!

Finally, the Software Engineering camp has a set of technical practices, the different agile methods have other sets of partly overlapping practices. Can we find a way to live with both? Yes, we can!

To do that we have had to invent a new concept Practices. We don’t talk so much about process anymore; instead practices are first class citizens. Process is just a composition of practices. Instead of talking about second generation processes that are passive and monolithic (big process), we talk about a third generation process which is active and practice-centric. If you already now want to have a peek at a third generation process, please go to my web site www.ivarjacobson.com.  

Barcelona is my last stop on an around the world trip that started Oct 17 from Stockholm. That morning I gave a talk to key developers at a key Swedish company developing key telecom products :). After my talk I rushed to the airport and flew to Tokyo to meet some key people from large software companies. After Tokyo I went to Taipei, followed by San Francisco and now Barcelona. This is my 6th around the world trip this year with one more to go before the end of year – three less than last year. As you can see I am slowing down.

Barcelona hosted one of the largest software conferences this year, the Microsoft Tech•Ed – Developer conference and yesterday I gave, together with my colleague Pan-Wei Ng, a talk on Next Generation Process (NGP). Here is the core idea of the talk:

The world of software development is constantly changing and evolving. New ideas arise all the time and existing ideas go in and out of fashion. Software development processes find it very hard to keep up with this rapid rate of change, especially as they find themselves quickly going out of fashion or becoming bloated as they bolt on more and more information. Teams find themselves struggling as they try to mix-and-match practices from various sources into a coherent way-of-working.

A new approach to capturing and sharing experience is required. In my company, Ivar Jacobson International, we have developed a fundamentally new way of managing software development that downplays the importance of process – in particular big process. Instead practices become important. A practice is a way of working with a clear beginning and a clear end. Following a practice should give a clear result of value. Instead of trying to give an academic definition, let me give some examples: Iterative development is a practice that you can imagine to use separately to how you specify requirement or define your architecture.

1. Practices are First Class Citizens
2. Practices can be made smart to truly help the developers in their work, this is achieved with WayPointer
3. Practices can be used individually or in a multitude of combinations
4. Process is just a composition of Practices, and
5. Teams compose the process they need by selecting just the practices that they want to use

As you can see this is a paradigm shift. We move away from working with a big monolithic process (like RUP) to working with practices. Process is downplayed to just become a composition of practices.

To enable this, a number of innovations are required: innovations related to the way that practices are collected, presented and applied. We have developed two components to realize these innovations:

EssWork is a new set of tools for deploying practices in an organization. These tools allow you to author practices separately and to compose them to a process. They allow you to use a completely new way of learning, adopting, scaling up, and changing your selected practices. You use cards to describe your practices, to plan your project and to guide you in developing software. Cards are good because they force us to be very brief in describing process elements. You focus on the essentials, which makes practices very lightweight.

EssWork is available on top of several platforms: Eclipse, EPF, VSTS so you can have the same process independently from which vendor tools you use. On top of EssWork we have built a first set of eight practices called EssUP (Essential Unified Process).

EssUP is just a subset of all the practices we eventually will see on EssWork. Five of these practices are traditional Unified Process practices: use cases, architecture, iterations, components and products. Moreover, there is one practice which supports social engineering practices unique for agile methods, and one practice for process improvement derived from the process improvement camp such as CMMI.

However, we are not dogmatic about which practices you should use. You can replace any practice that we offer with a competing practice from your favourite methodology. You can also add practices that you have discovered yourself. Mix and match is our recipe.

Best of all: EssWork and EssUP are going to be Open Source! I can’t give a date, but it will happen as soon as possible.

The presentation was completely full (around 500 people) which was a fantastic result – there were people standing in the auditorium and watching on the remote screen outside So there is much to do going forward.

Various Podcasts were arranged. This is one:
http://www.mseventseurope.com/TechEd/06/Pre/Live/VideoDev9.aspx?id=3#an3  

For many years I travelled regularly to Japan. I became acquainted with many companies and many individuals. Several of my books were translated to Japanese so I got many “friends” there. However, since 2001 I have not been back to Japan, so I was very excited to return when I hit the ground in Japan two days ago.

I was invited to give a keynote at a UML conference in Tokyo. My presentation was on the Essential Unified Process with the Essential Unified Modeling Language, EssUP+EssUML. You already know what EssUP is from my previous postcards, but I have so far not described our work on EssUML.

UML was originally developed by a very tight team with Grady Booch, Jim Rumbaugh and I as members. We were nicknamed The Three Amigos – a term I never really adopted myself. Once we had defined what we wanted to achieve other methodologists from around the world were invited to participate. The core of the UML came from the work by the three of us, no doubt about that, but several other individuals made significant contributions. Grady, Jim and I had regular teleconferences in which most decisions were taken. We worked very efficiently. The first reasonably well-defined version of the language was UML 1.1 which was adopted by OMG in the fall of 1997, which was less than a year after we started to work seriously together. This must be a record for any kind of language standardization effort.

UML 2.0 was the next step in the evolution of UML and it was achieved in a very different way following standard OMG procedures. Now a committee with more than twenty persons took over. While we worked on UML 1 in an agile way, the committee worked on UML 2 in any other way than agile. At the UML World conference in Austin, Texas, June 16, 2005, during a panel discussion on UML, Steve Cook said something to the effect “UML 1 was excellent, whereas UML 2 is terrible.”

UML 2 is too big. Its specification is about 1,000 pages long! It includes many good ideas but it is written by tool vendors for tool vendors. No practitioner will read this specification.

There is a lot to be said about what happened in moving from UML 1 to UML 2. UML 1 needed a consolidation period for experience gathering and fine tuning, which it unfortunately never got. Instead the work on UML 2 started with a, too a large extent, new group of persons willing and wanting to change anything they didn’t fancy.

Grady and I were not involved in the UML 2 work. Many questionable changes were made. As an example, I found out that the committee had decided to change use cases from being what they always had been since I first introduced them to being something completely different. They claimed that I really didn’t understand what use cases were, and they had to fix it! Yes, this is a true story.

Once the decision to change use cases was taken, and I was informed about it, I wrote an email to the committee and expressed my severe concerns. I told them that I would not be able to support UML 2 if they changed the fundamental meaning of use cases. That email had no effect, the committee knew better! Thus I had to bring my concerns to top management at Rational. Rational made it clear to the committee that if changes of this nature were made, the user community would react very negatively and the reputation of the language would be seriously damaged. Rational expressed that this was unacceptable and more or less threatened to walk away from the UML 2 effort. Our two participants in the UML 2 committee were instructed to be very cautious with changes and to consult with a team of Rational experts before accepting any changes.

As you can understand this period was quite dramatic for anyone involved. There was a time when I believed I would have to publicly denounce UML 2. Fortunately, the UML committee came to their senses and I didn’t need to take such a dramatic step. Still, UML grew too much under the influence of a large committee. Today all UML fans suffer from this mistake.

However, at its roots UML is good. The basic techniques are proven as practical for many years. As an example, Ericsson could claim that it used UML already in 1969, because at that time we used component diagrams, sequence diagrams, collaboration diagrams, state charts and state transition diagrams (a combination of state charts and activity diagrams).

Thus on one hand UML has a sound base but on the other hand it has become too bulky. We know that with less than 20% of UML you can model more than 80% of all software systems. What we need to do is to extract these 20% that people need and define it precisely. We have started doing this and we call these 20% EssUML. EssUML is not a new language. It doesn’t even change UML. It is just a proper subset of UML.

EssUML is one more thing though. It presents UML in a way attractive to developers, quite unlike the UML 2 specification which intended audience is meta-modellers and tool builders. To achieve this we use a similar technique as when presenting EssUP. Every language element is represented by a card or a set of cards, and its description is usage centric (or use-case centric).

Now you may ask yourself, what are those 80% that we don’t need in the UML? In this case the expression “the devil is in details” could not be truer. The UML elements and their graphical representation in diagrams are simply overloaded with too many nice to haves that rarely are useful (or used). Essential UML is, as the name indicates, about extracting what is truly essential. What this exactly means in terms of diagram types and element types is a bit early to say but my personal opinion is that, e.g., Activity Diagrams (and all their associated element types and detail) do not qualify as essential. The experience is that it is not cost effective to develop and maintain this kind of diagram, especially if you also produce textual flow-of-events descriptions.

While they are not really defined as part of the language there is often (at least in tools) an artificial division into various types of structural diagrams, such as class diagrams, component diagrams, deployment diagrams, and package diagrams. I think this division often misleads people to believe that they need all these diagram types and that they are distinctly different from each other and that is of course not true. Anyway, going forward we will define this and also specifically what the 80% that we don’t need are. Every diagram and element type will eventually have a card describing its essentials, child elements, relationships and so on. In particular the card should focus on describing the use cases of the diagram/element.

As an example a card representing the component diagram (if we choose to have such a diagram type) could look like this:

Of course a card doesn’t contain enough information about the component diagram so we provide a guideline that goes a bit deeper. In most cases this is enough as developers will learn more as they use the diagram in practice.

The guideline is 2-4 pages and it describes the essential content of the diagram, its use cases, hints and tips and common mistakes. It also references the OMG specification and other UML books. However, in general developers don’t read much. There is a law of nature:

Law of Nature: People don’t read books

Over the years I have come to realize that even if people may buy books it does not necessarily mean that they read them. This is certainly true for manuals such as process manuals or language specifications. I believe we can get everyone in a team to read the cards. If we write really well we might get up to 25% of the developers to read the guidelines, but almost no one will read the books being referenced. Additionally we of course have to acknowledge that the Essential 20% varies a bit depending on the specific situation at hand, i.e., the underlying business reason for modelling in the first place can vary.

• Are the diagrams/models key ingredients in driving the understanding and design as we go, or are they primarily useful as documentation to, e.g., the next project?
• Is our intent to generate code for large parts of the system or do we model to simply illustrate key aspects of the system and its design?
• What is the size, complexity and longevity of the product/project?

The answers to questions such as these will of course have an impact on the process we use and follow, the amount of documentation we produce, and of course how much and how detailed we model.

As you know, I am a strong believer in that intelligent agents is the most efficient way to capture and disseminate knowledge and that is of course applicable for practices as in EssUP as well as for technological underpinnings like EssUML. I strongly believe in having intelligent agents to provide context-sensitive advice and training on line. They can help you by performing mundane tasks, recognize design patterns that you may want to apply, review your work, etc. These are some of the things that WayPointer can help you with.

My keynote in Tokyo was very well received. I was invited to come back to this exciting city and I will be back real soon. It was five years since I was here the last time. As it looks like now I will be back almost every month. This is wonderful and I really look forward to it.

The weekend July 8-9, Dave Thomas visited me at my country house in the Stockholm archipelago. Some of you have already met Dave. He is a very dynamic and colourful Canadian. He is very popular in some camps and he is feared in others.

His great strength is really finding innovative, competitive solutions to people's business problems which can be solved by software; and assembling and motivating people to move towards a goal. He looks many years ahead at the business/competitive climate and navigates a path through to a goal. He uses the best things he can find to solve the problems and doesn't generally constrain his solutions to the practices or technologies that are popular or widely available. If everyone uses the same process, technology and tools then he doesn't see where one gets competitive advantage since everyone can hire smart people. Hence he seeks strategies which others don't see and competitive opportunities to use them. This is why so many companies have problems today including MS, IBM but also companies like Fujitsu, Samsung etc. Unlike Google for example they did not think out of the box, they just run hard. MS has a hard time to understand enterprise clients, and IBM doesn't really understand how to build and deliver software as a service. Smalltalk was successful because Dave made IBM do it (The IBM Smalltalk), and Eclipse is successful because Dave created a strategy to disrupt the tool market. This is his real strength, managing software is something he does for fun. :)

The first time he visited my country house was 1988. At that time I had founded Objectory a year earlier and we discussed what I should do. He strongly believed in my vision and technology. He gave me some advice which included the following:

You should write a book about use cases and objects – no more that 150 pages. Then that book will become a market leader. He saw my hesitation. I said something like software development is serious work; you can’t describe that in 150 pages. He responded, of course you can’t, but people want simple solutions, they don’t want complex books. And, then you can come out with an improved version every year, and people who bought the first version want to by every other version. :) This is how you create market leadership. Of course, I should have done as he suggested.

Dave and I had a great time together at my country house as you can see from the photo below.

We did some good work together on EssUP and NGP as this picture shows: 

He likes our work and he will help us communicate it in the software community. I will write again about the specific results of the meeting which will come in the next few months.