Year after year I feel fortunate that we as a company can report so many new technology advances – advances which help our customers build better software, faster and with happier users and clients.  A number of new practices have been developed to help our customers scale agile beyond small teams to whole organizations.  Our practices for scaling agile and for setting up coaching communities have, with our experienced mentors, helped many organizations to not just become successful with small projects but changed organizations in their entirety.  Several case studies confirm these successes.

Most important in 2011 is our release of Use-Case 2.0.  After having refined the way we work with agile use cases over many years, we have now put together all our experience in an ebook.  Our customers have already used this around the world, China, UK, Sweden, US; but now we have streamlined it close to perfection.  Use-Case 2.0 supports backlog-driven development with Scrum or Kanban making it as lean as it can be.  Many of our colleagues and friends are awaiting this new release ready to start using it.

Lean development has been a long term goal for our company.  Our approach, starting lean with a kernel of the most essential elements in software engineering and staying lean as more detailed practices are added on top of the kernel, is getting wide acceptance in the world.  During 2012, we will see standards being adopted, the academic world embracing it, new tools being available, the industry getting some governance of their ways of working, and developers systematically growing their competence in software development.  Most importantly, success stories will speak for themselves: agile and lean will be achievable targets for every organization developing software.

Welcome to the future, but don't forget to enjoy the holidays.
--Ivar Jacobson

Code engineering is an interesting term that I so far have not come across.  It instantly has a built in conflict because it implies that coding is an engineering activity and if it is, then it can't be an art or a science.    Neither of these would be an engineering activity. Thus, the question is broken.

It would be easier to answer the question "is coding an art or a science".  I think it is neither.  It is more like a craft.  However, when do you really need craftsmanship when writing code?

When you implement interesting new algorithms,  you really need excellent coding skills.  However, the total amount of code of this nature is small.  I dare say that more than 80% - 90 % of code written for applications in banking, telecom, defense, etc. are just plain implementations of relatively simple features.  The coding job is quite straightforward, most of it doesn’t require great craftsmenship. Understanding the business processes (or how you can change them), the user's needs, the architectural principles and values, the available solutions as prefabricated packages or frameworks etc. are then as critical as having good coding skills.

Now, is there a case when someone could motivate talking about code engineering.  Yes, if everything you do would be about coding.  When you think about new business processes, you express them as code.  When you think about what the software should do (requirements) you express it as code.  When you think about the architecture and design, you express it as code.  Then of course code engineering would make perfect sense.  On the contrary if you think about code engineering as a discipline besides other engineering activities (requirements, architecture, design, etc.), then you would sub-optimize the work you have to do while coding, and that would be a serious mistake – not smart!

Follow the conversation with me on Twitter.

So how can a 500 year old quotation be relevant to business today? Well, you’ll be surprised.

The concept of a “kernel” representing the essence of something is a good starting point for most things you build.  You start lean and you stay lean throughout the life of what you build.  The idea of a kernel has many practical applications in today’s business:

1) designing an agile business,

2) building products using agile techniques,

3)  re-engineering your method or way of working.

Proven in many practical situations, the kernel concept provides the ability to scale up the use of agile approaches whilst maintaining control and visibility. It is now being considered for adoption by standards bodies such as the Object Management Group to enable light weight, usable, agile approaches to knowledge management.

The justification for the title is that adopting a kernel approach focuses on the essentials and naturally leads to lean processes and systems. The renaissance is that instead of removing waste to get lean we start lean and stay lean.

I presented a keynote with the same title as this blog at the Agile Business Conference in London Oct 5.  You can download the presentation here.

Follow the conversation with me on Twitter.

Distinctive features of the new use cases are:

• As agile and light as you want it to be
• Scaling up, scaling out and scaling in - to meet your needs
• It’s not just about requirements - it's for the whole lifecycle
• It's also for non-functional requirements
• It’s not just for software development - it's for business as well

The presenation at IIBA was received very well by the around 400 participants.  Afterwards IJI staff were busy explaining the new use-cases and how to find slices of use cases to populate the backlog to support Scrum or Kanban like projects. Participants signed up for the new eBook on use-case 2.0, to be released shortly.

You can register to receive the powerpoint presentation here.

Follow the conversation with me on Twitter.

1) "You are a developer - what is in Semat for you".

2) "Agile in everything".  One of the underlying principles of Semat is that working with methods needs to be agile (not just the methods themselves but working with them).  This implies features not previously found in how to define, use and adapt methods.

3) "A Major Milestone: On the way to a new standard".  An RFP of a standard based on the key ideas of Semat has been issued by OMG.  Letters of Intent are due on November 22, 2011; submissions are due on February 22, 2012.

This is very good progress, but honestly I don't feel the acceptance of the RFP is a sufficient step to declare success.  In the blog "A Major Milestone: On the way to a new standard", we finish by saying:

"Getting the RFP approved by OMG was one of the major milestones of Semat. Quoting Churchill: “Now this is not the end. It is not even the beginning of the end. But it is, perhaps, the end of the beginning.” Now we need to create something that will go beyond anything previously done by any standards body working with methods: getting the standard adopted by the broad developer community.

This is a challenge that cannot be overestimated.  This requires new fresh ideas that are not typical for standard bodies and methods work.  Fortunately, the Semat teams have several such new ideas. ‘Separation of concerns’ and ‘agile in everything’ will guide us, but more is needed.”

We have fresh new ideas for how to describe methods and practices in a very light way, ideas that significantly will improve readability.  The kernel will allow us to not just learn practices easily, but most importantly also allow us to use them during real work.  Earlier approaches have been completely silent on use, but modern approaches such as Kanban and Lean rely on similar ideas.

The number of people working on Semat has more than doubled over the last couple of months.  New chapters of Semat are set up in China and Latin America.  Still we would like to welcome more talented people to work with us.

--Ivar

What is new about use cases?
The impact comes essentially from two areas: user stories and aspect-orientation.  The result is that we adapt them for backlog-driven development and managing cross-cutting concerns.

User stories:
In the past we had two concepts – use cases and scenarios.  Scenarios were a kind of user story.  In 2003 we introduced the concept use-case module (published in a paper [1] and in the aspect book [2]).  A use-case module was a slice through the system.  It included a use case (or a part of a use case), its analysis counterpart, its design, its code and its test.  Influenced by Scrum and user stories we sharpened these concepts and improved the terminology.  Now we talk about use cases, stories and use-case slices. Thus we have now:
1)    Use cases are, as they have always been, sets of structured stories (user stories if you want) in the form of flows of events, special requirements and test scenarios.
2)    Each story is managed and implemented as a use-case slice, which takes the story from its identification through its realization, implementation and test allowing the story to be executed.
3)    Thus a use-case slice is all the work that goes together with a particular story. Each story and thus its slice is designed to be a proper backlog element, and realized within an iteration or a sprint.
4)    The use-case strategy (starting from a use-case model) makes it significantly easier compared to the traditional user story strategy to identify the right user stories to work with and to understand how the ones selected make up the whole system.  As the use cases are now developed slice-by-slice, the size of the use cases is not any longer a problem!
Thus, use cases are what they always have been.  Stories are abstract scenarios a la user stories.  Use-case slices are use-case modules made smaller, suitable as backlog entries.   The terms scenario and use-case module will thus be replaced by story to remove the ambiguity between the abstract story-like scenarios and the concrete test scenarios and use-case slice.

Note: this can be compared with the user story approach where:
1)    The stories are captured as a set of unstructured user stories.
2)    Each user story is managed and implemented as one-or-more user story slices, which take the story from its identification through its realization, implementation and test allowing the user story to be executed.
3)    If a user story is too much too implement in one go the story is sliced up into a number of smaller user stories and the original user story disposed of. This illustrates how it is the user story slices that are implemented and not the user stories.
4)    Additional story types, such as Epic and Theme, are added to act as placeholders for user stories that we know will have to be sliced before they can be implemented.

Aspects:
Aspect-orientation has inspired us to deal with not just application-specific use cases (functional requirements), but also with infrastructure use cases (non-functional use cases).  The latter are dealt with as cross-cutting concerns, allowing us to add behavior to an existing system without actually changing the code of the existing system.  Examples of such non-functional behavior are persistency, logging of transactions, security.  This has helped us to deal with requirements (and their realizations) for systems of systems (enterprise systems, product lines, service-oriented architectures), and for partial systems such as frameworks and patterns.  See our book on aspects [ref. 2]

Thus, the key ideas have not changed, but they have been augmented with features that support backlog-driven development and working with non-functional requirements.

Use cases with stories and story slices address many of the issues now raised with the sole user story strategy.  Use cases with cross-cutting concerns address many of the problems analysts have raised with non-functional requirements.  To people, who already had adopted use cases, the new changes are not seen as large, but its impact on the way we develop software is dramatic.

-- Ivar

[1] Jacobson Ivar, Use cases and aspects - working seamlessly together, Journal of object technology, July-Aug 2003

[2] Jacobson Ivar, Ng Pan Wei, Aspect-oriented software development with use cases, Addison-Wesley, 2005.

The popularity of use cases has led to some misunderstandings and some distortions of the original technique.  This is natural, and while it is encouraging to see authors take the original concept and adapt it to solve new problems, some of the misconceptions and distortions have clouded the original vision.

Common misunderstandings

Before further discussing the improved use cases, let’s first discuss common misunderstandings about what we have had since their inception (1986-1992).  Many people believe that:

1) Use cases are for requirements only, which is not true. In fact, from the very beginning, they have also been used to analyze behavior, design software, and drive test identification, just to name a few uses.

2) Use cases are heavyweight; that you need to write fat specifications of each use case, which is also not true.  In fact, use cases can be exceptionally lightweight (a brief description only), to lightweight (just an outline of the flows), to comprehensive (full descriptions of all behavior), and every variation in between.  For most systems an outline can be very valuable and yet still be very lightweight.  Today, we express this in a better way: when describing use cases, focus on the essentials, what can serve as placeholders for conversations.

3) Use cases are a technique for decomposing the behavior of a system, which is also not true.  Some authors have introduced levels of decomposition, and others try to show use cases “calling” other use cases as if they were subroutines.  Neither of these is right.  A use case shows how a system delivers something of value to a stakeholder. Use cases that need to be “composed” in order to provide value are not real use cases.

4) Use cases are complicated.  In fact, using use cases, if done right, makes a system easier to understand.

o It is impossible to understand what a system does from looking at many hundreds of user stories; the equivalent use-case model might express the system’s behavior in a few handfuls of use cases. 

o A user is represented by a stick figure and a use case is represented by an oval.  Their interconnection by a simple line.

o The relationship between a use case and its scenarios are likewise very easy to represent. 

o To solve this problem with user stories, people have started to invent concepts such as themes and epics, making a case that the user story by itself is an incomplete concept. 

o The use-case approach can accommodate a wide range of levels of detail without introducing new and potentially confusing concepts.

5. Use cases are only seen as being good for green field development, which of course is not true.  They are great to explain large legacy systems as with such systems there is often little or no documentation left.  Use case modeling is a technique that is cheap and easy to get started with to capture the usages of the system.

What people like about use cases

The reason use cases have become so widely accepted is that since their introduction they are useful in so many ways in software development. 

1) A use-case model (a picture) already mentioned, which thus allows you to describe even a complex system in an easy to understand way, and which tells in simple terms what the system is going to do for its users. 

2) Use cases give value to a particular user, not to an unidentifiable user community.

3) Use cases are test cases, so when you have specified your use cases, you have also after complementing with test data, specified your test scenarios,

4) Use cases are the starting point to design effective user experiences, for instance for a web site.

5) Use cases ‘drive’ the development through design and code.  Each use case is a number of scenarios; each scenario is implemented and tested separately.

Moving forward
As we refine and improve use cases we are careful to make sure that we don't impact any of these things that are key to their popularity and success.  In my next blog I will describe how we adapted use cases to backlog driven development and managing cross-cutting concerns.

-- Ivar

During the last many months I have been very silent, but not inactive. I have been very active working with a dozen other people on moving Semat forward.  You will soon hear a lot more from us, but I would already now like to give you a quick update on the progress.

As you may recall, the Grand Vision of Semat was to re-found software engineering based on a widely agreed upon kernel representing the essence of software engineering.  The kernel would include elements covering societal and technical needs that support the concerns of industry, academia and practitioners.

The troika (Bertrand, Richard and I) were pleased, honored and gratified to find that in a short period of time, a dozen corporate and academic organizations, and some 3 dozen well-known individuals from the field of software engineering and computer science, became signatories to support the vision.  In addition, more than 1400 other supporters agreed to the call.

In November 2010, the troika agreed that we would move the work on the kernel to OMG (Object Management Group) to get the proper governance we needed.  Since then we have been working in three different but overlapping groups on three tasks:

Moving the development of the kernel to OMG.

In order to move the work to OMG, OMG first needed to submit a request for proposal (RFP).  A couple of people from Semat have worked together with a couple of OMG members to specify an RFP for what now is called 'A domain-specific language and a kernel of essentials for software engineering.’  Early December 2010, an early version of this RFP was presented to the Analysis and Design Task Force of OMG in Santa Clara. It was very well received.  Several other OMG members have now joined us to work on the RFP, which will be published within a few weeks.  March 21-24 the RFP will be discussed at an OMG meeting in Arlington/Washington DC.  We hope and expect it to be approved, and thus the work on proposals can start.  Anyone can submit a proposal, and so we will too.

Our proposal to a kernel

Semat itself will of course not give a proposal to the RFP, but key players are now working together to continue the work we started within Semat.  There is one team lead, Paul MacMahon, who along with 12-15 participants will now continue the work in a couple of tracks.  The idea of doing architectural spikes is continued.  The plan is still to be able to deliver something that can be used by the industry by April 1.  Personally, I think the work has slowed down because of the work with OMG and the continued work on Semat, which I will describe next.  However, we will deliver something of interest and also of value in a couple of months.

The kernel is just a first step in the Grand Vision of Semat.  However, much more work needs to be done.

Even if the development of the kernel now has been moved under the OMG’s umbrella, Semat still has a lot of work to do. We need for example to:

• be a demanding “customer” of OMG, making sure that as a community, we get what we want,
• support the community in its effort to get reusable practices,
• move the work to the academic community to inspire the development of new curricula and useful research.

Thus, a vision for the next couple of years is needed.  A team of 8 people have been working for more than a month to develop a proposal for a Three Year Vision of Semat. This proposed vision should be published within a couple of weeks.  We will focus on what impact we expect to have on three key user groups: the practitioner, the industry and the academia.  The impact should be measurable and not just hand-waving.  How we will work to get the results specified in the vision will be discussed separately.  First we want to agree on where we want to go.

As I am sure you understand, working to ensure that the vision of Semat becomes reality is a challenging task to say the least.  However, it is one well worth the effort.  Please join us.

“The best way to predict the future is to invent it!“ (Alan Kay)

A few days ago, a very simple but thought provoking question was raised to me: “what it is that drives me?” The simple truth is that I do not know. But I do know what it is that does not drive me. It is not about money. Actually, never has it been about money. Neither is it about power. I am happy to step aside and I am happy to delegate both up and down. It is not about popularity – but I do like to be appreciated for what I do.

No, it has to do with helping others improve themselves over and over again. I get a kick out of seeing others become successful because I helped them. It was like that in the late 1960s and the ‘70s when the Ericsson AXE system beat all competition and won every contract thanks to being component-based. Similarly, when Rational was successful because of UML and Objectory. And Telelogic because of SDL. I am happy when people are successful thanks to use cases.

To constantly improve, I have to come up with new ideas. But I can’t stop at just creating new ideas; I have to get these new ideas widely accepted, usually in the form of standards. The latter I will talk about in another column.

Then, how do I get new ideas? Do they come like flashes of genius? Unfortunately they do not.  Almost always, they come after a long period of painful, deep and heavy thinking.  Basically, I say to myself “now you have to think about this issue, and this issue is just vaguely known or just a symptom of a problem”.  Thus I have no clear problem definition, so first, I focus on identifying the core problem and once I have found it the solution lies close at hand. Or more simply put: find the problem and more than half the job is done. Since, the problem is new, there are usually no solutions already out there; which means I can work to find a solution in an open space. This is the trick.

Unfortunately many people do not worry about finding the problem; they feel they should only worry about the solution. I have heard many academics say that the industry does not know what problems they have, so how can they expect their problems to be solved? Even when an industry representative really tries to describe a problem, but doesn’t use the terminology of the science in question, his or her words sound like noise and he or she is not taken seriously. Academics should instead listen carefully and actually make every effort to seek to find the problem underlying what they are trying to say.

There are exceptions.  When ITU (International Telecommunication Union) worked on the standard programming language CHILL back in 1978, I presented to the workgroup (about 20 members from the industry, all language experts) a new communication mechanism called ‘signal’.  It was a message with ‘send no-wait’ semantics instead of the more traditional message types based on ‘send wait’ semantics.  The whole group was against my proposal.  And, when a group of 20 people downplay a proposal it can be quite ugly.  However, there was one man who stood up for my proposal – professor Dines Bjorner from Denmark.  He said he would study my proposal and compare it to recent work by Tony Hoare on communicating sequential processes.  After a couple of weeks of work, Dines Bjorner had formally defined signals.  He presented the new idea to the group saying that this is what the world has been waiting for the last ten years; and that it is time to retire the old ideas.  Two years later all of the major telecom vendors had turned to signals.  The language designers hadn’t seen the new idea, but their developers had been waiting for this change for a long time.  The moral of this story is that academics should go out in the field and learn.  The field is vast and there are lots of important problems waiting to be discovered.  You just need to listen respectfully and you will harvest the most beautiful ideas.

My ideas always come in pairs – a problem with a solution.

First I find a problem and then I find a solution to that problem. It is not the reverse, first finding a solution and then looking for a problem to that solution.

It starts with some known negative symptoms that I suspect are hidden in a deeper problem. I look for the core of the problem. When and if I find a problem that is not identified by the outside world; the solution is usually close at hand.  And my melancholy transforms into great relief.

In this way, a number of ideas have come to me over the years. In 1967, when I saw Ericsson wrestling with the problem that a computerized telephone exchange consisted of two giant systems (a program system and a database system) with an intense very low-level communication protocol (basically read and write only), the solution was to build systems with components. Sequence diagrams were the solution to describe collaboration between the components in the making of telephone calls. I had to sell both the problem (since it was unknown) and the solution. The idea was very controversial so that it succeeded was a mere miracle.

In 1986, Use Case was the solution to the problem that traditional functional specifications were immense and not testable. To start from the users and find their different use cases made the specifications understandable, while we also at the same time found the test cases. The result was a good way to do test-driven development, now being popular in agile teams.

First developed in 1978 and then further developed in 2005, Aspects were the solution to the problem to avoid changing what already works just because we want to add new features. It became a book and many large companies have embraced this, but it will take some more years to go before it will become the standard way of working.

Intelligent agents were, in 1981 and later in 2000, the solution to the problem that 80% of what we do when we build software is repetitive no-brain work.  When we develop something we follow patterns we have followed many times before. Having qualified resources just following standard patterns is not smart.   We built Waypointer, but at a too early point of time. The idea will come back in some years.

In 2003, Practices became the solution to allow many methods in a company but still have governance – enough governance, but no more than necessary.

Over the years there have been some ideas. But it is not enough to come up with new ideas. You must always show how to implement the ideas. Thus I designed the first two components at Ericsson. And in my business, I have driven the ideas into products. This also requires perseverance. You must not give up just because you meet good but tough resistance.  Actually, every good idea will meet substantial resistance.  If it doesn’t, then it probably is not a new idea.  However, don’t make the mistake and interpret every idea that meets resistance as a good idea J.

Moreover, many times my solutions had no support in existing technologies, such as no support in existing programming language or operating system. For instance, when I came up with components in 1967, there was no support for signals and interfaces in the assembler we used.  Thus, I had to come up with a solution that simulated these language constructs.  In working with my colleagues we came up with the idea that every signal was implemented by a macro owned by the receiving component.  Sending a message was thus implemented as calling a macro.  That was smart.  Similarly, when we did aspects we had to find a smart way to extend code without changing it.  This mechanism is today called pointcuts.  When we created Waypointer there was no support for object-oriented agents, so we had to simulate that.  Your ideas may not be supported by your platform, but don’t let that discourage you from implementing your ideas.

Thus, as a summary, what drives me is to make the people I work with successful. To achieve this, I invent new ideas, all of which come in pairs – a problem with a solution. Almost always, I have been selling both the problem and the solution. And, no idea has sold itself; you must always contribute to its implementation. It requires stamina.

I hope that you dear readers might learn something of this. Maybe that would be Smart!

Trend “Methods & Tools,” exemplified by Semat Initiative (www.semat.org), of which I myself am one of the founders, drives the thesis that the way we develop software is immature and in need of being revolutionized. Yes, these are strong words, but the initiative is supported by more than 30 renowned scientists, scholars and practitioners in the software engineering field, including leaders from major industrial corporations (e.g., ABB, Ericsson, Fujitsu, IBM, Samsung, Microsoft, etc), academia (researchers, professors, institutional managers), and thousands of practitioners around the world.

Specific problems addressed are that often methods seem to be based on fashion and fads; that there are almost an infinitive number of methods that cannot be assessed or compared with each other. The number itself is not a problem. There should be many methods, however, these methods must be designed in such a way that they can be compared, assessed and improved. Finally, there exists a big barrier between academic research and industrial practices, which must be torn down.

The way forward is based on observation and understanding that:

1. Every method is just a composition of practices, either human or technology related.
2. Practices are reusable over many application areas and technologies.
3. Underneath all practices is a small kernel of universals described by a small kernel language.
4. Universals are things that we always have, do or produce when building software.  We will discover them!
5. Practices and universals will be described by a small Process Kernel Language.

Using this kernel and the language, we can describe all known methods, and, because practices are comparable, methods that are composed from these practices can be compared. I would like to tell you a lot more now, but it can wait for later. The impact of this SEMAT initiative is that, if successful, we will streamline the entire software world: from academia to industry, from practitioners to teachers and researchers. We will become better, faster, and happier in developing software.

After all, it is people who develop software, not methods and tools. So we must address the “human” side of software development.

Trend “Professionalism & Craftsmanship”, which is popular with the original founders of the agile movement, for instance by Bob Martin (“Uncle Bob”), who takes a very different standpoint. From this perspective, the big problem is not the lack of methods or tools, but how we train, educate and mentor programmers to become professional craftsmen. Code can be written by anyone at any time, but what makes us professionals?

1. We must be proud of what we do. We must be able to say “no” to either the boss or the customer, if necessary. We have our professional practices and these cannot be compromised.
2. The boss and the client must accept the fact that our work is technical in nature; so let them think we are geeks, but respectable geeks.
3. Eliminate hourly rate - doctors or lawyers are not paid by the hour (even if under pressure they may say so). There must be better ways to charge.
4. Anything that is worth doing should be done well and with quality. When we ship code, we must know that it works. Acceptance testers should not find any errors.
5. Become competent through an apprenticeship program. Choose a master and learn from him or her.  After some years you may select a new master and also learn from him or her.

Both trends are of course important.

Proponents of methods & tools suggest that it is clear that we must constantly improve professionalism. However, it would be much easier to be professional if we can elevate the level of our understanding of methods & tools.

Proponents of professionalism & craftsmanship are concerned that such an elevation means enforcing restrictions, and many are therefore hesitant or reluctant to work with or support initiatives related to methods & tools.

It is clear to me that we must do both. Since Bob Martin is a signatory of Semat, I think it is clear to him as well.

It is smart!