I would like to draw your attention to three recent blog entries: http://sematblog.wordpress.com/

1) "You are a developer - what is in Semat for you".

2) "Agile in everything".  One of the underlying principles of Semat is that working with methods needs to be agile (not just the methods themselves but working with them).  This implies features not previously found in how to define, use and adapt methods.

3) "A Major Milestone: On the way to a new standard".  An RFP of a standard based on the key ideas of Semat has been issued by OMG.  Letters of Intent are due on November 22, 2011; submissions are due on February 22, 2012.

This is very good progress, but honestly I don't feel the acceptance of the RFP is a sufficient step to declare success.  In the blog "A Major Milestone: On the way to a new standard", we finish by saying:

"Getting the RFP approved by OMG was one of the major milestones of Semat. Quoting Churchill: “Now this is not the end. It is not even the beginning of the end. But it is, perhaps, the end of the beginning.” Now we need to create something that will go beyond anything previously done by any standards body working with methods: getting the standard adopted by the broad developer community.

This is a challenge that cannot be overestimated.  This requires new fresh ideas that are not typical for standard bodies and methods work.  Fortunately, the Semat teams have several such new ideas. ‘Separation of concerns’ and ‘agile in everything’ will guide us, but more is needed.”

We have fresh new ideas for how to describe methods and practices in a very light way, ideas that significantly will improve readability.  The kernel will allow us to not just learn practices easily, but most importantly also allow us to use them during real work.  Earlier approaches have been completely silent on use, but modern approaches such as Kanban and Lean rely on similar ideas.

The number of people working on Semat has more than doubled over the last couple of months.  New chapters of Semat are set up in China and Latin America.  Still we would like to welcome more talented people to work with us.

--Ivar

As we refine and improve use cases we are careful to make sure that we don't impact any of these things that are key to their popularity and success. Use cases as we deal with them today have gone through a major face-lift.  Without really changing the key ideas, the impact of the changes is dramatic.  The result is a fundamentally more efficient way of developing software than the original use cases.

What is new about use cases?
The impact comes essentially from two areas: user stories and aspect-orientation.  The result is that we adapt them for backlog-driven development and managing cross-cutting concerns.

User stories:
In the past we had two concepts – use cases and scenarios.  Scenarios were a kind of user story.  In 2003 we introduced the concept use-case module (published in a paper [1] and in the aspect book [2]).  A use-case module was a slice through the system.  It included a use case (or a part of a use case), its analysis counterpart, its design, its code and its test.  Influenced by Scrum and user stories we sharpened these concepts and improved the terminology.  Now we talk about use cases, stories and use-case slices. Thus we have now:
1)    Use cases are, as they have always been, sets of structured stories (user stories if you want) in the form of flows of events, special requirements and test scenarios.
2)    Each story is managed and implemented as a use-case slice, which takes the story from its identification through its realization, implementation and test allowing the story to be executed.
3)    Thus a use-case slice is all the work that goes together with a particular story. Each story and thus its slice is designed to be a proper backlog element, and realized within an iteration or a sprint.
4)    The use-case strategy (starting from a use-case model) makes it significantly easier compared to the traditional user story strategy to identify the right user stories to work with and to understand how the ones selected make up the whole system.  As the use cases are now developed slice-by-slice, the size of the use cases is not any longer a problem!
Thus, use cases are what they always have been.  Stories are abstract scenarios a la user stories.  Use-case slices are use-case modules made smaller, suitable as backlog entries.   The terms scenario and use-case module will thus be replaced by story to remove the ambiguity between the abstract story-like scenarios and the concrete test scenarios and use-case slice.

Note: this can be compared with the user story approach where:
1)    The stories are captured as a set of unstructured user stories.
2)    Each user story is managed and implemented as one-or-more user story slices, which take the story from its identification through its realization, implementation and test allowing the user story to be executed.
3)    If a user story is too much too implement in one go the story is sliced up into a number of smaller user stories and the original user story disposed of. This illustrates how it is the user story slices that are implemented and not the user stories.
4)    Additional story types, such as Epic and Theme, are added to act as placeholders for user stories that we know will have to be sliced before they can be implemented.

Aspects:
Aspect-orientation has inspired us to deal with not just application-specific use cases (functional requirements), but also with infrastructure use cases (non-functional use cases).  The latter are dealt with as cross-cutting concerns, allowing us to add behavior to an existing system without actually changing the code of the existing system.  Examples of such non-functional behavior are persistency, logging of transactions, security.  This has helped us to deal with requirements (and their realizations) for systems of systems (enterprise systems, product lines, service-oriented architectures), and for partial systems such as frameworks and patterns.  See our book on aspects [ref. 2]

Thus, the key ideas have not changed, but they have been augmented with features that support backlog-driven development and working with non-functional requirements.

Use cases with stories and story slices address many of the issues now raised with the sole user story strategy.  Use cases with cross-cutting concerns address many of the problems analysts have raised with non-functional requirements.  To people, who already had adopted use cases, the new changes are not seen as large, but its impact on the way we develop software is dramatic.

-- Ivar

[1] Jacobson Ivar, Use cases and aspects - working seamlessly together, Journal of object technology, July-Aug 2003

[2] Jacobson Ivar, Ng Pan Wei, Aspect-oriented software development with use cases, Addison-Wesley, 2005.

I am pleased, honored and gratified that use cases are still a popular way of working with requirements.  Googling “use case” yields 6 times more hits than Googling “user story”, but software development should not be driven by popularity.  Instead we should use the most practical way of working.  And, of course we have learnt something from other techniques.  For instance, as I will discuss in my next blog, user stories and aspect-orientation have inspired us to make use cases even better while maintaining their core values.

The popularity of use cases has led to some misunderstandings and some distortions of the original technique.  This is natural, and while it is encouraging to see authors take the original concept and adapt it to solve new problems, some of the misconceptions and distortions have clouded the original vision.

Common misunderstandings

Before further discussing the improved use cases, let’s first discuss common misunderstandings about what we have had since their inception (1986-1992).  Many people believe that:

1) Use cases are for requirements only, which is not true. In fact, from the very beginning, they have also been used to analyze behavior, design software, and drive test identification, just to name a few uses.

2) Use cases are heavyweight; that you need to write fat specifications of each use case, which is also not true.  In fact, use cases can be exceptionally lightweight (a brief description only), to lightweight (just an outline of the flows), to comprehensive (full descriptions of all behavior), and every variation in between.  For most systems an outline can be very valuable and yet still be very lightweight.  Today, we express this in a better way: when describing use cases, focus on the essentials, what can serve as placeholders for conversations.

3) Use cases are a technique for decomposing the behavior of a system, which is also not true.  Some authors have introduced levels of decomposition, and others try to show use cases “calling” other use cases as if they were subroutines.  Neither of these is right.  A use case shows how a system delivers something of value to a stakeholder. Use cases that need to be “composed” in order to provide value are not real use cases.

4) Use cases are complicated.  In fact, using use cases, if done right, makes a system easier to understand.

o It is impossible to understand what a system does from looking at many hundreds of user stories; the equivalent use-case model might express the system’s behavior in a few handfuls of use cases. 

o A user is represented by a stick figure and a use case is represented by an oval.  Their interconnection by a simple line.

o The relationship between a use case and its scenarios are likewise very easy to represent. 

o To solve this problem with user stories, people have started to invent concepts such as themes and epics, making a case that the user story by itself is an incomplete concept. 

o The use-case approach can accommodate a wide range of levels of detail without introducing new and potentially confusing concepts.

5. Use cases are only seen as being good for green field development, which of course is not true.  They are great to explain large legacy systems as with such systems there is often little or no documentation left.  Use case modeling is a technique that is cheap and easy to get started with to capture the usages of the system.

What people like about use cases

The reason use cases have become so widely accepted is that since their introduction they are useful in so many ways in software development. 

1) A use-case model (a picture) already mentioned, which thus allows you to describe even a complex system in an easy to understand way, and which tells in simple terms what the system is going to do for its users. 

2) Use cases give value to a particular user, not to an unidentifiable user community.

3) Use cases are test cases, so when you have specified your use cases, you have also after complementing with test data, specified your test scenarios,

4) Use cases are the starting point to design effective user experiences, for instance for a web site.

5) Use cases ‘drive’ the development through design and code.  Each use case is a number of scenarios; each scenario is implemented and tested separately.

Moving forward
As we refine and improve use cases we are careful to make sure that we don't impact any of these things that are key to their popularity and success.  In my next blog I will describe how we adapted use cases to backlog driven development and managing cross-cutting concerns.

-- Ivar

Semat moving forward

During the last many months I have been very silent, but not inactive. I have been very active working with a dozen other people on moving Semat forward.  You will soon hear a lot more from us, but I would already now like to give you a quick update on the progress.

As you may recall, the Grand Vision of Semat was to re-found software engineering based on a widely agreed upon kernel representing the essence of software engineering.  The kernel would include elements covering societal and technical needs that support the concerns of industry, academia and practitioners.

The troika (Bertrand, Richard and I) were pleased, honored and gratified to find that in a short period of time, a dozen corporate and academic organizations, and some 3 dozen well-known individuals from the field of software engineering and computer science, became signatories to support the vision.  In addition, more than 1400 other supporters agreed to the call.

In November 2010, the troika agreed that we would move the work on the kernel to OMG (Object Management Group) to get the proper governance we needed.  Since then we have been working in three different but overlapping groups on three tasks:

Moving the development of the kernel to OMG.

In order to move the work to OMG, OMG first needed to submit a request for proposal (RFP).  A couple of people from Semat have worked together with a couple of OMG members to specify an RFP for what now is called 'A domain-specific language and a kernel of essentials for software engineering.’  Early December 2010, an early version of this RFP was presented to the Analysis and Design Task Force of OMG in Santa Clara. It was very well received.  Several other OMG members have now joined us to work on the RFP, which will be published within a few weeks.  March 21-24 the RFP will be discussed at an OMG meeting in Arlington/Washington DC.  We hope and expect it to be approved, and thus the work on proposals can start.  Anyone can submit a proposal, and so we will too.

Our proposal to a kernel

Semat itself will of course not give a proposal to the RFP, but key players are now working together to continue the work we started within Semat.  There is one team lead, Paul MacMahon, who along with 12-15 participants will now continue the work in a couple of tracks.  The idea of doing architectural spikes is continued.  The plan is still to be able to deliver something that can be used by the industry by April 1.  Personally, I think the work has slowed down because of the work with OMG and the continued work on Semat, which I will describe next.  However, we will deliver something of interest and also of value in a couple of months.

The kernel is just a first step in the Grand Vision of Semat.  However, much more work needs to be done.

Even if the development of the kernel now has been moved under the OMG’s umbrella, Semat still has a lot of work to do. We need for example to:

• be a demanding “customer” of OMG, making sure that as a community, we get what we want,
• support the community in its effort to get reusable practices,
• move the work to the academic community to inspire the development of new curricula and useful research.

Thus, a vision for the next couple of years is needed.  A team of 8 people have been working for more than a month to develop a proposal for a Three Year Vision of Semat. This proposed vision should be published within a couple of weeks.  We will focus on what impact we expect to have on three key user groups: the practitioner, the industry and the academia.  The impact should be measurable and not just hand-waving.  How we will work to get the results specified in the vision will be discussed separately.  First we want to agree on where we want to go.

As I am sure you understand, working to ensure that the vision of Semat becomes reality is a challenging task to say the least.  However, it is one well worth the effort.  Please join us.

Last June at Innovate 2010 in Florida Kurt Bittner envisioned the new role and responsibilities of the next generation business analyst. If you were not able to attend, his presentation is available online so you can check it out: Transforming the role of the Business Analyst. The need for a different role and responsibilities is to provide solutions for ongoing problems a lot of companies are faced with. These are common problems like:

• Users expecting  functionality they did not initially ask for
• Users demanding functionality they will never use
• Contradictory of conflicting requirements

In order to be more successful, a number of changes are to be made and lessons are to be learned. One of them is that business analysts need to be more focused on desired outcomes rather than features. And another is that business analysts need to probe into root causes rather than being satisfied with just identifying the wants. Being focused on outcomes and unraveling root causes can be hard work and sometimes it is easy to mix them up or to get stuck. A smarter way it is to be more aware of the language that is used for questioning and context frames . Read the rest of this entry »

Mijn collega Kurt Bittner heeft afgelopen juni  tijdens IBM Innovate 2010 (Florida) zijn visie gegeven op de nieuwe rol en verantwoordelijkheden van de nieuwe generatie informatieanalisten. Wanneer je geen kans hebt gezien om zijn presentatie bij te wonen, bekijk die dan via Slideshare: Transforming the role of the Business Analyst. Hieronder volgende enkele observaties of veel voorkomende problemen die hebben geleid tot zijn visie:

• Gebruikers verwachten andere  functionaliteit dan waar ze oorspronkelijk om hebben gevraagd.
• Gebruikers eisen functionaliteit die ze nooit zullen gebruiken
• Gebruikers geven tegenstrijdige of conflicterende requirements Read the rest of this entry »

De laatste paar dagen hebben verschillende mensen mij dezelfde vraag gesteld: “Hoe kunnen we meer heldere requirements krijgen”. Hoewel ik het eens ben dat het niet netjes is om een vraag met een vraag te beantwoorden, is het in mijn visie beter om in dit geval soepel om te gaan met die etiquette. En wel  omdat deze vraag eigenlijk niet zo eenvoudig te beantwoorden is. Bijvoorbeeld, waarom is dit nodig en welk probleem lost het op? Of, wat bedoel je precies met helder? En heb je heldere requirements en wil je er meer? En in dat geval, meer dan wat? Of heb je requirements die niet helder zijn en die je beter wilt kunnen communiceren. En zo ja, hoeveel beter?

Je zou kunnen zeggen dat dit gewoon spelen met woorden is. En dat klopt! Maar is het formuleren en communiceren van requirements niet feitelijk hetzelfde? Voor meer heldere requirements zijn een tal van zaken benodigd. Er is echter een belangrijk element dat vaak over het hoofd wordt gezien, namelijk inzicht in de structuur van taal en de wijze waarop taal wordt geïnterpreteerd. Read the rest of this entry »

Bij IJI staan we een practice-gedreven aanpak voor. Zoals met veel concepten en principes geldt ook hier dat een goed begrip van wat ermee wordt bedoeld essentieel is om er echt de vruchten van te plukken. Een manier om daar invulling aan te geven is om te kijken naar criteria waaraan je een goede practice kunt herkennen. Maar voor we dat doen, is het goed om eerst te kijken naar de definitie en kenmerken van een practice en een practice-gedreven aanpak.

Onder een practice-gedreven aanpak verstaan we onder meer “het samenstellen van een effectieve manier van werken op basis van relevante procescomponenten”. In plaats van moeizaam te proberen een procesraamwerk toe te snijden op een specifiek project draaien we het om; we selecteren alleen relevante procescomponenten en assembleren die tot een consistent proces. En zo’n procescomponent noemen we dan een practice. Use Cases, User Stories en Iteratief ontwikkelen zijn enkele voorbeelden van practices. Andere voorbeelden zijn Operationaliseren Systeem en Datamigratie. In deze context is het ultieme doel van een practice-gedreven aanpak “betere resultaten boeken met softwareontwikkeling”. Hierbij kun je dan denken aan betere software, goedkoper, sneller en een prettigere manier van werken. Read the rest of this entry »

In the last Kernel Journal we looked at the problem that Barry Boehm was aiming to solve back in 1995 when he first proposed his three standard process milestones (which later gained industry prominence as the milestones in the Unified Software Development Process and the Rational Unified Process), namely that “the proliferation of software process models provides flexibility”, but leaves us “with no common anchor points around which to plan and control.” [Barry Boehm, November 1995]. We looked at how a small set of domain entities with simple state progression models (which we call “Alphas”) can make these common anchor points much more practical and useful while ensuring that they remain process-neutral and do not become “document-driven”.

The alphas, when used with common milestones such as the Unified Process Milestones, can actually give us much more than this – they can provide a project status and health dashboard that can be used by the customer and supplier organizations to assess the current status and health of any / all projects, irrespective of which processes or practices they are following. The graphic below shows just such a dashboard, with a set of kernel alphas and a traffic-light status for each alpha, which is derived by comparing where the project is now (the state machine to the left of each traffic light) with where it needs to be to achieve the next project milestone (the state machine to the right of each traffic light).

In Kernel Journal 5: “Making the Invisible Visible” I described how we can easily “skin” a process kernel, by providing a portal for projects to capture, share and agree the essential project information that is needed to achieve each state progression (for example, using a set of templated Wiki pages). Once we have done this, we can make the alpha dashboard much more useful to the project teams themselves, by flagging which sections of the project portal need to be updated and agreed to get the project to where it needs to go next.

This gives us the equivalent of a Satellite Navigation System for our software projects project that enables us to:

• Set our journey destination and waypoints (milestones)
• Track where we are now, compared to where we want to be
• Get guidance on what to do next in order to progress towards our destination.

“The best way to predict the future is to invent it!“ (Alan Kay)

A few days ago, a very simple but thought provoking question was raised to me: “what it is that drives me?” The simple truth is that I do not know. But I do know what it is that does not drive me. It is not about money. Actually, never has it been about money. Neither is it about power. I am happy to step aside and I am happy to delegate both up and down. It is not about popularity – but I do like to be appreciated for what I do.

No, it has to do with helping others improve themselves over and over again. I get a kick out of seeing others become successful because I helped them. It was like that in the late 1960s and the ‘70s when the Ericsson AXE system beat all competition and won every contract thanks to being component-based. Similarly, when Rational was successful because of UML and Objectory. And Telelogic because of SDL. I am happy when people are successful thanks to use cases.

Read the rest of this entry »