Architecture

Learning by example by Ivar Jacobson

In the work that we do people often want a recipe for developing software, a series of steps that predictably produce a result. Recipes are good, whether in cooking or in other areas, but they are not enough, and not everything that is interesting can be reduced to a simple recipe.

Over the years I've had the chance to observe how people learn. Reflecting on how I learn new things as well, I've come to the conclusion that many people, myself included, don't learn very well from following a recipe. In fact I'm rather hopeless at following step-by-step instructions.  As a kid I liked to tear things apart, figure out how they worked, and then put them back together. And most of the time they actually worked when I did eventually get them back together.

Taking something apart and putting it back together is a special case of learning by example where the thing you are taking apart is the example.  Once you've done this enough you can start improvising and designing new ways to solve the problem.

A lot of software development works the same way - whether it is a piece of code or a requirements specification, a lot can be learned from tearing apart a good example, understanding why it is good and how it works, and then, over time, starting to improvise those lessons learned on new problems.  In fact, given the choice between templates and a good example I'll choose the good example any time.  Even if you don't understand all the principles right away, most of us are clever enough to copy the parts that work that we don't understand and be creative in areas where we need to.  Over time we learn and the need to mimic goes away.  This is the way that all of us learned our native tongues, and the approach still serves us well today.

Will MDD Ever Come to Fruition? by Ivar Jacobson

I am often asked the question: “will MDD ever be successful and will the tools ever really be put in place to support it?” and I was recently asked this again, so here are my thoughts.  

The following is solely my opinion and can be argued or agreed with, but it comes from 15+ years of experience building application and data modeles, modeling tools including ERwin, Rational Rose and other tools, writing 2 books on UML and working directly with clients who are modeling. 

Model Driven Development in concept is great, but to date the tools and the people who would have to use them have not been able to keep up with the concepts.  There are some very good tools on the market like Telelogic Tau, Rational RoseRT and a few others which do a fairly complete job of generating the needed code, but this is usually focused in the "systems" space and has not translated well to IT as it is based on state's and generating the logic from those states, etc.   

On the IT side, we do have similar concepts, but they start from business process models using tools like WebSphere Business Modeler and similar tools from BEA for example which connect to Business Process runtime engines and generate the connection points of existing applications to automate communication and business process execution.   

This all said, the uptake MDD has not been that of other areas for what I believe are 3 reasons: 

1.      Developers are developers because they want to write code and don't see models as being complete enough nor for their role to build models beyond simple architecture structures.

2.      Most projects today are starting with something already in production and therefore doing modeling of the architectures, use cases and processes are quite good to understand what needs to be done, how it connects together and prioritize work, but it makes it difficult to generate a system that is only a piece to connect up or an update.

3.      I believe #3 can stand on its own, but also lends itself to the first 2 comments and that is the creation of a "Black Box".  Using MDD tools creates a black box effect where runtime generation, transformations and other constructs are managed by a proprietary engine which is difficult to alter and manage. 

a.      For Developers, they often think they can do it better and faster and don’t want to rely on something they cannot touch. 

b.      Because of the black box approach, it is often requires a rewrite of the engines that have already been put into place for the existing systems causing added cost that nobody is willing to fund. 

We have tried in the past similar types of technologies which few have been successful as well.  Runtime Data Access is a great example where tools and companies popped up in the late 90's which created object-to-data mapping and automated the creation of the runtime access at what they claimed to be much faster and cheaper than doing it yourself.  Yes, this was good at least in theory, but few bought into it.  Why?  Black box approach, hard to test, developers think they can write faster algorithms, management doesn't trust something they cannot see, etc.  This is very similar to MDD and its lack of success in my opinion.   

That all said, I do have some collegues who are using MDD on some very interesting projects building components for airplanes for example which they feel are quite successful, but these also seem to be few and far between.

Do we need Event-Driven Architecture? by Ivar Jacobson

A software system with an Event-Driven-Architecture (EDA) is built around the idea that events are the most significant elements in the system and that they are produced somewhere in the system and consumed somewhere else in the system.

The business value is that you can easily extend such a system with new things that are ready to produce or consume events that already are in place in the system.  Of course you can add new events as you go. 

Yes, this is absolutely great.  If you build something new there is no reason why you shouldn’t use this kind of architecture.  However, focusing on the events is not the only thing you should do.   

Instead, you should just build an architecture in which you have components or services and some kinds of “channels” between some of these components.  Over a channel an event can flow from one component – the producer – to another one – the consumer.  These components are loosely coupled and can exist in a distributed world.  Some of these events are such that you broadcast them to anybody that has subscribed to them. 

Thus don’t constrain your architecture to just be event-driven.  There is really no money to save by doing just that.  Let it be components with channels.  The channels I am talking about were already adopted in the telecom standard SDL back in 1982.  In EDA it is basically a mechanism for brokering events.   In the OMG standard CORBA from the early 1990s it was called the “Event Service”.  What a coincidence!  Actually one way of thinking about EDA conceptually is really that it is all that CORBA was meant to be, but in the Web/Internet world. 

The most interesting components are services.  You get service-oriented architecture at the same time, and more. 

However, those of you who think this is fundamentally new have really not done your homework.  It is probably true that the three letter combination EDA is new as it was for SOA.  We have also got some new great platforms that make it easier to implement these ideas.

Over the years I have seen trends in the component world that put more focus on the components than the channels (and thus the events) between them.  Other times it has been the other way around.  However, there is absolutely no reason to choose.  You should allow for both. But what we don’t need are more buzzwords. They don’t help us at all. 

To summarize, you should go for a component architecture without any compromises.  This is what made the Ericsson AXE system such an incredible success story more than 30 years ago.  And thanks to its architecture it is still probably the best selling product of its kind in the world.  However, Ericsson had to build its own infrastructure managing components with channels since such solutions didn’t exist at the time.

Of course, this is still new to people who have not previously developed a component architecture.  Thus those people have to come up to speed and that means training and mentoring.  And, to start with you need some good technical practices.  It is as easy as that! 

 

 

Executable SOA by Ivar Jacobson

Today everyone talks about Service-Oriented Architecture. And as with all buzz words there are many interpretations. As the name suggests SOA was originally about architecture, but as time goes by its proponents put more and more meaning into it in the same way as proponents of EA put more and more into it. This is why some people have nicknamed SOA to Service-Oriented Ambiguity.      

What is SOA at its core? It is a kind of software architecture which allows you to connect course granular components called services through well-defined interfaces. These components can reside on any kind of platform (mainframe, client/server, etc.). They can be web applications, java or .NET applications, ERP systems or your own legacy systems.  

What is the business case for SOA? Since, services are essentially reusable components, you can efficiently support new business processes by using old services and interconnecting them. Since, they have well-defined interfaces they can be replaced by any other component complying to the same interface. 

Is this really new? Yes and no. No, because SOA has existed almost forever in the telecom space. The most successful commercial product ever built in Sweden was the Ericsson AXE switch. AXE’s success came to some extent from that its architecture was service-oriented. However, Ericsson had to build its own infrastructure - a unique computer architecture and operating system. My Software Reuse book (Software Reuse: Architecture, Process and Organization for Business Success, ACM Press) from 1997 is essentially a book on SOA. Yes, it is new because big vendors like IBM provide an infrastructure that allows services to reside basically anywhere in a network. 

How to succeed with SOA? It goes without saying that you need good people – leaders as well as champions. And they need to be smart – have common sense, practical, do what is needed not more, and of course be competent. Having said what is obvious let’s move to a more techie perspective:  

Design top down, bottom up and then find the balance as a trade-off between value and cost. Top down means you start from some business process ideas and deduct a solution as a set of interconnected components. If you just go top down it will usually be unreasonably expensive. Bottom up means that you start from what is available/reusable, such as legacy systems, web services, ERP systems, whatever, and build something close to what you need by interconnecting these components. This may not be exactly what your business modeling people were dreaming about, but something you can build quickly with low costs. Finally, you balance the top down and the bottom up and get what you may call a good enough solution. Your solution reuses a lot, but you may also have to build something new. This is smart! However, it is not enough. 

Do this very light (don’t major in modeling or post-its), develop quickly a road-map, build in small executable steps (even if the enterprise system is huge). And beware of the service companies that want to sell you big solutions – lots of billable hours without being accountable for delivering executable software!Don’t fail with SOA. It is so incredible expensive, and there is no reason you should fail. You just have to be smart ;-)  

EA Failed Big Way! by Ivar Jacobson

Enterprise Architecture failed big way!

Around the world introducing an Enterprise Architecture EA has been an initiative for most financial institutions (banks, insurance companies, government, etc.) for the last five years or so, and it is not over. I have been working with such companies and helped some of them to avoid making the worst mistakes. Most EA initiatives failed. My guess is that more than 90% never really resulted in anything useful.

 

Why did people fail? There are many reasons, but they can all be summarized by the word smart. They were not smart when they selected solution. They were not smart when they selected way of working. They were not smart when they organized their business and IT resources. Building an EA is not rocket science. 

There are two common reasons specific to EA failure:

  1. Focus on paper-ware instead of executable software.  When enterprise architects work in an ivory tower without caring about what can be implemented, they produce too much models and documentation without executable solutions.  Enterprise architectures should be implemented incrementally, starting as early as possible. We call such architectures for executable EAs. 
  2. Big gaps between layers instead of seamless relationships.  Usually there are several layers such as a business layer, an application layer, a data layer and a technical layer. There are huge gaps between these layers which results in very brittle architectures. It is like trying to stand on a skateboard which is on top of another skateboard which in its turn is on top of yet another skateboard, etc. To have a chance these skateboards need to behave like one which is hard enough.  Thus the relationship between the business layer end the application and data layers are not straightforward and the relationships between the application layer and the data layer is as hard to manage as it was 20-30 years when we used methods like functional decomposition, or structured analysis and design. It is amazing that people haven’t learnt anything from component based development (with or without objects).

There are many other mistakes that people have made, many of which are related to organizational change in general. Examples include lack of business support for EA, not communicating the scope and purpose of EA, no strong IT leadership etc. In addition to these common challenges, all it takes to succeed at EA is to use best practices for modern software development, avoid upfront academic modeling, build both top down and bottom up, look upon the whole enterprise system as a system of interconnected systems.

Many of the companies that failed are now looking for the next silver bullet – Service Oriented Architecture SOA. To me SOA is what EA should have become. SOA can be described as EA++ -- it is Enterprise Architecture made better. SOA is clearly on the right path, but again adopting it requires that you work smart!  

SOA by Ivar Jacobson

March 30, 2004

Before being invited to Tallahassee, I had never heard about it. I flew in to the city in the morning and back in the evening. I spent a day with the State of Florida. Bill Lucas did a wonderful job in making me feel very welcome and everyone I met was very friendly and interested in my work. I enjoyed my day very much. One of the questions we discussed was web services. The last couple of years services have become important elements for describing and building software. As with everything new, the software world has a tendency to believe that something fundamentally different has surfaced and that a new way of thinking is required. As a consequence we have got a whole arsenal of new concepts around the concept of services. We have got “service-oriented architectures”, “on demand”, “utility computing”...you name it. However, there is nothing fundamentally new with services. To organize software in services is an old practice.

Services were once a very important construct in RUP, actually in the version of RUP that we called 3.8. (It was the version prior to Rational buying my company, so it was called Objectory 3.8.) Unfortunately, the RUP team thought that downplaying services in RUP would make it significantly simpler. I disagreed with this opinion, but accepted it because almost everything else was adopted. It was very hard to argue for service-oriented design when the concept hadn’t hit the software industry. With Service-Oriented Architecture (SOA) on the table, the need is there.

In 1998, I wrote about services in the Unified Software Development Process book: Apart from providing use cases to its users, every system also provides a set of services to its customers. I made a distinction between end-users of the system and the customer who purchases a system for its users. For instance, a bank system has users which may be clients of the bank, and the bank itself is a customer of the system (maybe buying it from some system integrator). A customer acquires a suitable mix of services. Through these services the system will provide the necessary use cases for the users to do their business:

  • A use case specifies a sequence of actions: a thread is initiated by an actor, followed by interactions between the actor and the system, and completed and stopped after having returned a value to the actor. Usually, use cases don’t exist in isolation. For instance, the Withdraw Money use case assumes that another use case has created a user account and that the user’s address and other user data are accessible.
  • A service represents a coherent set of functionally related actions - a package of functionality - that is employed in several use cases. A customer of a system usually buys a mix of services to give its users the necessary use cases. A service is indivisible in the sense that the system needs to provide it completely or not at all.
  • Use cases are for users, and services are for customers. Use cases cross services, that is, a use case requires actions from several services. A service usually provides several use cases or parts of several use cases.

In the Unified Process, the service concept is in analysis (platform independent modelling) supported by service packages. The following can be noted about service packages:

  • A service package contains a set of functionally related classes.
  • A service package is indivisible. Each customer gets either all classes in the service package or none at all. Thus a service package is a configuration unit.
  • When a use case is realized, one or more service packages may be participants in the realization. Moreover, it is common for a specific service package to participate in several different use-case realizations.
  • A service package often has very limited dependencies toward other service packages.
  • A service package is usually of relevance to only one or a few actors.
  • The functionality defined by a service package can when designed and implemented be managed as a separate delivery unit. A service package can thus represent some “add-in” functionality of the system. When a service package is excluded, so is every use case whose realization requires the service package.
  • Service packages may be mutually exclusive, or they may represent different aspects or variants of the same service. For example, “spell checking for British English” and “spell checking for American English” may be two different service packages provided by a system. You configure the system with one or the other, but maybe not with both.
  • The service packages constitute an essential input to subsequent design and implementation activities, in that they will help structure the design and implementation models in terms of service subsystems. In particular, the service subsystems have a major impact on the system’s decomposition into binary and executable components. This is of course only true if the development is going top-down with no reuse of existing components: legacy systems, packaged solutions, web services. And fact is, we develop more and more with reusable components.

By structuring the system according to the services it provides, we prepare for changes in individual services, since such changes are likely to be localized to the corresponding service package. This yields a robust system that is resilient to change.  

Given that most software of today is developed with ready made components, why would you like to design an analysis model (a platform independent model) with service packages. There is one good reason: we still need to understand what we are doing. Building software is about understanding, understanding components developed by different vendors, divisions, teams. An analysis model - maybe even just a partial model - used as a start help you overcome these difficulties.  

Software Reuse by Ivar Jacobson

February 8, 2004

Many organizations are trying to achieve software reuse. Many of them think that they will achieve reuse if they just put their components in some sort of repository. Then they just expect others to reuse their valuable assets. I never cease to be surprised that people still do that. We have said for at least a decade that you don’t get any substantial reuse by just posting components. It seems as if people, instead of learning from others experience, always have to make the same mistakes as others over and over again.

The book I wrote with Martin Griss and Patrik Jonsson on Software Reuse is very explicit about how to achieve reuse. The subtitle of the book is: Architecture, Process and Organization for business success.

It all starts with Architecture. You design an architecture which identifies which components are reusable and which are not, thus a layered architecture. Usually two layers on top of middleware are enough. The top layer contains application components, which are not designed to be reusable. The second layer contains the domain components intended for reuse by several applications in the top layer. The only practical way to get a domain component is by designing an architecture in which the component has a clear role. Without this architectural setting, a component will never really be reused.

Process comes next. The way you develop the domain components is different from how you develop application components. They are documented for reuse. We suggested in our book that you design a façade for each domain component. This façade contains an extract of the models of the component, an extract defining how the component can be reused. In case the domain is big enough (e.g. banking, telecom, airline), the façades can through careful design evolve into a more powerful tool – a domain-specific language (DSL). Instead of designing on top of the interfaces of the domain components, expressed through their façades, you will design using a domain specific language.

Domain components also come with a kit containing tools for its reuse, examples of how to reuse it, and other things that help in getting it reused.

Developing application components is different. Application components don’t require to be documented for reuse. Instead they are designed using domain components and through interaction with other application components.

Finally Organization. In order to get successful reuse, you also need to develop an organization which maps 1:1 to the architecture. This means that for each component in the architecture there is a corresponding responsibility in the organization. Higher level components are the responsibility of a subsystem-owner. Lower level components are the responsibility of an individual developer. Without such a simple mapping between architecture and organization it will be practically impossible to get good reuse. Such an organization will become an expert organization – people will become experts in a subject matter. It grows people that want to create quality products. I call such an organization an architecture-based organization as opposed to a project-based organization. A project-based organization has a line manager for each project. The manager should only care about his own project. Such an organization is counterproductive to reuse.

In an architecture-based organization, the project managers are not line managers. Instead they own the budget for their project. Using this budget they can buy resources from the line organization (that is the subsystem-owners with domain expert developers) to get their projects done in time. The line organization can work with several projects concurrently and they have to prioritize them based on delivery dates. Yes, there may be conflicts, but these are usually not large. The total costs are substantially lower because the domain components are developed and maintained by experts. Thus in the case of conflicts, more resources can be added to remove the conflict, and still the costs are much lower than with a project-based organization.  

The approach I just presented also has the advantage that you grow the reusable components as you work with real projects. You will start with an empty architecture, but you fill it as you go. You don’t sit there and speculate about what might be reusable; instead you design the components in the architecture that are first needed. They have probably not stabilized for the first project, so they may have to grow through several projects before they really become reusable without redesign.

To achieve reuse is not easy. People have tried simple solutions like class libraries and component libraries into which component designers post their components hoping that other people will reuse them. We have seen very little reuse coming this way. Managers have thought it was the lack of incentives so they have stimulated all kinds of initiatives to get reuse. This is ignorant. You will get reuse, but only by standing on top of an integration of A, P and O, that is Architecture, Process and Organization. In another postcard I will tell you how you can move to an architecture-based organization from a project-based organization without revolution. It is really fascinating to see how easy it can be done, given that you are smart. However, that is not a problem, since you are smart, aren’t you?