Will MDD Ever Come to Fruition? by Ivar Jacobson

I am often asked the question: “will MDD ever be successful and will the tools ever really be put in place to support it?” and I was recently asked this again, so here are my thoughts.  

The following is solely my opinion and can be argued or agreed with, but it comes from 15+ years of experience building application and data modeles, modeling tools including ERwin, Rational Rose and other tools, writing 2 books on UML and working directly with clients who are modeling. 

Model Driven Development in concept is great, but to date the tools and the people who would have to use them have not been able to keep up with the concepts.  There are some very good tools on the market like Telelogic Tau, Rational RoseRT and a few others which do a fairly complete job of generating the needed code, but this is usually focused in the "systems" space and has not translated well to IT as it is based on state's and generating the logic from those states, etc.   

On the IT side, we do have similar concepts, but they start from business process models using tools like WebSphere Business Modeler and similar tools from BEA for example which connect to Business Process runtime engines and generate the connection points of existing applications to automate communication and business process execution.   

This all said, the uptake MDD has not been that of other areas for what I believe are 3 reasons: 

1.      Developers are developers because they want to write code and don't see models as being complete enough nor for their role to build models beyond simple architecture structures.

2.      Most projects today are starting with something already in production and therefore doing modeling of the architectures, use cases and processes are quite good to understand what needs to be done, how it connects together and prioritize work, but it makes it difficult to generate a system that is only a piece to connect up or an update.

3.      I believe #3 can stand on its own, but also lends itself to the first 2 comments and that is the creation of a "Black Box".  Using MDD tools creates a black box effect where runtime generation, transformations and other constructs are managed by a proprietary engine which is difficult to alter and manage. 

a.      For Developers, they often think they can do it better and faster and don’t want to rely on something they cannot touch. 

b.      Because of the black box approach, it is often requires a rewrite of the engines that have already been put into place for the existing systems causing added cost that nobody is willing to fund. 

We have tried in the past similar types of technologies which few have been successful as well.  Runtime Data Access is a great example where tools and companies popped up in the late 90's which created object-to-data mapping and automated the creation of the runtime access at what they claimed to be much faster and cheaper than doing it yourself.  Yes, this was good at least in theory, but few bought into it.  Why?  Black box approach, hard to test, developers think they can write faster algorithms, management doesn't trust something they cannot see, etc.  This is very similar to MDD and its lack of success in my opinion.   

That all said, I do have some collegues who are using MDD on some very interesting projects building components for airplanes for example which they feel are quite successful, but these also seem to be few and far between.

Giving requirements a bad name by Ivar Jacobson

A while back I was giving a presentation on requirements errors and someone made the observation that they thought the term "requirement" was, in itself, misleading. The  gist of his argument is that the very term implies something that is "required", that it has to be done.  In reality, most "requirements", at least as initially stated, are somewhat vague statements of things that could be done, but they often need quite a bit of work to determine what is really needed.  It would be better to call them, at least initially, "wishes".

 

Unfortunately we are probably stuck with the term "requirement", but it can help to realize that as initially identified "requirements" tend to be vague, filled with incorrect assumptions about what the real problems and solutions look like.  Some organizations call these sort of things "stakeholder requests", which gives them a way to capture them without anyone inadvertently assuming that these are the actual requirements.  They then go through an investigation and refinement process before discovering the actual "requirements".  Along the way, various other kinds of things may be discovered: needs, goals or objectives, desired outcomes, as well as terms (such as would be found in a glossary), among other things.

 

The important thing is not how you categorize (this can get out of hand as well), but that you recognize that even when someone tells you something is a requirement you should not just accept their statement at face value - you need to make sure you understand the real need, and especially what they need to achieve. Only by doing so will you know whether the thing they are asking for is really "required".


Do we need Event-Driven Architecture? by Ivar Jacobson

A software system with an Event-Driven-Architecture (EDA) is built around the idea that events are the most significant elements in the system and that they are produced somewhere in the system and consumed somewhere else in the system.

The business value is that you can easily extend such a system with new things that are ready to produce or consume events that already are in place in the system.  Of course you can add new events as you go. 

Yes, this is absolutely great.  If you build something new there is no reason why you shouldn’t use this kind of architecture.  However, focusing on the events is not the only thing you should do.   

Instead, you should just build an architecture in which you have components or services and some kinds of “channels” between some of these components.  Over a channel an event can flow from one component – the producer – to another one – the consumer.  These components are loosely coupled and can exist in a distributed world.  Some of these events are such that you broadcast them to anybody that has subscribed to them. 

Thus don’t constrain your architecture to just be event-driven.  There is really no money to save by doing just that.  Let it be components with channels.  The channels I am talking about were already adopted in the telecom standard SDL back in 1982.  In EDA it is basically a mechanism for brokering events.   In the OMG standard CORBA from the early 1990s it was called the “Event Service”.  What a coincidence!  Actually one way of thinking about EDA conceptually is really that it is all that CORBA was meant to be, but in the Web/Internet world. 

The most interesting components are services.  You get service-oriented architecture at the same time, and more. 

However, those of you who think this is fundamentally new have really not done your homework.  It is probably true that the three letter combination EDA is new as it was for SOA.  We have also got some new great platforms that make it easier to implement these ideas.

Over the years I have seen trends in the component world that put more focus on the components than the channels (and thus the events) between them.  Other times it has been the other way around.  However, there is absolutely no reason to choose.  You should allow for both. But what we don’t need are more buzzwords. They don’t help us at all. 

To summarize, you should go for a component architecture without any compromises.  This is what made the Ericsson AXE system such an incredible success story more than 30 years ago.  And thanks to its architecture it is still probably the best selling product of its kind in the world.  However, Ericsson had to build its own infrastructure managing components with channels since such solutions didn’t exist at the time.

Of course, this is still new to people who have not previously developed a component architecture.  Thus those people have to come up to speed and that means training and mentoring.  And, to start with you need some good technical practices.  It is as easy as that! 

 

 

Measuring Project Success and Managing Expectations by Ivar Jacobson

There are a number of studies that cite poor performance of software projects - The Standish Group being the authors of one of the more often cited, their Chaos Report (and old version from 1995 is posted here, and although the data is old the conclusions are not dramatically changed).  The gist of these studies is that the majority of projects (as high as 70%) fail when measured against original schedule, budget, and expected features. I would be the last to argue the general conclusion: that it is very hard to manage a project to success. 

Most projects lack clear direction and purpose, and many are rife with disagreements about what success looks like.  There is, however, something in the assumptions behind these studies that rings hollow: that the initial schedule, budget and expectations for projects is a reliable milepost against which to measure. Most projects are vaguely conceived at best - they often lack a clear understanding of why they should exist and what problems they need to solve.  At their initiation they are usually poorly scoped and vaguely purposed, and the funding associated with them is often assigned based on an  allocation of an arbitrarily assigned budget.  Their schedules, at least those produced at the start of the project, are largely speculative endeavors, a mixture of gut and guesswork, that bears little basis in reality.  Measuring project performance against the initial schedule, scope and budget is of little value except to illustrate the point that there is a large disconnect between the expectations of business sponsors and the ability of teams to deliver against those expectations.  There are, to be sure, rampant problems with performance, but there are also widespread woes of expectations that are just as important to address.

Where should we start?  The first place is probably with project funding and measurement. The real thing of importance to measure is whether the project produced (or exceeded) the business value  expected of it.  If a delay in the project caused a market window of opportunity to be missed, that is significant, but it is the decline in value delivered that needs to be measured, not a schedule variance that cannot be correlated with economic activity.  Forcing a focus on business value produced would also put the right attention on the role of the business in following-through on their assertions of the value that will accrue from having the solution.  Requesting projects based on business needs has an opportunity cost - choosing one project over another should affect the value delivered to shareholders - and accountability for assertions by the line of business is just as important as accountability for project delivery.

If we shift our attention to value delivered rather than meeting schedule and budget, we may free the development team to find better ways to deliver the value, which may or may not include the initial set of features envisioned by the business sponsor.  Initial feature lists are usually vaguely conceived and don't provide a very good target for delivery.  Work is usually required to ascertain the real needs from this initial list of "features", some of which contribute to satisfying real needs but many of which are simply good initial starting points for discussion about real needs.  It may very well take longer than expected to solve the real problems (it usually does, as we all tend to be more optimistic than we should about how long things will take).

The problem is that most teams are set up to fail from the start.  By measuring them against budgets and schedules based on arbitrary assumptions and often a poor understanding of the real business value that needs to be produced, we find them constantly struggling against a plan that cannot possibly succeed.   Measuring against initial schedule, budget and expected features is not merely meaningless, it's actually part of the problem.  We need to shift our focus to better articulating problems to be solved and needs to be satisfied, and measuring business value produced.  Once we start to do that, we can focus on the plans and milestones needed to ensure the delivery of business value.

Theory X and Theory Y by Ivar Jacobson

Management styles have a huge effect on software development teams.  A significant shift required in  the movement toward a more agile approach is a change in the way that teams are managed and measured.Management styles can be described in  a number of ways - Theory X and Theory Y are popular approaches dating back to the 1960's but still applicable today.   Basically, Theory X assumes that people are basically untrustworthy slackers who need to be constantly monitored and told what to do and are only working because they need the money.  Theory Y assumes that people want to do a good job and are motivated by more than money, and that people produce their best results when they are working in a supportive environment that frees them to be creative and productive.It should come as no surprise that a necessary condition for a movement toward agile teaming is a "Theory Y" management culture - a team trying to adopt an agile approach in an organization with a "Theory X" management culture is doomed to failure and frustration; it will constantly be fighting the management system and the surrounding culture. In these organizations, the management culture (and its supporting measurement system) must change along with the team approach in order for both to be successful. Before you can change it, however, you need to understand it.

Agile or not Agile – that is the question ? Or is it ? by Ivar Jacobson

I often get called into companies who are thinking of going "Agile".  They have invested many thousands of dollars on a very complex SDLC taking the best ideas from the industry, but that process is not being followed and their teams are effectively not working together, and now think that Agile will solve those problems.  They come to IJC with a simple question "how do I adopt agile ?".  But when you dig, you find that this question is not as easy as it would appear.  When you question their motivation and their constraints, you find a whole list of issues and problems that Agile by itself can not solve.  Issues can range from off-shoring, to quality and performance.  Issues that no one approach can solve.

Agile software development seems to be a way of describing everything that is good in software process today.  It combines techniques and team practices with ways of changing organizations.  Thus, when people talk about agile they are talking about so many different things it is really hard to get a handle on them all - it is like saying I like European food and everyone knows what that means.  Do I mean Spanish, Swedish, French...?  Of course I can not mean English.  There is a lot of great stuff that has the label Agile, but the area that I will discuss today are the team techniques that Agile has brought to the table.  In particular SCRUM. 

First let's define a structure that SCRUM fits into.  SCRUM provides a fantastic set of very simple project management processes that help teams better function, but it does not provide great guidance in the areas of actually building software or making sure that project fits into something much larger.  Thus, I always position SCRUM in the context of three other processes.  Firstly, on top is an organizational process.  These are typically described in SDLC milestones, gates or phases.  They represent the key stages a project MUST go through from a funding and control point of view.  The second view of process is the team.  How does the team function to deliver software in support of this lifecycle?  This I label as team.  The third is the techniques that an individual must employ to actually build software.  Techniques such as OOAD or Test Driven Development - Also this is where techniques such as Use Case Driven development fit. SCRUM fits nicely into the middle layer - It is a set of team techniques that really help the team become a team.  But without the top and bottom views of process being in place SCRUM by itself would not work. 

Firstly SCRUM provides a very simple set of roles.  Product Owner (the go to person who owns the problem you are solving), the SCRUM Master (the person who runs the SCRUM meetings and protects the team) and the Development Team (the people that do the work).  I also often add a fourth role that of the Technical Owner, the person who owns the system from a technical point of view and is the go to person about all things about the architecture.  Secondly it provides three meetings.  A kick off meeting at the start of a sprint.  Oh, I had better introduce the idea of a sprint - A sprint is like an iteration.  A small chunk of time when stuff happens - has a clear set of goals and delivers stuff.  In terms of what it delivers they are classically stories, but can also be features or other units of stuff.  What is important about a sprint is that real work is done, that means working software is produced, tested and deployed (maybe to a pseudo production environment).  The second type of meeting, and the one that gets all the press is the daily Scrum - this meeting is short, say 15 minutes, and focused on three questions.  How did you do, what are you doing and what is stopping you?  These three questions enable the project to move along at a rapid, focused rate.  The third meeting is all about retrospective and proving you did what you were promising in your sprint.   So, SCRUM provides a great way of organizing the team, but without the lifecycle the team can not decide on what sprints they will do and the management can not appreciate the value of this sprint in the context of something bigger.  Without some techniques for building software individuals do not know what to do. 

So SCRUM must always be part of something bigger to really make an impact.  I would therefore argue that agile on its own is like a single food item; really to enjoy it, it needs to be put into the context of a meal.

Welcome to my new blog! by Ivar Jacobson

For many years I have written postcards to those of you who have subscribed to them on our web sites.  I will discontinue writing the postcards, but instead talk about the many different aspects on software development on this blog.  And I am not the only one who will write for you.  I am so lucky because I am surrounded by world-class experts coming from around the world.  These people will publish notes here.  Thus, you will here from Kurt Bittner, Ian Spence, Svante Lidman, Pan Wei Ng and several other people.  I will introduce them as they come forward.We will use this blog to talk about our visions and our results.

I like to dream about the future, but I also would like to reach that future at some point in time.  I believe that when it comes to software, dreams only come true if the dreamer drives the realization of the dream.  Too many people have dreams.  Very few actually have the guts to make their dreams a reality.

In 1992 I wrote an editorial in IEEE Software that started like this:  “We all have dreams. Just a little over 25 years ago, I had a dream. I dreamed that the technology known today as "object- oriented" could be used to design telecommunications systems. I dreamed that this technology would be generalized and given a wealth of wholly different applications. I dreamed that the computer scientists who - in those days - ridiculed this technology and saw it as a dead-end street would one day accept it. I dreamed that this technology would in time become an indispensible element in the mainstream of computer science.”

In my summary I wrote: “Our long-term goal (which I hope we can achieve within another 25-year period) is to become as industrial in our handling of computer techniques as engineers are within their various disciplines. This requires, among other things, development of the architecture in large- scale object-oriented systems; it requires that the development process support project management as well as product management; and it requires that we find other, more sophisticated languages - specification languages as well as programming languages - compared with those we have today.”

Now 15 years after I wrote this article, it is interesting to see how far we have gotten.  We have got UML which is a standard modeling language for software development.  Before UML we had lots of different notations with almost no semantics.  UML basically took all of these notations out of the picture and replaced them with one single notation with reasonable well-defined semantics.  It is not perfect but it is practical.  It is being applied around the world for all kinds of software and for all kinds of models including business, requirements, architecture, analysis and design. 

When it comes to process and methodology, we have also made substantial progress.  When I wrote the IEEE article there were more than 26 published methods identified by the OMG (Object Management Group).  The most successful one of these methods (and tragically basically the only one that really survived) was Objectory which became the Unified Process. 

In parallel with the success of the Unified Process, new kinds of methods have come into play.  The most successful of these other methods are labeled agile.  However, the meaning of agile has been confused by its proponents.  What really is unique about agile is that it cares about people.  Thus it is about social engineering.  No process has ever developed software.  Only people develop software.  They may get help by tools.  They may get advices from process or methodology, but eventually we must make our people motivated and competent.  Nothing is as effective in getting software quickly and at low cost as having motivated and competent people.

Given that no methodology has everything you need, and even if it has most of what you need, it won’t be the methodology you want to use in ten years time.  New ideas and new technologies will come into play.  These will come from individuals and companies throughout the world.  If you try to extend your methodology with these ideas, it will grow until it collapses under its own weight. Instead of publishing branded complete processes or methodologies we should focus our skills on single practices.  Usually a methodologist is good on one or a few practices, but not on all practices that a team needs when developing software. 

Now it is time to formulate one of my new dreams.  My dream is that great people around the world will contribute their best practices to the world maybe in the form of a community.  These practices can be of different kinds.  They can be technical such as those from the Unified Process camp, or they can be social engineering practices from the Agile camp.  A team can then select from these practices and compose the selected practices into a way of working that helps the team develop good software, quickly and at low cost.  This is my dream.  Now, we are not just dreaming.  We have come a long way in realizing this dream.  We have developed a practice platform EssWork where you can find a whole set of practices, you can author your own practices and you can select the practices you like and compose them.  There is still a lot to be done before we are finished, but with your help the dream will come true.

Cheers

Ivar 

Let's be Smart by Ivar Jacobson

One of the most popular movements in software development in recent years is the move toward agility. Today everyone wants to be agile. That is good! However, the essence of being agile is being smart. I have for several years expressed that the most important character you need to have to be a great software developer is to be smart. In several of my columns I have summarized what you need to be successful by saying: You need to be smart! What does that mean? Most people know intuitively what “being smart” means in everyday language, but what does it mean for software.      

It is not smart to model everything in UML for instance when building software. It is not smart to model nothing and go straight to code. It is however smart to find exactly that something that is of importance to model and code.  

Advice: What is that something? It is about the most essential use cases and in particular about the most essential scenarios through these use cases. It is about the components and in particular the parts of those components that realize these essential scenarios. Thus, it is about the essentials. Now you may ask what makes a scenario essential. An essential scenario is the response to the question: “what are the most important scenarios the system is supposed to do”. Which scenarios exercise the critical parts of the architecture? Which scenarios need to work in order for us to say that the highest technical risks have been eliminated? It is not smart to write requirements without caring that these requirements are testable. It is smart to make sure the requirements also are test cases. 

Advice: try use cases since they are also great test cases. It is not smart to work in a waterfall manner, first specify all requirements, then do the design and the code, and finally test it all. If you do, you will discover serious problems with performance, architecture, usability too late. It is smart to first build a small skinny system and then build a little bit more, and a little bit more, before you release the system to customers. Each time you build something, you must be able to run, validate and test it. 

Advice: use a controlled iterative approach such as the iteration practice in the Unified Process or sprints in scrum.It is not smart to run off and build a lot of “stuff” before first assessing if you can source the whole or parts of the application (Open Source or commercial offerings). It is not smart to develop software with a process that cannot scale if your system is successful and customers want much more. It is smart to use a way of working that is no more than what you really need, but that can grow as you succeed with the product. 

Advice: make sure your process has a dial for each interesting process parameter so you can turn the dial to the proper position for your project.  It is not smart to throw out your existing process and adopt a completely new process. That is doomed to fail in most cases. Not everything you did in the past was wrong so why should you start all over. It is smart to improve your process in small manageable steps.  

Advice: add one practice at the time. It is not smart to find a shortcut that in reality becomes a detour, such as skipping requirements and going straight to code.  It is smart to do enough requirements to find what to use to build your first increment, your skinny system. In general it is not smart to be extreme in what you do such as: model everything or model nothing, follow a strict waterfall process or an unstructured iterative approach, throw out what you have and start all over. It is smart to be balanced to do what is needed right now but with an eye to the future.  Above I have given a number of examples. In each case, there are some ideas on how to think about being smart. And each case can be expanded further. You will become smart with experience, but experience on the other hand is not a guarantee for being smart.  Of course eventually, it comes back to you. Smart is not the same thing as being intelligent. You can be intelligent without being smart. And you can be very smart without being very intelligent. Smart is not the same as having common sense. You can have common sense without being smart, but if you are smart you must have common sense.  

Smart is to do exactly right, not to find a broad solution that is just about right 

Executable SOA by Ivar Jacobson

Today everyone talks about Service-Oriented Architecture. And as with all buzz words there are many interpretations. As the name suggests SOA was originally about architecture, but as time goes by its proponents put more and more meaning into it in the same way as proponents of EA put more and more into it. This is why some people have nicknamed SOA to Service-Oriented Ambiguity.      

What is SOA at its core? It is a kind of software architecture which allows you to connect course granular components called services through well-defined interfaces. These components can reside on any kind of platform (mainframe, client/server, etc.). They can be web applications, java or .NET applications, ERP systems or your own legacy systems.  

What is the business case for SOA? Since, services are essentially reusable components, you can efficiently support new business processes by using old services and interconnecting them. Since, they have well-defined interfaces they can be replaced by any other component complying to the same interface. 

Is this really new? Yes and no. No, because SOA has existed almost forever in the telecom space. The most successful commercial product ever built in Sweden was the Ericsson AXE switch. AXE’s success came to some extent from that its architecture was service-oriented. However, Ericsson had to build its own infrastructure - a unique computer architecture and operating system. My Software Reuse book (Software Reuse: Architecture, Process and Organization for Business Success, ACM Press) from 1997 is essentially a book on SOA. Yes, it is new because big vendors like IBM provide an infrastructure that allows services to reside basically anywhere in a network. 

How to succeed with SOA? It goes without saying that you need good people – leaders as well as champions. And they need to be smart – have common sense, practical, do what is needed not more, and of course be competent. Having said what is obvious let’s move to a more techie perspective:  

Design top down, bottom up and then find the balance as a trade-off between value and cost. Top down means you start from some business process ideas and deduct a solution as a set of interconnected components. If you just go top down it will usually be unreasonably expensive. Bottom up means that you start from what is available/reusable, such as legacy systems, web services, ERP systems, whatever, and build something close to what you need by interconnecting these components. This may not be exactly what your business modeling people were dreaming about, but something you can build quickly with low costs. Finally, you balance the top down and the bottom up and get what you may call a good enough solution. Your solution reuses a lot, but you may also have to build something new. This is smart! However, it is not enough. 

Do this very light (don’t major in modeling or post-its), develop quickly a road-map, build in small executable steps (even if the enterprise system is huge). And beware of the service companies that want to sell you big solutions – lots of billable hours without being accountable for delivering executable software!Don’t fail with SOA. It is so incredible expensive, and there is no reason you should fail. You just have to be smart ;-)  

EA Failed Big Way! by Ivar Jacobson

Enterprise Architecture failed big way!

Around the world introducing an Enterprise Architecture EA has been an initiative for most financial institutions (banks, insurance companies, government, etc.) for the last five years or so, and it is not over. I have been working with such companies and helped some of them to avoid making the worst mistakes. Most EA initiatives failed. My guess is that more than 90% never really resulted in anything useful.

 

Why did people fail? There are many reasons, but they can all be summarized by the word smart. They were not smart when they selected solution. They were not smart when they selected way of working. They were not smart when they organized their business and IT resources. Building an EA is not rocket science. 

There are two common reasons specific to EA failure:

  1. Focus on paper-ware instead of executable software.  When enterprise architects work in an ivory tower without caring about what can be implemented, they produce too much models and documentation without executable solutions.  Enterprise architectures should be implemented incrementally, starting as early as possible. We call such architectures for executable EAs. 
  2. Big gaps between layers instead of seamless relationships.  Usually there are several layers such as a business layer, an application layer, a data layer and a technical layer. There are huge gaps between these layers which results in very brittle architectures. It is like trying to stand on a skateboard which is on top of another skateboard which in its turn is on top of yet another skateboard, etc. To have a chance these skateboards need to behave like one which is hard enough.  Thus the relationship between the business layer end the application and data layers are not straightforward and the relationships between the application layer and the data layer is as hard to manage as it was 20-30 years when we used methods like functional decomposition, or structured analysis and design. It is amazing that people haven’t learnt anything from component based development (with or without objects).

There are many other mistakes that people have made, many of which are related to organizational change in general. Examples include lack of business support for EA, not communicating the scope and purpose of EA, no strong IT leadership etc. In addition to these common challenges, all it takes to succeed at EA is to use best practices for modern software development, avoid upfront academic modeling, build both top down and bottom up, look upon the whole enterprise system as a system of interconnected systems.

Many of the companies that failed are now looking for the next silver bullet – Service Oriented Architecture SOA. To me SOA is what EA should have become. SOA can be described as EA++ -- it is Enterprise Architecture made better. SOA is clearly on the right path, but again adopting it requires that you work smart!  

Page 12 of 16« First...78910111213141516