Scaling Scrum? by Ivar Jacobson

Scaling Scrum?

Scrum is excellent for lightweight management of small projects developed by co-located teams. However, it is not designed for large complex systems, or systems that build a SOA (banks, car companies) or PLA (product line architecture for instance Ericsson) or for large organizations with or without outsourcing. Moreover, Scrum is just one practice. You need many more to develop software.

But can’t we just scale Scrum? Scaling is challenging. In the 1970’s people tried to scale to ‘programming in the large’. Later people tried to scale objects to ‘objects in the large’ (mega-programming). Both never made it. Hundreds of books and even more papers were written on the subjects, but it didn’t help. Instead ideas that were scalable at their roots became successful such as components and objects. At Ericsson we never had to struggle with ‘programming in the large’ since we went straight to components and later to objects.

Why is it so difficult to scale? Explaining this is not difficult but takes a lot of space. Instead let me make an analogy. Suppose you have been able to produce a new chemical substance by using a test tube and a Bunsen burner and the new substance helps with an important problem. Now you want to produce tons of the new substance. Nobody would dream of industrializing the laboratory method by simply building a larger laboratory with hundred meter long test tubes and fifty meter high Bunsen burners. Instead a chemical process would be built in which you would not recognize the original method.

Now, why do we need to scale Scrum itself? Why not let Scrum stay the pearl it is. Treat it as one of your practices in your portfolio of practices also including for example practices for requirements to test, architecture, components. Scale by composing Scrum with other practices.

To the fathers of Scrum I would say: Please, don’t extend Scrum with all kinds of other practices such as use cases, user stories test-driven design, etc. and brand the whole soup as Scrum. Doing so will make Scrum heavier and it will threaten the survival of Scrum because of deficiencies in these other practices.

We help companies around the world to adopt Scrum. However, we don’t see Scrum as the center of all you need. Instead, the center is the practices you already have in your organization, the ones your people already have adopted. Replace them carefully with better practices – one by one. Scrum could be one of these better practices but not the only one.

Do you agree with this approach? Now, it doesn’t happen by itself. It is clear that a practice-based approach is what is needed.  We have been promoting and working with practices since 2003.  Now, even IBM and Microsoft have started to join this movement. And this is really good for the industry. It is very smart!

12 Comments
  1. cwcentral | April 24, 2009 at 4:30 am Reply

    avatar

    Ivar, a very interesting perspective. I like it.

    We just started a new scrum project with a new team of 5 and so far so good. We originally started with a RUP-spiral hybrid approach, but it was received with a developer revolt: for instance, 4 of the 5 members didn’t want to write UML [guess who did (me)!] nor did they want to sit down spend some days writing use cases, initial list of requirements, etc… Scrum really attracted them: run with the short-term-fun stuff, push the complex things down the road, build an app, and just code. And management is happy cause they “see” productivity via user stories and points, but not the big picture, how it will integrate, scale, or transform the business over time (i.e. what they’ll really get since it’s assumed by the devs that “they don’t know”). What is interesting in our objective and scrum is we’re looking to develop an OS-level framework and enforce a set of unproven business logic–basically most seasoned folks call this “a solution looking for a problem”  . And this framework is for real-time applications, needs to scale, is mission critical, and needs to stand the test of time (5-8years). That’s where scrum has me worried since a lot of mission-critical systems require deep analysis, well defined test plans (though not fully test driven), directly impact long-term planning, a projected workflow, documentation that can be used by completely different teams and give us the ability to roadmap new functionality for 6-8 months, not weeks. It’s like building the Mars Lander, but in the fast pace, never-ending scrum fashion. Hence, originally RUP seemed more ideal at start (I’m the most experience developer, but not a domain expert by a long shot).

    From that I wish a lot of nomenclature consolidates to common terms. Iterative development has become too stylistic in order to provide a business view (track progress and “hours”) at the expense of technical robustness [in a design]. Having been through 2 scrum projects, a good number of RUP, spiral, cowboy and waterfall projects, the current trend is a lot of the same pattern, just a different syntax to communicate the same thing, “define a task”–and that’s why projects continue to fail–too much syntax and window dressing confuses everyone, even the senior developer. This confusion can be seen at the coding level as using Java and C# in the same project. It forces developers to follow a methodology rigorously & religiously. Even with Kanban/Lean upcoming as the “new” methodology, their syntax/nomenclature (and attitude) is reaching the same “religious” levels of scrum. Not taking a shot at scrum, it’s has good practices, I love it for small product libraries, but the internet chatter is overwhelmingly “it’s not our process, it’s you!” doesn’t help on the complex stuff (like scaling products/projects).

    In conclusion, until methodologies reach a UML-ish or XML-ish stability in syntax and nomenclature, we’re stuck in this hype cycle of trying new ways to explain good upfront analysis with iterative development and close customer support/understanding.

  2. How to Get Six Pack Fast | April 15, 2009 at 4:14 pm Reply

    avatar

    My fellow on Facebook shared this link and I’m not dissapointed at all that I came to your blog.

  3. Ivar Jacobson | April 7, 2009 at 7:58 am Reply

    avatar

    …let me continue…Agile methods are not designed to this.
    Here I refered to scale up, not to working is small teams. Note, I am not saying that Agile methods cannot be used. Everything CAN be used. First, since it is used by people, good people can always make it work. Second, good practices makes it easier to the good people to make it work.
    Finally, since agile methods were designed for the small colocated teams, it is likely that they are not the optimal guidance for large destributed organizations.

  4. Ivar Jacobson | April 7, 2009 at 7:42 am Reply

    avatar

    You are right, there are lots of failures in the industry. I would say that 90% of all EA initiative fails. Similarly with SOA and PLA. Cloud projects will most likely get screwed up as well.
    One reason that they fail is because they are not agile. Too much paperware, less focus on executables. I talk about xEA, xSOA, xPLA. Not a lot of upfront architecture, modeling, but get results.
    Large projects are the nature of big initiatives, but a large project should be delivered through many small teams working iteratively in some synchronized manner. Agile methods are not designed to do this.
    Due to its popularity many people try to scale up Scrum. For instance, Scrum has now adopted the lifecycle model of RUP — of course with some cosmetic changes — the phases are the same but with different names. That is OK, because most of what we do is cosmetic; there is very little new in Scrum…Iterative development as defined by Barry Boehm in his famous paper on The Spiral Model, around 25 years ago, has everything important that you get from Scrum, and it scales. Newer variants of his work, such as Iterative development in Unified Process and now in the Essential Unified Process is even more scaleable.
    Having said that I believe Scrum is a good starter with some new ideas, but too much cosmetics.

  5. David Vujic | April 7, 2009 at 7:21 am Reply

    avatar

    Isn’t the whole idea of SOA really Scaling down? Less of programming by ourselves, and using existing services by partners.

    There are many examples of failing Large Scale Projects in our industry. A lot of them (probably most of them) are not very much Agile.

  6. Ivar Jacobson | April 6, 2009 at 11:04 pm Reply

    avatar

    David,

    It is not a question of either-or. We need to do both-and.

    Since many years I have loud and clearly said that no team should be more than ten people. A large project may consist of many synchronized teams. There is a lot to be said here.

    Scaling up is necessary for large complex systems, for instance an Enterprise Architecture or for a Service-Oriented Architecture(banks, car companies) or for a Product Line Architecture (Telecom vendors, airline manufacturing) or for large organizations with or without outsourcing. Now we can add Cloud Compting to the crowd. The agile methods are not designed to help here.

    Thanks

    Ivar

  7. David Vujic | April 6, 2009 at 2:06 pm Reply

    avatar

    Ivar, perhaps it is the Scaling itself that is the real problem?

    Maybe we should focus on Scaling Down Projects instead of Scaling Up Methods.

  8. Ivar Jacobson | March 8, 2009 at 9:14 am Reply

    avatar

    In my whole career I have been cautious answering questions whether something can be used or cannot be used for a particular purpose. The fact that it can be used doesn’t mean that it is the best tool to be used for that purpose.

    Take the question, can Scrum be used for large systems and organizations? Of course it can! Basically anything can be used because people have to make it work. If you are passionate about something you can get that something to work for anything. That doesn’t mean that it is the best tool.

    There was a time when Pascal was used for specification of software, later Ada and Chill were used for the same thing. The lovers of these languages felt it worked great, and of course they could make it work. However as we probably all would agree today, they were not the best tools.

    Scrum was never designed to scale, so we shouldn’t expect it will scale easily. But can it scale? Of course, it can scale, anything can scale. However, is it the best tool to scale to large, distributed organizations working on large products with a complex architecture.

    No, it is not. You would need much more. You would need to separate crosscutting teams from the line organization. You would need to synchronize teams to form big projects.

    The line organization would not be the silos they are today, but more be resource centers and they would be responsible for preserving the architectural integrity. There is much more into this, but to make Scrum move into this space would remove the beauty of Scrum.

  9. Daniel Wildt | March 6, 2009 at 6:51 pm Reply

    avatar

    About changing an environment. Scrum will just help to bring problems to the surface. So I see Scrum as something good for any team that wants to see their problems to show up. With the right structure to execute continuous improvement they will do fine.

    Scrum will not tell you nothing about engineering practices, so you will have to use other practices from the market to evolve on that space.

    I have done an implementation of Scrum of Scrums that as far as I know it is working fine until today. Done that’ 3 years ago.

    And recently I worked on a project that use that approach to keep teams synchronized, that also worked.

    Do be able to understand how to use Scrum in a distributed way you need to understand distributed development, and more, you need to understand different cultures involved (people culture, team culture), all that is part of your problem, not just process and tools.

  10. Giovani Salvador | March 6, 2009 at 6:42 pm Reply

    avatar

    Hi Ivar.
    What measure have you done to say it does not scale? Compared to what project? Have you worked in a scrum project with teams the were not collocated? So I say I did. I worked in a big, big scrum project with 3 regions involved. As any other project we had our challenges but it succeded in the goal of delivering the product to the customer on time. The main point was: The client was available all the time via phone, IM and live meeting. Different regions were not a problem when we are based on trust and open communication.
    So global teams are not an excuse that scrum can’t be used.

  11. Hiranya | November 25, 2008 at 12:09 pm Reply

    avatar

    I also prefer an iterative approach where practices are introduced carefully and there are no big changes. There’s no one perfect way of building software. Whatever practice we introduce to an organization has to be carefully adapted to the specific characteristics of that organization.

    I have heard about Scrum of Scrums as a way of scaling Scrum. Scrum Alliance has some information on this. However I my self have not seen a project that has successfully done this.