I agree that a Theory of Software would be valuable. I see the development as having 2 parts:

1) observe what works and what doesn’t in practice and extract workable principles based on it — the Agile approach strives to do this

2) begin with first principles and form hypotheses, develop practices based on the hypotheses, and return to step 1.

I will attempt a summarization of a majhor set of reasonig from first principles.

Assert: Software development (of the sort that needs a Theory of Software) is done by teams of people using practices and processes in an organizational / management / cultural context.

Assert: These people are knowledge workers, and much can be learned from a study of the work done with other knowledge workers. The book “Changing Softeare Development” Learning to Become Agile” by Allan Kelly makes this case very well.

Assert: Software encapsulates knowledge: software allows us to do what the developer does without knowing what he knows.

Corollary: Since software is developed by people, the process sthould take into account their strengths and weaknesses. First and foremost, software development is about strategies for managing complexity and supporting attention to detail, both of which are weak in people. At any level of abstraction and at any point in the space, the size of the “working set” has to be small enough to be encompassed by a single mind. We use various mechanisms of decomposition, chunking, and pattern recognition to reduce the complexity to a level that we can manage.

Corollary: Software development can never proceed in a totally top-down manner. People think associatively, possible even holographically. Our systems whould allow us to capture knowledge whenever it occurs and make us of it later. David Parnas in “A Rational Design Process: How and Why to Fake it” contends that no system of any size was ever built strictly top-down and none likely ever will be, but that we can endeavor to construct the trail of documentation that would have resulted had we been able to do so.

Corollary: Since software is built by teams, all of the team-building and communication processes are of great importance. As the productivity of individual varies dramatically, so does the productivity of teams. When a team jells, individuals do what is needed when it is needed with little or no direction, at any point the focus is on the person who is “on” and has something to contribute at that time. Individual strengths are capitalized upon, and individual weaknesses are strengthened.

Corollary: Since development is done in a management context, a workable Theory of Software must address that context as well as the skills of the developers. A team whose management requires that 100% of the requirements be done and frozen before development can begin starts out with a tremendous handicap. The Future Combat Systems Program run by Boeing for the Army was planned with a single requirements phase to cover nearly 3 decades worth of development, and it doomed the project from the start.

Corollary: Since this is knowledge work, our tools and practices should facilitate that work. There is a reason that wikis work well in discussing a proposed system. There are reasons why Use Cases work well. Humans have been using stories and narrative to communicate for as long as there has been spoken language, and we need to facilitate and utilize that communication. Note that all such collaborative efforts have found the need for regular refactoring when an are gets too jumbled, or enough work has been done to be able to discern usable patterns in the material. Conventional documents have more in common with PhD theses than with communication tools. Requirements as merely lists of shall statements lose all context, grouping, rationale, and the coherence that makes them understandable.

And now we can begin to look at technical issues.

Assert: At every level of abstraction in the development of a system, the general activity is the same: Given a source of system objectives (needs, desired outcomes, functions, and features) evolve concurrently a system concept (concept of operations), an architecture (what is doing the work), and an expanded description at the new, lower level of abstraction. The process is essentially the same at the high end of system concept development from needs expressed by potential users and domain experts to software coding based on design documents. We don’t need different practices for each level of abstraction.

Corollary: Where we use natural language, we must endeavor to use it with the same precision as is demanded during coding. Names need t be unambiguous, statements need to be clear and complete, we need data dictionaries to support controlled vocabularies and possibly even full ontologies in the problem domain.

Assert: The approaches we apply to code to improve clarity, modularize and group descriptions, and reduce duplication work at all levels of abstraction. The application of aspects to Use Cases is one good example, but there are others. Use of interfaces and patterns in requirements can reduce volume, improve clarity, and improve maintainability. For example, for disk-based data, we used to use CRUD (Create, Read, Update, Delete). I routinely see these called out as separate actions on individual data items today. These things cannot be tested independently. They need to be grouped into the equivalent of an interface and specified with a single requirement.

There is much more to be said, clearly, but I believe that detailed effort on the above with application of experience in software development can provide the basis for a Theory of Software Development that is robust and workable because it si based in reality.

I first got into computers to get away from people, and discovered during a 45-year career that is is all about people. It is about more than psychology, which Gerald M. Weinberg has studied and written on so effectively I believe passionately that appropriate incorporation of knowledge management can bring a next jump in productivity.

–
Thanks,

Garold L. Johnson

Ron

We will continue “presenting the society with the image that software is fragile, full of defects and poorly done” because Software Engineering has little or nothing at all to do with these outcome.

The Customer has different priorities to the Engineer and these typically drive in a direction that leaves the result as fragile, full of defects and poorly done. The Customer wants the product done, the feature implemented and shipped to make a sale. Whatever that takes, and well crafted isnt on the radar especially if it will slow things down.

Well crafted doesn’t have to slow things down, but there isnt enough skilled developers to get things done quickly and well. Again, the Customer wants the cheap option here too.

What the Customer wants is cheap, emergency, temporary housing, but then get upset when you need to spend time and money making it work once they decide they don’t want to stay in it for 10 days, but for 10 years.

A discipline put in place by Software Engineering is not going to work, the Customer (Product buyer or Service buyer) needs to put things in place, but they wont, because they have other things that take priority.

I’d like to discuss this more, if you have a contact I could use?

Rgs, James.