Feedback: Software Development Paradigm Trap

John Stanton - 6/1/2006

I was deeply impressed by the wisdom in your article on software quality and methodology. Thank you for sharing your knowledge.

It resonates particularly with me because I had come to similar conclusions, but had not thought it through to the point of the devastating "universal bolt" type of analogy.

We have, however, used the general principle to create some highly reliable pieces of software by discarding the use of "black box" software modules and coding from the ground up, building our own certified modules as we go. The results were quite different from the dire predictions of our critics and actually the projects completed in a quite reasonable time frame and deployed as fast or faster than those cobbled together from available parts. The quality was better and there was a welcome bonus in the form of a much higher level of performance and maintainability.

You other point about software which doesn't fall to pieces when a minor component fails sounds so obvious to me, educated as a mechanical engineer. It is not an accident that an aircraft with part of the wing shot away makes it back to base. It was designed that way. Over the years we learned subtle ways of applying engineering principles to program design and coding so that defects would not necessarily create a catastrophe.

John S

Mark Bereit - 6/10/2006

John,

Thanks for your comments!

I, too, have built and helped to build some great code, if I say so myself, trying to grow a library of useful components along the way, with good payoffs. Although I note that the difference between a "black box" module and our own modules built as we go is only a matter of gut-level understanding. Add a new member to my development team and there is no difference between the module we built and the module we bought: they are both black boxes now.

I am just increasingly concerned about complexity, because the pressures driving more code aren't letting up, and we as humans and as teams and as users of present tools aren't able to scale the same way. And the thing that makes me nuts is the knowledge of how little it takes to bring the whole system down:

    *pFlag = 1;

One write through an uninitialized pointer is absolutely fatal. Absolutely. Because there is no practical limit to the harm it can do. It will, naturally during testing (in the debug build where memory locations are all different from what you ship) have no ill effects and never show up. Best case, it writes to totally unused memory and is benign. Other times it writes to data memory and corrupts information, inconsistently, yielding the occasional wrong result. Other times it writes to memory that doesn't exist, which depending on system architecture may be benign, may cause a protection fault (often executing untested exception handler code!), or may cause an unterminated bus cycle, freezing up the works. And still other times, where code and data are in the same memory space and code is not write-protected, the write changes the code, leading to disaster long after the actual write, and virtually impossible to correctly diagnose. With the rise of complexity the chances of discovering this intermittent fault approach zero.

If the ability of a module to do harm is to be constrained to that module, this sort of thing can't be possible. My suggestion of putting every module in its own MPU is one drastic solution. Obviously, putting every module in its own memory space, enforced by the processor, is another. Either approach requires drastic rethinking of how modules work together. Another solution is to use a language where pointers, as we C/C++ programmers know them, do not exist: every reference to every data memory location is meticulously (and expensively) checked and cleared by the runtime system. Java does some of this (while being reviled for its poor performance!). But we need to do something differently. If my article prompts people to take a step back and think about things like this, then I have accomplished something.

Thank you for sharing your thoughts. It has been interesting to me to hear additional perspectives and suggestions, and I thank you for yours.

Mark Bereit