the island of eleusis

IEEE Software recently published an opinion piece from Tom DeMarco titled ‘Software Engineering: An Idea Whose Time Has Come and Gone?’ in which DeMarco appears to declare the death of ‘Software Engineering’ as a discipline. Unsurprisingly, this is generating some waves in the tech blogging community, with pieces such as ‘Software Engineering: Dead?’ from Jeff Atwood’s Coding Horror, and Myles Eftos’s ‘Yep, Software Engineering is dead’.

From reading DeMarco’s article, it is apparent that what he is addressing is the emphasis placed on ‘control’ on software projects - that ‘software engineering’ is often taken to mean that it’s a means of controlling the outcomes of developing a software project. It should be obvious that DeMarco’s stance is one of reflecting upon his previous work and making some comments about how his work has been perceived in the software industry over the years. Aside from some “Aw shucks, I wish my younger self had been wiser” style reflection, DeMarco laments the fact that his work has been used over the years to justify the creeping bureaucratisation of the software development process. From that point, DeMarco goes on to prescribe that, rather than controlling minutiae, those in charge of software development should be treating their projects (by way of analogy) as unruly teenagers - by attempting to instil a sense of moral values and principles, rather than by being overbearing and all-controlling.

Unfortunately, Jeff & Myles appear to either be sensationalising, or reacting to a rather superficial reading of DeMarco’s opinion. Straight off the bat, Jeff points out DeMarco’s gravitas in his field, to hammer home the point that ‘yes, this is a big deal’. From there, however, Jeff switches to his usual agenda of elevating the romantic notion of ‘software craftsmanship’. Not that I’m averse to this position - au contraire, I’m a big fan of Jeff’s frequent expounding on matters of software development principles. What I disagree with, however, is the utter sensationalism with which this is being made here. At least, I am heartened that Jeff appears to have grasped the main point of DeMarco’s analogy: “If you want to move your project forward, the only reliable way to do that is to cultivate a deep sense of software craftsmanship and professionalism around it.”

I have no idea what has been said about either DeMarco’s piece or Jeff’s blog post on the 220 mailing list, though reading through Myles’ blog post left me slightly incensed at certain points. The analogy between building a bridge and building software is somewhat tired. Compared to building bridges, building ‘software’ encompasses a vastly larger scope. That much we agree on. You might as well be comparing building software with building anything made of materials sourced from the ground. From here, however, it seems that Myles launches into a tirade about his least favourite academic subjects, or at least the ones that he found ‘stupid’. What I fail to grasp at this point is the relevance to DeMarco’s piece… But I’ll play along anyway.

Yes, formal proofs are arduous, and can be rather silly for trivial programming tasks, but it should be obvious that such techniques aren’t applicable to all software development contexts. Would you apply formal proof methods to a small dynamic website you’ve just built? Hell no. Would you apply formal proof methods to mission-critical software that could potentially lead to injuries, fatalities or serious damage to the environment or property, if anything were to go wrong? Hell yes. (Well, some variant of a formal method, anyway) It’s a simple cost-benefit (or risk analysis) scenario: which costs less - the development process, or the consequences of something going seriously wrong? I’d further argue that formal proof exercises done in an undergraduate academic setting would be more geared towards teaching and learning the process of doing the proofs than anything else. This is undergrad CS that we’re talking about, with its endless assortment of silly assignments that require students to produce little of value beyond the end of their courses. I might also add that, even with a ‘simple’ program having 9 lines of code, this degree of simplicity may be afforded by a myriad of much more complex underlying supporting systems - the degree of simplicity of a program may simply be a matter of perspective, or from where you’re standing to look at the code.

What I find most troubling about Myles’ post, however, is the complete denigration of UML. Yes, I’ll admit that I’m a big proponent of UML. There may be a point about UML working really well with the waterfall model of software development, but to then imply that UML is almost completely useless - by association with the inadequacy of the waterfall model to address ‘real’ problems - seems a lot like missing the point of what UML is actually about. Yes, UML may superficially look like ‘architectural drawings or flow diagrams or something’. More importantly, UML is a standard notation to communicate about elements of software design. Beyond the ability of expensive software to create specky, neat and professional looking UML diagrams, UML itself is a tool for software developers to use when thinking about, crafting and communicating about the systems they’re building. By analogy, I would say that an ability to work with UML is like an ability to work with words and sentences and paragraphs. That is probably the reason why IT recruitment agents started using ‘Object Oriented Analysis & Design Skills’ and ‘Sound understanding of UML’ in their job ads - those *are* fundamentally useful skills to have.

As a programmer, I can have a conception of something that is a ‘class’ or an ‘object’ and be able to express the idea in a diagram, say on a whiteboard, while talking with fellow programmers, and still be able to more or less directly translate this concept into code using Java or Python. Using UML, I can model relationships between such classes and talk about such relationships - beyond mere representation, UML can be a valuable tool for solving software design problems, especially in team environments. [And the good news is that there are tools nowadays to automate parts of the translation between UML and code, so you don’t have to ‘go and update hundreds of UML documents every time a minor change was required’] None of this is really dependent upon the specific project management methodology being used. The real problem, as far as IT recruitment is concerned - at least in this region anyway - is that there is often a disconnect between what software development shops need and what their recruitment agents end up publishing. Given that IT recruitment agents are often more experienced in HR and marketing, they end up developing formulas for easily pushing job ads out there seemingly without much consideration for what is really required. That is how you end up with a litany of job ads that look like the same stuff rehashed over and over again. Thanks to all this, graduates who do have such skills end up in jobs which are advertised as requiring their skillset, yet never use them in any useful capacity.

Getting back to the main article, however - the point that DeMarco is trying to make here is rather subtle, one which can best be understood in the very context of his analogy. Developing software, in general, applies to a vast field of human knowledge and applications. As such, decisions about how software should be developed necessarily need to vary based upon the context and constraints within which such development is occurring. By way of analogy, humans, as they grow into adults, cannot be robotically programmed to gracefully handle every possible situation they might encounter. (Or, at least, one could imagine that this approach could fail rather disastrously if tried) Given these limitations, it is far better to learn about core principles and knowing how and when to apply them to produce generally favourable results. DeMarco’s point is that, as this applies to humans, that may be the real way forward for software development.

p.s. Some interesting and relevant articles relating to this topic:
- “Is Software Engineering Engineering?” - Peter J. Denning, Richard D. Riehle
- “Software Engineering ≠ Computer Science” - Chuck Connell

This entry was posted on Tuesday, July 21st, 2009 at 01:22 +0800 and is filed under Software. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.