the island of eleusis

Being a software developer, I often spend considerable amounts of time reflecting on the way I do create software. Thinking about the processes involved, how things could be improved, figuring out who to blame when things don’t go very smoothly… I’m also a visual thinker - things make more sense to me when there are diagrams involved. So it causes me a great deal of consternation when simply given a chunk of code and told “go make it better!”. OK, consternation might be a bit strong there. On the one hand, it’s rather exciting to dive into some unknown chunk of code and figuring out how it all works and comes together. It’s a bit like an adventure game, exploring some strange and dark land. On the one hand, it is very logical, but on the other hand, it can sometimes look like whoever wrote the code was not actually thinking very rationally about the problems that the software was meant to be solving.

That logical disconnection tends to be mitigated if you can get the author of the code to actually explain his approach to doing whatever he was meant to be doing. Even then, you can think to yourself, hey, I could have solved this more easily by doing it this way. That is, you could have solved the problem using patterns familiar to yourself - patterns that you have used multiple times and proven to yourself that they work quite well. But, I digress slightly.. I often think that.. no, I almost always think that documentation is essential to any software project. I say ‘almost’, because there are instances where you can get away with little to no documentation, for example, when writing a script to ri^Wback up DVDs that no one else is going to use, besides yourself.

In fact, you might find that most people involved in the IT industry - at least those who come from academic backgrounds, such as those who have studied Computer Science at a university - *know* that documentation is an essential aspect of anything that is done in IT. System and network administrators know they have to come up with written policy documents about how procedures are to be implemented. Programmers know they have to ‘document’ whatever they’re working on. But too often, ‘documentation’ is left as a rather vague term. Documentation, when it comes to software engineering and programming, often covers things like manual pages, README files, web/wiki pages, e-mails, and countless other items that can purport to describe what, how or why a system does what it does, all the way down to comments within source code files. Getting this mess of knowledge organised in some form can take a fair amount of work, which is why most people end up skipping it and simply carry the most useful information within their heads.

We all know of tales of disgruntled programmers who write strange and convoluted code, and who leave a serious mess behind when they leave, with other hapless souls having to make heads or tails of whatever they’ve written. Yet, we also acknowledge that it’s not hard to simply ask someone when we don’t understand what a piece of code is meant to be doing. Unfortunately, while this simple approach does work, it does not scale to large projects and large teams, as software engineering researchers have been warning since the 60’s and 70’s. Adding more and more people to a project exponentially increases the number of possible communication links between team members, and the time required for each member to ’service’ those links also increases. This all means that people end up having to talk to each other for increasing amounts of time, just to understand what they’re meant to be doing, and they end up spending less time actually working on whatever they’re meant to be fixing or implementing. It’s not a hard algorithm to understand.

So the way we try to compensate for this lack of real-time communications scaling is to say documentation, documentation, documentation. Unfortunately, as I mentioned above, this means different things to different people. Fewer people actually seem to want to put some thought into coming up with ways that ‘documentation’ can be more strictly defined in certain contexts, and managed so as to maintain the usefulness of the knowledge being generated.

I often like to think of documentation as providing a map of how a system should work. Due to particular software engineering courses I did at uni, I also happen to think that a large part of this map should be composed of UML diagrams. The process of designing software seems rather straightforward when you decide to build your process around UML data. (hand-wavy bit) You start off with a rough set of specifications (describing the problem), and you begin building a mental model of what a possible solution could be. During the process of building this model, you draft some diagrams to try and flesh out some major components of the solution. While this is happening, everyone involved in the project (or in SE-speak, the ‘major stakeholders’)  gets to see the model evolve and provide some feedback. This whole process of drafting and feeding back is rather crucial, in my opinion, in giving everyone involved an appropriate big picture view of the solution. Without the feedback process, we end up with one model on paper and multiple, slightly varying versions of the model in people’s heads. This simply happens naturally, as different people make different assumptions about what they see, based on their own previous experience, whatever information they get given - and maybe more importantly, whatever information they *don’t* get given, or miss out on - and even in what order they receive information about the system.

It makes intuitive sense to me that UML is named as it is because the process of designing software is largely about designing a model of a solution, and doing so in a manner that allows multiple participants to be explicitly informed about various aspects of the system. As such, UML provides a common ‘map’ to navigating the system being built. It’s true that various UML diagrams give you various views of a system, up to different depths, and this is exactly what I think the beauty of UML is. An almost perfect analogy for this, to my mind, is Google Maps. With Google Maps, you can start off with a global view of the world, seeing continents and oceans. You can think of this as being a UML deployment diagram. From there, you can zoom in onto various countries and see more details, such as rivers, cities, beaches, and so on - zooming in onto state diagrams, class diagrams and sequence diagrams. Ultimately, you get to the street level and see details such as cars, bicycles, people, cats… You’d think that the analogy to code documentation breaks here, but it doesn’t.

The ultimate close-up ‘documentation’ or ’specification’ of a system is the code itself, as has been famously been said by someone whose name I can’t remember anymore. This is exactly where comment blocks and inline comments come into use. One interesting way of thinking about comments in code is to think of them as ’speaking to your future self’, (hey Richard!) which does make a lot of sense. Going back to my DVD backup script from earlier, it would certainly help to put in some comments to reveal some of the more obscure low-level commands that get used by the script (hello mplayer switches). And that is fair enough. But simply commenting code by itself is not sufficient ‘documentation’. Dropping breadcrumbs so you might find your way again later may be useful, but it’s only so useful as long as you find the breadcrumb trail again, and follow it in the right order.

The analogy to gaming here would be like going into a single player FPS game without a map of the level that you’re on. Sure, it’s a lot of fun to stumble around in the dark while armed to the teeth with futuristic weapons, just enough to blast the jaws off any nasty zombies/bugs that might pop around a corner, while having no idea where you’re going to end up. This even applies to my earlier example of being given an unknown chunk of code to explore - it’s a lot of fun to stumble around the code to explore its many intricate paths, but the fun is quickly diminished when put under time pressure. Ask yourself: would you enjoy Half Life 2 or Doom 3 if you absolutely had to race through the game and finish it within a few hours? That is, doing it on the first go, while dying as few times as possible, and without pausing to enjoy the billion-polygon rendering power of your shiny new graphics card? Where would the fun of exploring go? Where would the fun of jumping in fright while reflexively pumping half a dozen shotgun shells into some twisted-face zombie go?

You might even like to race through adventure games while having no explicit map telling you where you are - simply following signs you find around you in the environment. However, even when going from the single player FPS experience to an online multi-player FPS experience, maps become necessary. Without maps in online multiplayer games such as Enemy Territory, or Battlefield simply become impractical. The presence of maps in such games allows teams of players to co-ordinate their actions much more precisely and more efficiently than saying “let’s meet at the palm tree next to the fortress entrance”. Having the map available and updated in real time allows for ’swarming’ actions where players spontaneously congregate towards strategic points of the map in order to force a favourable outcome for their team. (Or in the case of a game like Battlefield 2, commanders and squad leaders set specific goals for their teams to achieve, by using the interactive maps) This sort of behaviour simply isn’t possible if all you have, as a player in an FPS environment, is the limited field of view around you, obstructed by walls, buildings, rocks, vegetation, terrain, etc.

This is exactly how a programmer feels, when left to hobble along with only inline comments and comment blocks guiding his way. There is no way for him to ‘zoom out’ to a slightly higher level. He might eventually achieve what he set out to achieve - fixing a bug somewhere in the code, or implementing a new feature, but he will have a limited (if any) idea of what his fellow co-developer is doing in another part of the same codebase. Switching to a high level view of the system doesn’t help him either in that situation, especially if 1) there are no intermediate levels of documentation between code + comments and a diagram simply showing all the classes to be present in the final product, and maybe more importantly, if 2) whatever intermediate or higher level documentation is not updated to reflect lower level changes being made. The very high level documentation doesn’t help because.. well, imagine trying to navigate your way by car from one end of a city to another, while having Google Maps zoomed out all the way to show you all of Australia..

One has to wonder.. where is the Google Maps of software engineering?

This entry was posted on Sunday, July 15th, 2007 at 23:34 +0800 and is filed under Games, Software. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.