Why is Software Documentation So Hard?

It is perhaps a truism that software developers tend to be binary thinkers. The bit is either on or it’s off. My software compiles successfully or it doesn’t. My program ran successfully or it crashed. The user/customer is happy with my changes or they’re not.

(This simple truth is key to many of the developer practices recommended by the agile community, by the way. Many of these practices reduce more complex activities to simpler, pass/fail propositions: the change I just checked in broke the build or it didn’t; the new code successfully passes all the automated regression tests or it doesn’t; the other member of my programming pair approves of the code I just wrote, or doesn’t; and so on.)

When we produce documentation rather than software, however, none of these same binary indicators of success are present. Instead, measures of success tend to be more subjective, and involve matters of judgment and degree, rather than simpler pass/fail indicators. Did I produce enough deliverables? Did those documents contain sufficient information? Was the content easy to understand? Will the information prove helpful? All of these are hard questions that require judgment calls.

The result is that production of documentation on software projects often swings from one extreme to another. For a developer to produce no documentation is easy. For a developer to fill out a Word template for every single deliverable called out by a process is also easy, in the sense that it doesn’t require much judgment.

To produce just enough documentation, though… that is hard!

This is one reason why CMMI-based process improvement efforts so often go astray. Organizations at CMMI Level 1 often produce little or no documentation. As they work their way towards Levels 2 and 3, the many subtleties of the CMMI often get misinterpreted as a few simple messages, such as “more documentation,” and “show me evidence that you are following the process.”

In many ways, the Agile movement was born as a backlash against exactly this sort of excess. This is one reason why Agilistas seem to develop such a fixation about the use of index cards. It is not that such cards are conducive to better writing, or easier to use and reference, but rather that both their size and lack of permanence provide very obvious barriers to this sort of misuse. Give a developer a word processor, turn your back for a few weeks, and he or she may well produce the sort of Victorian novel style of requirements document that looks impressive, but that few if any will ever actually read. Give that same developer a stack of index cards and a Sharpie, though, and this sort of thing is much less likely to occur. (Even Charles Dickens couldn’t have written a Victorian novel on index cards.)

Like Goldilocks, then, what we want is a collection of documentation that is not too big or too small, but just right.

But how do we know when we have too little, and when we have too much?

Tune in again next week for some clues to address this difficult topic.