The document dilemma

The Agile manifesto states “Working software over comprehensive documentation” – this seems to be one of the biggest mindset shifts that organisations need to make when adopting any Agile framework. For some people that simple statement brings up visions of chaos, lack of control, and the worst fear of developers doing what they want. At the other end of the spectrum that’s exactly what some people hope it does mean.

If you ask people within most software development teams / companies what they do it is very unlikely that you would get the response “write documents about the products for our clients”. Most responses would be about the latest website or the latest Google hack they are doing. So inherently at an external client facing level we do value working software over comprehensive documentation. Where I believe this value changes is during the production of the working software and the legal requirements from the iron grip of the contract.

Any organisation wanting to adopt an Agile framework, needs to cover all of the bases when it comes to making a change at this level. You can just adopt Scrum for example and expect everything to be fine, it’s not just getting the developers to plan their own work and using a release plan instead of a gantt chart.

I don’t want to talk in detail about the other areas of work on Agile adoption as I specifically want to talk about documentation in this post.

So lets assume that all the other changes are happening and that for the most part it’s all going well. The first thing to remember is not to try and fight your battles on too many fronts at the same time and secondly to understand the nature and purpose of the documentation practices you want to change.

“Documents what are they good for, huh absolutely nothing”

When you want to shift your team or even organisation’s view of documentation, you first need to understand the purpose and audience of each documentation step. Essentially documents are tools, tools that are supposed to help us communicate effectively. They were not written to be a pain or to hide things or to make the author look clever (we hope). Take each step in your product / project delivery cycle in turn and work out the audience and the purpose of each document.

For example the requirements specification as you currently know it is effectively redundant once you adopt an Agile framework, however the audience’s needs for the communication medium around project boundaries, depth and context has not. You can’t just say we now do user stories so don’t need requirement, the audience has not gone through any form of change and in most cases will react very negatively (even if they really know it’s the right thing to do) to your well meant suggestion. You may need to train certain people how to write and create user stories, others you will have to educate on the product backlog and how that acts as a scoping document. Your quoting/sizing generation processes will need some overhauling too. How about your existing clients? Your relationships with them will help, but how about your sales pitch for new work? Stories are for children right? Why would we want to deal with you? Again it comes down to knowing why your audience needs that communication tool and how you can sell on a better process.

Even better is to replace documentation with a far more powerful tool: verbal communication. Can the daily Stand Up with the on site customer combined with some other project metrics (burn up/down, bugs etc) replace the (groan) weekly project status report? Does the product demo cover the need for client milestone sign off documents? Can the retrospective outputs replace the post project review document?

“If you chase two rabbits you won’t catch one”

This old hunting adage still applies to delivering a project at the same time as changing your processes. Delivering new software is not easy at the best of times. It requires skill, focus and clear direction. If you dilute this with a business process change then you dramatically increase the chances of project failure.

Try and split the changes into two parts: documents that your company needs to succeed and documents that your project / product needs to be a success. For the project specific documents the best place to get them right is on the project, and the best time to do this is at the start. If you use the concept of Sprint Zero then this is the ideal time to get a working outline of documentation levels and processes, test and agree them with team members and where possible the client.

Use the power of “Done” (a good “done definition” that describes when a feature is complete) to ensure that the necessary but boring bits get completed for each story. Use stories to produce the larger documents, especially the ones you still disagree on the value of. Make the product owner decided on the relative priority between a new product feature or a document. This will really help to ensure that the most valuable documents get written.

Solving the document dilemma is as essential to successful Agile adoption as having a release process that works or getting your teams to self organise. It takes time to get right is often overlooked until too late and can drag an otherwise successful Agile project to a standstill.