Martin Fowler recently wrote something with which I found myself jarringly disagreeing. The post is about a concept called observed requirements.
Although I strongly disagree parts of his recent post, I did want to say upfront that I really like Martin Fowler’s work. Ever since I read Martin‘s book UML Distilled, I have been a fan. For something that is meant to simplify understanding, UML always appeared overly complicated to me and his book does a nice job of focusing on making it useful. Martin is also a big proponent of Agile Methods and Extreme Programming, both of which have improved software development practices by turning software development on its head.
Martin’s post starts with this quote from the book “Mastering the Requirements Process,” which I read last spring. (Note: I read the first edition of this book, which contains the quote Martin uses. I have not seen the second edition. Then again, I only have the second edition of UML Distilled. Such is the life of a grad student.) Here’s the questionable quote:
Requirements are the things that you should discover before starting to build your product. Discovering the requirements during construction, or worse, when you client starts using your product, is so expensive and so inefficient, that we will assume that no right-thinking person would do it, and will not mention it again.
Suzanne and James Robertson
When I first read this quote, I had pretty much the same initial, gut reaction that Martin had. It advocates an extreme position in a field where many different development methodologies have been successful. However, Martin’s post takes a position at the other extreme and is equally questionable.
Martin seems to think that the word “requirement” itself is a bad word, and that requirements are incompatible with agile methods. He claims that web sites developed using agile techniques violate the “waterfallish” requirements process and suggests four specific ways that such web sites can observe requirements throughout development:
- Look at what people are trying to do with the site and provide easier ways for them to do it.
- Look at where people are abandoning doing something, and look for ways to fix whatever was frustrating them.
- Build a new feature and see if people use it.
- Build an experimental feature and make it available to a subset of the user base. Not just can you see if they like it, you can also assess how much load it puts on your servers.
He goes on to say that web sites should monitor how their users actually use their site because what a user really does is much more accurate than what a user says they do. First, let’s get this out of the way. Suzanne and James Robertson dedicate a significant portion of their book (about half a chapter) to requirements elicitation through observing users! Let’s look at two more quotes from the first edition of their book.
First quote:
It is unlikely that many users can explain what they do in enough detail for the developer to completely understand the work, and thus capture all the requirements.
Second quote:
For example, one of our clients, [...], had 20 different products. [...] The way the users handled each of these products at first looked to be different. However, a common pattern emerged as we studied the structure of the work – we were looking for similarities, not differences. We observed that each product was in fact a different way of [...]. The end result was that we found a common set of requirements, and were able to make a single core implementation, and then dress it differently for each of the products.
I have edited the second quote to remove project-level details and focus on the “observed requirements” concept. Quite simply, I don’t understand how these quotes are irreconcilable with Martin’s four suggested approaches to observing requirements. In fact, they seem to be quite compatible. The jarringly disagreeable part of his post is that Martin paints requirements as a software artifact that can only be used in Waterfall development, which is completely untrue.
Second, let’s talk about requirements. Requirements can, and should, be a part of any agile method of development. They answer a critical question: “why?” In the case of an observed requirement, the answer is obvious: “Because that is, by definition, exactly what the user wants or needs!” In fact, requirements are even easier to integrate into the agile process than other software development processes. The regular meetings with customers provide a great opportunity for requirements-based techniques, but agile proponents typically eschew actually documenting any of the contextual information involved in these meetings in favor of self-documenting code and UML.
Self-documenting code has always seemed a bit silly to me. Just as an author can’t write to two audiences at once, a programmer can’t satisfy the compiler and provide complete documentation at the same time. Martin Fowler writes in UML Distilled (2nd Edition):
The fundamental reason to use the UML involves communication. I use the UML because it allows me to communicate certain concepts more clearly than the alternatives. Natural language is too imprecise and gets tangled when it comes to more complex concepts. Code is precise but too detailed. So I use the UML when I want a certain amount of precision but I don’t want to get lost in the details.
Many agile proponents like UML and in particular, use cases. Unfortunately, use cases, and other UML artifacts, typically don’t offer enough contextual information to answer “why” questions. (Though, they are excellent for answering “how” questions.) It is possible to augment use cases with contextual information. Later in UML Distilled, Martin says that developers should feel free to modify UML to meet their needs. Of course, once you have added the needed contextual information, you’re effectively just writing requirements.
Any agile proponent would find a lot of Suzanne and James Robertson’s book useful if they went into it with an open mind. Many of the techniques for discovering customer requirements could improve the efficiency of regular customer meetings in an agile development process. Conversely, many proponents of detailed requirements specs would find a lot of useful information in agile-based books like Martin Fowler’s Refactoring book and Kent Beck’s Extreme Programming Explained book.
Third, and finally, let’s talk about the current use of user interaction data as an “observed requirement.” The one thing in Martin’s post with which I completely agree is that observed requirements are extremely useful and haven’t been fully explored. As a privacy researcher, I think there are some unresolved issues for user data protection, but as a developer it is clear that this data can improve the product. The often cited example in this is Amazon.com’s book recommendation service, which I enjoy. For the moment, let’s set aside the privacy concerns because that could be a whole post in and of itself.
Martin mentioned at the end of his post that he hasn’t found much advice on leveraging customer website use for the express purpose of improving their systems. I don’t think the idea is being used as well as it could be, but it is out there. John Musa has been talking about Operational Profiles, which are effectively a set of observed requirements, for years as a way to improve software development. He’s even got his own book out there on the subject.
That’s the only work that I know that gives extensive, useful guidance on how to take user interactions with a software product and directly tie it back into the development process. I certainly don’t have all the answers here. If anyone knows another place where this concept has been studied, I would love to hear from you.