What do we mean when we say software quality? How can we think effectively about it in a practical way? Quality is an elusive topic, and so is product quality for that matter. I have caught myself saying things like “we are not compromising on quality” and when pressed to describe what quality I meant, what was exactly what we were not to compromise on, I found myself in real difficulties.
I am not writing in this blog post about the true nature of quality or other philosophical disquisitions, but focusing on a practical approach that enables a team to take decisions and move forward in their product development journey.
A typical scenario of such an exploration can be receiving feedback describing our product not behaving the way customers expect it to do. Is the customer describing a defect in the system or a new functional request? Depending on the context of our product development efforts the question may have practical implications: What is the speed at which we should react? Should our customer support team work on this? Does the solution gets to be released in the next version of the product or as a hot-fix to the previous version?
A traditional way to answer that question has been to refer to the specification of the product: if the behavior described by the customer was not complying with our specification it is a defect, otherwise it is just an additional requirement. That is a useful distinction sometimes as it helps us discern whether our design team have foreseen a particular aspect of the solution or not, but maybe not very helpful when it comes to understand if we are serving our customers well. In particular, there are many cases in which the customer is trying to do something which is reasonable and useful and even the obvious answer but that we did not imagine when writing the specification. Some times our specs are completely right in reflecting our wrong assumptions about what the customers want.
For a product development organization there is a question line which in my opinion is more useful: Is the product we have released to the market making it possible for our users and customers to achieve what they expect to be able to do with it? Are we satisfying our customers? That can help us identify quality but it is difficult to answer those questions. Customer feedback is always telling us something important, but sometimes we need help understanding what it means precisely.
Fit for purpose is a thinking razor helping us move in the right direction. These are some of the question lines I use to facilitate the team inquiry about the products we create.
- Is the usage the customer is describing covered by our goals when we released the product?
- Is the customer in a demographic that we are not focusing on?
- If the problematic case was not to be covered by our current version, does the product still achieve something meaningful for its intended users?
- Does the product resolve a problem for the user beginning to end?
- Can we consider the missing functionality as an add-on or is it part of what the current release of the product is aimed at solving?
- Does it sound like something is broken or forgotten? Are we comfortable stating that the requested function is something our product does not do yet?
Let me put an example (I know, I know, I should have included one earlier…): Melomaniac Bit has just released their second version of RareRecordsHoarder (RRH), a great piece of software that allows record collectors to catalog any vinyl record, using the data collected by the company about almost any vinyl pressing out there. One of the most applauded features in version 1 was the capability to scan the barcode to add all modern vinyl records. For older editions the user can type a few details and the matches in the database will be offered as options, when everything else fails all details can be added manually and the user will be asked her permission to contribute those details to the shared database. After all the hard work the reward is superb you can brag in facebook about your collection and share as much or as little as you prefer. I know… record porn sharing. Version 2 introduced a great new feature the team would expect will double their customer base, using the user self-appraised state of each owned vinyl pressing and a database of vinyl sales the program calculated the estimated value of a user’s collection.
Immediately after releasing RRH v2 the team received feedback from Sheila, an upset customer. He had just bought RRH v2 and it still does not allow him to get information from the database for his shellac 78 RPM records, what kind of an upgrade was that?
Tanner comments he owns two copies of the same rare 50’s single, both are in mint condition one of the pressings has an (accidental) tan coloration and the other is just plain black. Unfortunately, both copies are not valued the same, the rare colored one sells for 50 times the price of the black one.
Lets analyze the two situations: Sheila is trying to catalog her 78s, and even though shellac records are an expansion area for the future the current target is vinyl only, which according to the team’s research accounts for 85% of the record collectors market. The team decides to explain that to Sheila reinforcing how much they value her business. Tanner is really trying to achieve what the team attempted to release, they even think without an accurate estimate of the value of the collection there is no market for v2. The team never thought about that, those are unique cases… adding a new field will not make it as each unique record is so for a unique reason… Panic! The team knows their software is not fit for purpose, it is a defective product. Suddenly Marie, kind of a quiet genius, says “what if, when there is such a disparity in prices we simply ask the user which one is theirs? They are self-appraising their collection after all. That is easy to implement and it can be online in two days. Data is ready for it too.” Saved. Email to Tanner, solution in two days. (I know, I could not resist the simplistic happy ending…
What about your team? How do you look at the quality of your product? Do you find fit for purpose a useful concept? How do you explore it?