It is not enough that we deliver systems that are merely functional, they must also be usable. Paraphrasing from usability expert Jeff Attwood, to the users of the system, "the interface is the application"; it is their most visible window into our work. Whether we finish on time, on budget, or in scope can be immaterial if the final deliverable is frustrating or inefficient to use.
On most enterprise applications, in my experience, usability is promoted via a number of low-cost methods: following usability standards, deferring to user interface specialists, trusting our intuitions, or simply conferring with customers and business owners. And while these methods are sufficient in most cases, when usability is a high priority, there is simply no substitute for actually watching people, without interference, use the software we build. This is the practice of usability testing.
In particular, I believe that usability testing should be strongly considered on projects where...
- The user base of the system is numerous, not technically-savvy, or external to the customer.
- The cost of user error is high (e.g. software for managing financial transactions, medical procedures, etc.) and features are complex.
- The UI framework or paradigm is nascent or highly flexible (e.g. RIA, iPhone, etc.) and a consensus on usability patterns or best practices has not yet been reached.
In addition to making our products easier to use, usability testing can also be of great benefit to us, the developers of the software, by improving our usability intuitions, educating us on the functional domain and application of the software we build, and re-enforcing for us that what we create has a tangible impact on real people.
In this post I'll describe usability testing within agile projects from a project management perspective, describing how to estimate, prioritize, and "burn-down" the usability test. In a follow-on post, I'll dig into the actual activity of usability testing, breaking down some best practices within the framework of a simple 3-step process: (1) plan for the test, (2) perform the test, and (3) analyze the results. Let's get started...
Managing Usability Testing with Scrum
Effective usability testing can be easy, quick, cheap, and, as the diagram below shows, can fit seamlessly within the Scrum methodology. Here are some tips, given my experiences with usability testing on Agile projects, on how this can be achieved:
Usability Test as Backlog Item: Unlike other development best practices (like code reviews or unit testing), usability testing should, in my opinion, not be "built-in" as a part of normal development tasks, but rather should be treated as a separate backlog item - given an estimate by the development team, a priority by the business owners, and is assigned to a sprint for completion. Unlike other backlog user stories or tasks, however, usability testing has the property that it is never truly "done" - there are cases when (indeterminately) many rounds should be performed, and so the task of usability testing should never really be permanently plucked from the backlog.
Estimation: Although the amount of time it takes to perform a usability test depends on many project factors (e.g. scope or complexity of the application, availability of users, etc.), in my experience (using the methodology to be laid out in the next post) effective usability testing can usually be completed with roughly 30 man-hours of work: 15 for the preparation, 10 for the test itself, and 5 for the analysis and communication of results.
Prioritization: Oftentimes, even when usability testing should be performed (see above), business owners or other project stakeholders will still be reluctant or apprehensive to do so - especially when other methods (like just trusting intuitions) are cheaper and seem just as effective. It is important in these cases to stress that (a) usability is important, (b) we are notoriously bad judges of what is usable (too technical, too close to the system, etc.) and (c) usability testing can be cheap and extremely fruitful. Further, it is crucial to note that the earlier in the lifecycle that usability issues are identified, the easier they are to resolve - for example, redesigning a system's confusing navigation structure once it is in production can be extremely costly and risky. In the end, however, usability testing should be prioritized along side all other user stories on the backlog, and this prioritization should be driven by the product or business owner.
Usability Testing as a Burndown Task: Once usability testing has been selected for inclusion in a sprint, it should be assigned to one developer (where in agile teams, "developer" can mean tester, programmer, UI designer, etc.). Note that all activities of usability testing (planning, testing, and analysis) should fit within one sprint, and the task should be "burned-down" like any other development task. The practice of usability testing does not require an "expert". Anyone is capable of performing one, and in fact, should if given the opportunity - it can be a rewarding and broadening experience.
Incorporating Feedback: Once the usability test is performed, observations from the test will be analyzed and discussed (see next post), and eventually some consensus will be achieved regarding how the application should change to improve user experience or efficiency. This feedback can flow back through the Scrum development process in one of two forms: either as (1) new user stories added to the backlog (e.g. "as a customer, I want to be able to use wild cards in text searches") or, when improvements are simple, as (2) bugs managed by the defect tracking system (e.g. "error messages should be red, not yellow").
As always, I'd be happy to hear about your experiences with usability testing on Agile projects. How often do you usability test? How do you track it from a project management perspective? Please share!
Also, check back in a few weeks for part 2 where I'll dig in to the actual practice of usability testing in more detail.
