5 Lessons from Usability Testing: Designing for the Real World

When designing a new system (or redesigning an existing one), it’s important to keep the user's real-world context in mind. A lot of thought and effort will hopefully go into making sure the product delivers the right set of features, has the right look and feel, and abides by standard UI conventions. But designs that seem solid conceptually can still fail if they do not take into account how real users will interact with them in the real world. So we need to ask:

• Where, when, and how will users engage with the system? How does this constrain the type of interaction that is possible?


• What do users need to do physically and cognitively to use the system effectively? Is this realistic for the target users?


• How will the interaction unfold over real time and in real space? Does the flow work logistically, as well as conceptually?


• Besides the expected user, who else will participate in the interaction (directly or indirectly)? Does this change anything?

To illustrate how real-world logistics can affect the user experience, here are a few examples from some of our recent usability projects:

1. Mobile Application. We observed users downloading and installing banking software designed for use on smartphones. During installation, users were given a long confirmation number and were told to write it down, as they would need it again later in order to launch the software. Our study participants balked: One noted that if he was on his BlackBerry, that meant he was away from his desk, with no pen and paper in sight.

Lesson: On a mobile device, it's important to make sure that tasks can be performed with a minimum of additional resources.

2. In-Store Kiosk. Employees at an electronics store were asked to walk through a buy-flow scenario in which they were helping customers subscribe to internet service at an in-store kiosk. Employees' concerns focused less on the UI itself and more on how they would manage customers at the kiosk. For example, how would they minimize congestion and wait time? Which screens should the customer fill out and which screens should the employee fill out? Would they be able to print from the kiosk?

Lesson: The real-world environment of an in-store kiosk requires complex user scenarios. In spaces where sales associates and customers work together to complete a task, the design should help facilitate this interaction.

3. Web Activation. To activate a new hardware device, users had to complete a web activation that involved entering their device's 12-character identification code. Success depended on users' ability to coordinate action and attention between keyboard, computer monitor, and device. This was a real challenge for many users because they were not touch typists. Many made simple – but extremely frustrating – errors such as failing to click into the field before typing the code, missing a character, or mistaking zeros for ohs.

Lesson: Whenever tasks simultaneously burden cognitive and motor skills, user errors (and frustration) are likely. In such cases, preventing errors is important but not always possible. Helping users recover quickly and gracefully from errors – e.g., precise error messages, auto-correcting typos – can be vital for a positive user experience.

4. Online Quotes. When requesting an insurance quote online, users were asked detailed questions about their current deductibles and levels of coverage. Most people in the study said they would want to complete the quote form at home, where they would be able to look up their current policy.

Lesson: When information required to complete a task is not likely to be top-of-mind, tell users up front what will be required and/or allow them to save their work and return to finish it later. This will prevent wasted time, task abandonment, and entry of inaccurate information.

5. Time-Tracking Software. Time-tracking was a component of a larger personal information management suite we tested in the lab and in the field. While very impressed at the program’s ability to automatically associate time spent on the computer with a given project, most study participants' also spent significant time offline – in meetings, phone calls, or out of the office – and they wanted to be able to allocate this time to projects as well.

Lesson: When a program cannot automatically account for or predict all real-world behavior, allowing simple manual editing or the ability to insert events after the fact is a must-have feature.

These examples give a taste for some of the real-world issues users confront… and designers have to plan for. They also underscore the value of field testing systems in users’ natural environments, where these types of issues will naturally surface. For lab-based usability work, the goal should be to create tasks and scenarios that evoke as much of the richness of the real-life user experience as possible.

About the Author: Jen Amsterlaw is a Usability Specialist with Blink Interactive.  Jen joined Blink in 2007 with a background in experimental psychology and education. She has a Ph.D. in Psychology from the University of Michigan and over 10 years of experience conducting laboratory and field research.