Usability Testing — it’s not rocket science

Lean Usability

For startups, the idea of conducting usability research and doing actual tests can seem very intimidating and far-reaching. Many would tend to argue that there is soo much other things to do — like actually building the product in the first place. This is true; however, the key thing is to know what & why your building your product. By incorporating gorilla or small usability tests, you’ll be able to validate 5 key quality components for your product:

  • Learnability: How easy is it for users to accomplish basic tasks for the first time they encounter the design?
  • Efficiency: Once users have learned the design, how quickly can they perform the tasks?
  • Memorability: When users return to the design after a period of not using it, how easily can they reestablish proficiency?
  • Errors: How many errors do users make, how severe are these errors, and how easily can they recover from the errors?
  • Satisfaction: How pleasant is it to use?

Having this information will only help you hit your target as you go through the cycles of ‘Build, Measure and Learn‘ for each of your product iterations. Regardless of how cool your product looks, if it’s too difficult to use and fails to allow the user to accomplish their function they’ll leave. Your design team should be making those excellent decisions about both design and implementation based on proper data and not hunches about how your users are using your product.

The good news is that sure there are ‘best practices’, but you don’t and shouldn’t get too caught up in the possibility of doing it wrong. It’s really difficult to do a usability test wrong if you have clear parameters.

What is usability testing?

It generally involves having a series of tasks for people representing your users to complete by putting the in front of your product and observing what they do. In a nutshell that’s all it is.

Your goals should be centered around observing problems and listening to all the questions that your participants are asking.

Why should startups care?

The point of user research is so that you can later make confident decisions about your flow and designs. Jared Spool, who founded User Interface Engineering and all around smart guy, is known for saying that when his company has researched causes of many failed designs, he found out that the lack of proper research and testing data was at the root of all the bad decisions. This doesn’t mean that years of experience means nothing as a UX designer, it just puts the power back into the gathered data and your interpretation through iterations — not design hunches.

Why usability testing as opposed to using other methods? I contend that 80% of the value of testing comes from the magic of observing and listening as people use a design. The things you see and the things you hear are often surprising, illuminating, and unpredictable. This unpredictability is tough to capture in any other way. –

Six steps for a usability test

The process for conducting these tests don’t have to be held in any type of fancy state of the art room, with two-way mirrors, cameras everywhere, or etc. Testing something is better than testing nothing, meaning that if you have access to a conference room, that would work. If not, than any place that’s quiet and free of distractions (this depends on your product, but at first it’s best for isolation — ask me why). Once you have the location set, below is a great list to use as a high-level outline view:

  • Plan the test
  • Find and select participants
  • Write tasks and prepare materials
  • Facilitate tests
  • Analyze data & observations
  • Communicate results

In a later article I will break down into more detail and walk through each of the six steps mentioned above. Until then feel free to view the presentation below and / or ask me a question on twitter.

[ Download deck here ]

Every situation is unique, but are you conducting usability tests at your company? If not, why is that?

2 thoughts on “Usability Testing — it’s not rocket science

  • I’ve conducted hundreds of usability sessions and tons of up-front user research. Your article and presentation slides sum up things wonderfully.

    I’d only a few things:

    1) Try to find out the context that your product will exist in your target user’s life. Why would they use your app verses your website? Does something they already use already solve the user problem? How would they find your product if they had to find a solution to their problem?

    2. What USER problem are you trying to solve. It’s great if a user can easily get through your app or website. But would they really use it in real life? What problem do the participants in your usability session think your product is going to solve for them? Many times, I’ve found that the problem the user states the product is going to solve doesn’t totally match what the business thinks they are solving. That disconnect can make a product fail.

    3. OBSERVE. I can’t stress enough that what users say and what they do are two different things. You want to create tasks that allow you to observe behavior and not have to lead the user through screens. Just because a user loves blue or hates the way a section is layed out, doesn’t mean anything. It’s opinion. You want to observe behavior.

    4) Recruit the right users. You can recruit anybody for your tests and it definitely better than nothing. But ideally, you want to recruit users who you think are likely to use your product. Again, your product could be easy to use, but that doesn’t mean it meets the needs of your actual users.

    5) Gather stories. Conducting an up-front interview with the user before giving them tasks will help you understand how their lives work. Little things like how many kids they have or the kind of work they do, what their hobbies are, paint a vivid picture of your users. These pictures are very powerful when you are discussing improvements and new features or products.

    5) Repeat often. There are so many ways to talk to users now. Show them concepts, observe their behavior and ask follow up questions to clarify their behavior. If something is repeated often by different users, *THATS* something that you know is a true issue. And conduct usability often. Do it remotely. Do it in person. Do it in the corner cafe. User online usability tools. Use screen sharing.

    The article and presentation didn’t cover how to find the right users for your study or how to compensate them for their time, how to know an issue is an issue or how to write good tasks. There’s lots to know when researching users, but I totally believe that anybody can conduct a good study. It’s not that hard once you know how to do it and are clear on what you want to get out of your time with users.

    If anyone has any questions, feel free to reach out to me on twitter @eddiejams. I love to discuss diy usability.

    Again, great post and presentation!

    eddie @eddiejams

Leave a Reply

Your email address will not be published. Required fields are marked *