Usability testing

Effective user research to make the web more usable for more people

Why usability testing?

This isn't a talk about tools, technologies or frameworks

Who am I

Calum Ryan

Front-end Developer at Helpful Digital
Twitter calum_ryan
calumryan.com

My satisfaction

Satisfaction comes from… Knowing what we create satisfies and solves problems for users

Making too many assumptions

Our users have the latest devices/software
Our users don’t have any disabilities
Our users should already know how to do that
Our product isn’t that widely used to matter
Previous products/approach worked fine

We build features people don’t need or hardly ever use Outcomes of not doing usability testing
We may fail to consider different use cases, challenges/constraints
Potentially break the law, get sued, bad press and reputation
We turn people away to alternatives/the competition

Usability testing / research

We’re testing the website/product. Not the user

Quote from Just Enough Research

“The goal is to determine to what extent the product or service as designed is usable”

Erika Hall Just Enough Research | A Book Apart

Measuring usability

5 components of usability - Nielsen Norman Group

Learnability
Efficiency
Memorability
Errors
Satisfaction

5 components of usability - Nielsen Norman Group

Learnability

Do I need to consult documentation to use your website?

Efficiency

I have a 100-odd other tasks to do today. Will this take long?

Memorability

This website isn’t how I remember it. How do I do that again?

Errors

Users struggle to find X resource. How can we make X easier to locate?

Satisfaction

Is the website accessible and somewhere you’d happily visit again?

Methods of user testing

Explorative
Comparative
Thinking aloud
Remote
Hallway
Assessment
Review-based

Consider one or more different approaches to user testing

Explorative

Ideally performed by experienced testers
Evaluate new or initial design ideas
Iterative tests, defines future tests
Test sessions are defined by goals

Comparative

Two or more design variations
Rate positives/negatives in user experience
Typically A/B testing
Combine best features

Thinking aloud

Typically a series of pre-defined tasks
Participant describes what they’re doing
Gather feedback as they progress (or struggle) to complete the task

Remote

Synchronous testing Real-time user testing via video link or remote sharing tools
Asynchronous testing Automated/prescribed tests the user may choose to do in their own time and in their own environment

usertesting.com

Hallway

Testing done in high footfall public places
Voluntary participation random individuals
Quick, low-cost way to get feedback

Assessment

Testing of a prototype ahead of production
Real-time monitoring often from another room
Progress and reactions to series of tests

Review-based

Typically tested by experienced testers
Pre-defined tests, often using automated tools
Deep interrogation tests pre-production

Rating the problems

Rate each problem users encountered during the tests with a common scoring system

Severity

Fails to complete task (High)
Completes with difficulty (Moderate)
Minor problem (Low)

Frequency

30% of participants (High)
11-29% (Moderate)
10% (Low)

Participants

Find the right participants and treat them well

Aim for 5-10 participants and usability tests

Avoid using your own team as participants

Testing for accessibility

Get representative users
Choose participants with a range of different disabilities and combinations of disabilities
Allow users to test with their own aids such as screen readers, devices or other assistive tech
If required to attend in-person, check the venue and transport to there is accessible

Testing environment

Go to where the people are
Have separate rooms for the participants and observers with good quality mics and network connections
Choose a comfortable space: a relaxed, home-like environment away from distractions and lab-like conditions
Encourage honest feedback

What to ask

Get to know your participants and build a rapport
Profile them and their web usage habits
Have scenarios to work through
How would you do…?
What do you make of this?
What would you do here?

Usability Demo

View video demo from Rocket Surgery Made Easy by Steve Krug

Handling feedback

Choose a good facilitator
Friendly but neutral
Encourage honest feedback
Acknowledge problems/failures positively
Be confidential and protect user data

Guerrilla User Research

“help the participant visualise the future product clearly enough to give useful feedback on whether they can achieve their goals” Jamie Levy Chapter 8 | UX Strategy | O’Reilly

Guerrilla User Research

Validated research conducted in short space of time on a small budget
Small, structured experiments with a prototype and 5-10 participants
Typically done at a coffee shop with good WiFi, power sockets, not too busy and acceptable to stay for the day
1 participant, 1 interviewer, 1 note taker

Final thoughts

What usability testing isn’t going to do

Provide a breakthrough design
Guarantee a successful product in the marketplace
Prepare you for every single scenario or use case
Substitute for QA testing (but can help define QA tests)

Some typical feedback from usability testing

Difficult to use and overwhelming forms
Links and resources aren’t highlighted
Challenging/hidden navigation menus
Content isn’t presented or tailored for different audiences

Be open about your usability testing. Share your findings

“approach usability testing with the right expectations and conduct it early and often”

Erika Hall Just Enough Research | A Book Apart

Just Enough Research by Erika Hall