Usability testing

A presentation at Front-end London in April 2019 in London, UK by Calum Ryan

Usability testing — Usability testing
Effective user research to make the web more usable for more people

Why usability testing?

This isn't a talk about tools, technologies or frameworks

Who am I — Who am I
Calum Ryan
Front-end Developer at Helpful Digital
Twitter calum_ryan
calumryan.com

My satisfaction — My satisfaction
Satisfaction comes from… Knowing what we create satisfies and solves problems for users

Making too many assumptions — Making too many assumptions
Our users have the latest devices/software
Our users don’t have any disabilities
Our users should already know how to do that
Our product isn’t that widely used to matter
Previous products/approach worked fine

We build features people don’t need or hardly ever use Outcomes of not doing usability testing
We may fail to consider different use cases, challenges/constraints
Potentially break the law, get sued, bad press and reputation
We turn people away to alternatives/the competition

Usability testing / research

We’re testing the website/product. Not the user

Quote from Just Enough Research — Quote from Just Enough Research
“The goal is to determine to what extent the product or service as designed is usable”
Erika Hall Just Enough Research | A Book Apart

Measuring usability

5 components of usability - Nielsen Norman Group — 5 components of usability - Nielsen Norman Group
Learnability
Efficiency
Memorability
Errors
Satisfaction
5 components of usability - Nielsen Norman Group

Learnability — Learnability
Do I need to consult documentation to use your website?

Efficiency — Efficiency
I have a 100-odd other tasks to do today. Will this take long?

Memorability — Memorability
This website isn’t how I remember it. How do I do that again?

Errors — Errors
Users struggle to find X resource. How can we make X easier to locate?

Satisfaction — Satisfaction
Is the website accessible and somewhere you’d happily visit again?

Methods of user testing — Methods of user testing
Explorative
Comparative
Thinking aloud
Remote
Hallway
Assessment
Review-based
Consider one or more different approaches to user testing

Explorative — Explorative
Ideally performed by experienced testers
Evaluate new or initial design ideas
Iterative tests, defines future tests
Test sessions are defined by goals

Comparative — Comparative
Two or more design variations
Rate positives/negatives in user experience
Typically A/B testing
Combine best features

Thinking aloud — Thinking aloud
Typically a series of pre-defined tasks
Participant describes what they’re doing
Gather feedback as they progress (or struggle) to complete the task

Remote — Remote
Synchronous testing Real-time user testing via video link or remote sharing tools
Asynchronous testing Automated/prescribed tests the user may choose to do in their own time and in their own environment

usertesting.com

Hallway — Hallway
Testing done in high footfall public places
Voluntary participation random individuals
Quick, low-cost way to get feedback

Assessment — Assessment
Testing of a prototype ahead of production
Real-time monitoring often from another room
Progress and reactions to series of tests

Review-based — Review-based
Typically tested by experienced testers
Pre-defined tests, often using automated tools
Deep interrogation tests pre-production

Rating the problems — Rating the problems
Rate each problem users encountered during the tests with a common scoring system
Severity
Fails to complete task (High)
Completes with difficulty (Moderate)
Minor problem (Low)
Frequency
30% of participants (High)
11-29% (Moderate)
10% (Low)

Participants

Find the right participants and treat them well

Aim for 5-10 participants and usability tests

Avoid using your own team as participants

Testing for accessibility — Testing for accessibility
Get representative users
Choose participants with a range of different disabilities and combinations of disabilities
Allow users to test with their own aids such as screen readers, devices or other assistive tech
If required to attend in-person, check the venue and transport to there is accessible

Testing environment — Testing environment
Go to where the people are
Have separate rooms for the participants and observers with good quality mics and network connections
Choose a comfortable space: a relaxed, home-like environment away from distractions and lab-like conditions
Encourage honest feedback

What to ask — What to ask
Get to know your participants and build a rapport
Profile them and their web usage habits
Have scenarios to work through
How would you do…?
What do you make of this?
What would you do here?

Usability Demo — Usability Demo
View video demo from Rocket Surgery Made Easy by Steve Krug

Handling feedback — Handling feedback
Choose a good facilitator
Friendly but neutral
Encourage honest feedback
Acknowledge problems/failures positively
Be confidential and protect user data

Guerrilla User Research — Guerrilla User Research
“help the participant visualise the future product clearly enough to give useful feedback on whether they can achieve their goals” Jamie Levy Chapter 8 | UX Strategy | O’Reilly

Guerrilla User Research — Guerrilla User Research
Validated research conducted in short space of time on a small budget
Small, structured experiments with a prototype and 5-10 participants
Typically done at a coffee shop with good WiFi, power sockets, not too busy and acceptable to stay for the day
1 participant, 1 interviewer, 1 note taker

Final thoughts

What usability testing isn’t going to do — What usability testing isn’t going to do
Provide a breakthrough design
Guarantee a successful product in the marketplace
Prepare you for every single scenario or use case
Substitute for QA testing (but can help define QA tests)

Some typical feedback from usability testing — Some typical feedback from usability testing
Difficult to use and overwhelming forms
Links and resources aren’t highlighted
Challenging/hidden navigation menus
Content isn’t presented or tailored for different audiences

Be open about your usability testing. Share your findings

“approach usability testing with the right expectations and conduct it early and often”
Erika Hall Just Enough Research | A Book Apart

Just Enough Research by Erika Hall

UX Strategy by Jaime Levy

thank you

Calum Ryan
@calumryan

1 / 46

Testing usability of a digital product takes on a wide variety of test types and feedback processes that I will elaborate on in this talk. Designers, developers, product owners and anyone with a vested interested should all partake in this process, from defining the tests to measuring the success and failures of different usability testing.

Video

Resources

The following resources were mentioned during the presentation or are useful additional information.

Alice Bartlett: Burn your select tags - EpicFEL 2014

Inspiration by this talk helped plan my own presention on this subject. Alice Bartlett takes a look at specific challenges faced when developing and testing interface for many different users.

Buzz and feedback

Here’s what was said about this presentation on social media.

You looked extremely professional from where I was sitting :)
Really enjoyed your talk: great topic, solid info, beautifully presented. Thanks for sharing your expertise 👍
— Oliver (@oliverturner) April 25, 2019
You were great man! Keep at it 🤘🏼
— Luke Jackson (@lukejacksonn) April 25, 2019