Usability testing

A presentation at Founders and Coders in January 2020 in London, UK by Calum Ryan

#usabilitytesting @calum_ryan Usability testing Effective user research to make the web more usable for more people

#usabilitytesting @calum_ryan ? Why usability testing?

#usabilitytesting @calum_ryan We make assumptions

#usabilitytesting @calum_ryan ‣ Our users have the latest devices/software ‣ Our users don’t have any disabilities ‣ Our users should already know how to do that Making too many assumptions ‣ Our product isn’t that widely used to matter ‣ Previous products/approach worked fine

#usabilitytesting @calum_ryan ‣ We build features people don’t need or hardly ever use Outcomes of not doing usability testing ‣ We may fail to consider different use cases, challenges/constraints ‣ Potentially break the law, get sued, bad press and reputation ‣ We turn people away to alternatives/the competition

#usabilitytesting @calum_ryan Usability testing / research

#usabilitytesting @calum_ryan We’re testing the website/product. Not the user

#usabilitytesting @calum_ryan “The goal is to determine to what extent the product or service as designed is usable” Erika Hall Just Enough Research | A Book Apart

#usabilitytesting @calum_ryan Measuring usability

#usabilitytesting @calum_ryan ‣ Learnability 5 components of usability ‣ Efficiency Nielsen Norman Group ‣ Memorability nngroup.com/articles/usability-101-introduction-to-usability ‣ Errors ‣ Satisfaction

#usabilitytesting @calum_ryan Learnability Do I need to consult documentation to use your website?

#usabilitytesting @calum_ryan Efficiency I have a 100-odd other tasks to do today. Will this take long?

#usabilitytesting @calum_ryan Memorability This website isn’t how I remember it. How do I do that again?

#usabilitytesting @calum_ryan Errors Users struggle to find X resource. How can we make X easier to locate?

#usabilitytesting @calum_ryan Satisfaction Is the website accessible and somewhere you’d happily visit again?

#usabilitytesting @calum_ryan ‣ Explorative ‣ Comparative Methods of user testing ‣ Thinking aloud Consider one or more different approaches to user testing ‣ Remote ‣ Hallway ‣ Assessment ‣ Review-based

#usabilitytesting @calum_ryan Explorative ‣ Ideally performed by experienced testers ‣ Evaluate new or initial design ideas ‣ Iterative tests, defines future tests ‣ Test sessions are defined by goals

#usabilitytesting @calum_ryan Comparative ‣ Two or more design variations ‣ Rate positives/negatives in user experience ‣ Typically A/B testing ‣ Combine best features

#usabilitytesting @calum_ryan Hallway ‣ Testing done in high footfall public places ‣ Voluntary participation random individuals ‣ Quick, low-cost way to get feedback

#usabilitytesting @calum_ryan Thinking aloud ‣ Typically a series of pre-defined tasks ‣ Participant describes what they’re doing ‣ Gather feedback as they progress (or struggle) to complete the task

#usabilitytesting @calum_ryan Review-based ‣ Typically tested by experienced testers ‣ Pre-defined tests, often using automated tools ‣ Deep interrogation tests pre-production

#usabilitytesting @calum_ryan Assessment ‣ Testing of a prototype ahead of production ‣ Real-time monitoring often from another room ‣ Progress and reactions to series of tests ‣ GoToMeeting / Google Hangouts / Skype

#usabilitytesting @calum_ryan Remote ‣ Synchronous testing Real-time user testing via video link or remote sharing tools ‣ Asynchronous testing Automated/prescribed tests the user may choose to do in their own time and in their own environment

usertesting.com

#usabilitytesting @calum_ryan Rating the problems Rate each problem users encountered during the tests with a common scoring system Fails to complete task Completes with difficulty Minor problem High Severity Moderate Low 30% of participants Frequency 11-29% 10%

#usabilitytesting @calum_ryan Participants

#usabilitytesting @calum_ryan Find the right participants and treat them well

#usabilitytesting @calum_ryan Aim for 5-10 participants and usability tests

#usabilitytesting @calum_ryan Avoid using your own team as participants

#usabilitytesting @calum_ryan Testing for accessibility ‣ Get representative users ‣ Choose participants with a range of different disabilities and combinations of disabilities ‣ Allow users to test with their own aids such as screen readers, devices or other assistive tech ‣ If required to attend in-person, check the venue and transport to there is accessible

#usabilitytesting @calum_ryan ‣ Go to where the people are Testing environment ‣ Have separate rooms for the participants and observers with good quality mics and network connections ‣ Choose a comfortable space: a relaxed, home-like environment away from distractions and lab-like conditions ‣ Encourage honest feedback

#usabilitytesting @calum_ryan ‣ Get to know your participants and build a rapport What to ask ? ‣ Profile them and their web usage habits ‣ Have scenarios to work through ‣ How would you do…? ‣ What do you make of this? ‣ What would you do here?

#usabilitytesting @calum_ryan Handling feedback ‣ Choose a good facilitator ‣ Friendly but neutral ‣ Encourage honest feedback ‣ Stop if there’s any discomfort/not learning ‣ Observe and listen to suggestions and wishes from user

#usabilitytesting @calum_ryan Fixing problems ‣ Don’t wait for a redesign to fix serious problems ‣ Make small/simple changes to keep people from having the problem again ‣ Make adjustments between users such as removing difficult tasks

#usabilitytesting @calum_ryan Final thoughts

#usabilitytesting @calum_ryan ‣ Provide a breakthrough design ‣ Guarantee a successful product in the marketplace What usability testing isn’t going to do ‣ Prepare you for every single scenario or use case ‣ Substitute for QA testing (but can help define QA tests)

#usabilitytesting @calum_ryan Be open about your usability testing. Share your findings

#usabilitytesting @calum_ryan ‣ Difficult to use and overwhelming forms ‣ Links and resources aren’t highlighted Some typical feedback from usability testing ‣ Challenging/hidden navigation menus ‣ Content isn’t presented or tailored for different audiences

#usabilitytesting @calum_ryan “help the participant visualise the future product clearly enough to give useful feedback on whether they can achieve their goals” Jaime Levy Chapter 8 | UX Strategy | O’Reilly

#usabilitytesting @calum_ryan “approach usability testing with the right expectations and conduct it early and often” Erika Hall Just Enough Research | A Book Apart

#usabilitytesting @calum_ryan Rocket Surgery Made Easy Steve Krug Just Enough Research Erika Hall UX Strategy Jaime Levy

Usability Demo https://youtu.be/QckIzHC99Xc

@calum_ryan calumryan.com thank you noti.st/calumryan

Calum Ryan
@calumryan

1 / 44

Testing usability of a digital product takes on a wide variety of test types and feedback processes that I will elaborate on in this talk. Designers, developers, product owners and anyone with a vested interested should all partake in this process, from defining the tests to measuring the success and failures of different usability testing.

Resources

The following resources were mentioned during the presentation or are useful additional information.