A presentation at axe-con by Rebecca Topps
How to do remote research studies for accessibility by Rebecca Topps
How can I include accessibility in my research? What are e ective methods for nding participants? Which research tools should I use? How do I ensure my research sessions are inclusive? fi ff What do I need to consider in the analysis and reporting?
But before we get into answering those questions, you might be wondering why am I only talking about remote research today? Well, right at the start of the pandemic I started a new job at Atlassian in Sydney, Australia - Atlassian create work management software tools such as Jira, Confluence and Trello. My role at Atlassian was initially to lead a moderated accessibility research study which looked at the current state of accessibility of our core products. During the pandemic, Atlassian announced Team Anywhere, and as a result we were working remotely during lockdown, and all our research became fully remote. Although this was a difficult time and transition, doing my research remotely made me question why I had not been advocating for remote research sessions sooner and in previous roles. Or, at least providing the participants with a choice depending on what works best for them.
Remote research has a number of advantages for everyone, including those who are living with a disability. Including… - A person can stay within their own environment, no need for travel - A person can use their own device and potentially assistive tech or setup - mention example story around a user who had a specific version of Dragon with settings at home, but they could not replicate that in our research lab. This immediately impacts the findings as it will take longer for the user to adjust to Dragon without their usual, everyday settings. - A person is likely to feel more comfortable at home within their own environment than being recorded in a lab. Personally a lab can make me feel anxious and I’m a researcher - so I can imagine it can be the same experience for participants. Source (Mobility Works )
In recent years I have noticed an increase in accessibility research roles and advocacy across the industry - including Google, Microsoft, Government departments, and now Atlassian. The reason more companies are prioritising accessibility within user research is because it There are a number of benefits, these include:
Whitney Quensenbery, a co-author of the book “A web for everyone” explains the importance of including accessibility to ensure our user research is fully inclusive, she says… (Say the quote)
Before you start including accessibility within your research you should try to consider the following things:
You can do dedicated (moderated) accessibility research studies if you have the budget, however, you don’t need to conduct an accessibility-only focused study to do accessibility research.
A useful method is to include 2 users who identify as living disability for every 10 users. This results in 20% of your user group being users with an accessibility need - and worldwide around 20% of people identify as living with a disability. As a result you have a panel which also provides an inclusive representation to the population. This technique is practiced by Government Design Service in the UK, where it is a requirement to include accessibility in your research. However, it can be a useful method to include accessibility in your research, even if you start with a small percentage of users. Also, I view accessibility a as a big spectrum which many of us are experience at some point of our lives - whether that is due to mental health, an accident or the ageing process. Therefore, eventually you can use this technique to slowly start to include areas such as assisted digital, mental health and so on within your research participant recruitment screener.
The most common question I am asked is how to you find, source and recruit participants for accessibility research…
Some of the methods I recommend and have used are the following: Internal Panel - a good technique to build a panel is to advertise it online, Google and Microsoft do this to find users who want to be involved in QA testing and research. Build a panel or recruit in collaboration with a vendor (this removes some of the work around privacy and GDPR). Contacting charities and organisations - find local disability orgs, in the past I have focused on digital or creative workshops. You can also look for digital literacy workshops at libraries if you are doing assisted digital research & often you will find that a lot of those users are living with a disability. Advertising on social media and/or forums -you can advertise by saying you are looking for people who are living with a disability and who use your product to be involved in some research.
Iconography: https://www.flaticon.com/free-icons/instagram-like
Recruitment considerations Plan for extra time to recruit Prepare for additional costs Check your vendor is accessible Adjust your recruitment screener
Pre-research questions Do you use any assistive technology or adaptive strategies when using our product? If yes: What assistive technologies or adaptive strategies do you use? Are there any accommodations, support or changes we can provide during the research session?
How would you prefer to receive the research questions or tasks? E.g. email, verbal, chat… Would you like to have breaks during the session? How often do you prefer a break? Are you okay with sharing your video and audio? (if possible offer the option to turn off video)
When planning to include accessibility in your research, often people ask me what user needs they should focus on for different research methods and stages of the product lifecycle…
Interviews You can research all areas of accessibility and include all user needs. Paper prototyping / Mockups Cognitive; language; communication, and auditory. Prototypes Colour; language; communication; cognitive; mobility (e.g. keyboard only) and vision (e.g. ZoomText & magni cation). fi Production code Colour; vision (e.g. Screen readers); auditory (e.g. captions), and mobility (Speech to text control).
So once you have your strategy, recruitment and plan, you need to think about what tools you can use for accessibility research. However, the one blocker I have (and still) come across with accessibility research is that there are not many tooling options on the current market. The majority of research tools are available are only partially accessible, or they cater to providing design feedback, which doesn’t give the opportunity to do in-depth and longitudinal research.
When it comes to doing remote accessibility research, because of the tooling limitation, so far I have used video communication software such as Zoom. If possible, I do prefer to adjust the remote video software to the participants preference and what they use to chat with family and friends or at work - this may be Google hangouts, Zoom or another option. With your video communication, make sure you ask participants who are using assistive technologies to share their computer audio, if you want to hear any potential issues. You should also remember to turn on live captions options in your zoom settings and turn on if a participant prefers it - they are not perfect but better than nothing! Source (Zoom Accessibility)
There are many things to consider during a research study to ensure we create an enjoyable participant experience for each individual, but also ensuring we use the correct language, etiquette (eticate) and consider potential individual user needs throughout the study.
A key point I would say to keep in mind during a research session and in your report writing is to make sure you put the person first and not their disability.
Source (Microsoft Inclusive Toolkit)
Some useful examples which I use for inclusive language guidance includes the National disability authority guidelines for effective consultation with people with disabilities. This guide was created many years ago but it is still useful as a guidance for inclusive language - but it is also very detailed. Source: nda.ie/nda-files/-Ask-Me-Guidelines-for-Effective-Consultation-with-People-with-Disabilities1.pdf
At Atlassian we use our own inclusive writing guidelines created by the design systems team. In the guidelines there are a selection of examples for speaking about accessibility and people who are living with disabilities. These guidelines are set up to help people within Atlassian understand the recommended inclusive language to use in meetings, at work and when conducting accessibility research. You can access this online https://atlassian.design/content/inclusive-writing
Source (NDA Guidelines) Source (Atlassian Guidelines)
When conducting accessibility research, you should use the same methods and techniques as you would for all research.
Finally you have completed the sessions and now it is time for the analysis! It can seem daunting to write about accessibility and accessibility issues that may surface from your research - but it is actually very straight forward!
Conclusions and reporting Con rm that any categorisations are appropriate and useful Be careful about reporting comparisons, especially with small sample sizes fi Explain what accessibility is, if your audience may not understand
When reporting your findings, it is beneficial to pull out the accessibility issues for your devs and designers. One way I have done this in the past is to create a Jira ticket for each accessibility issue. Mention hearing Jira mentioned in axe con. The example ticket shown here is a template only which I created for others to use and Is only for decortative purposes. , the issue will usually have significantly more detail but this is just a template and example. In the heading I would normally say what the issue is… SAY heading In description Go through points and say content (Ticket image is decorative only)
Utilise quotes and video highlights - this quote is from a research session with a screen reader user talking about a drop down. They say… With this example I would then discuss with my team how the inaccessible drop down options impact the user, not just from using the tool but also how it impacts their team, working together and their productivity. As the team has to stop working to help their colleague use the drop down, but the product should be accessible & the person should be able to use the drop down independently.
Different presentations for product teams, managers and stakeholders - this one shown is more aimed towards a product team. I have found that quotes and videos where users speak about the impact the accessibility has on them at work or personally makes more impact with leadership and stakeholders. Especially if it potentially is a business risk and customers considering going elsewhere.
Thank you for watching! Rebecca Topps Twitter: @beckytopps Email: hello@rebeccatopps.com
There’s a lot to consider if you’re going to do accessibility research effectively and respectfully.
Using examples from previous user research and accessibility studies she’s conducted throughout her career, Rebecca will share lessons learned and guidance to consider when planning and executing remote usability testing with participants with disabilities.
The following resources were mentioned during the presentation or are useful additional information.
for free. You
can too.