How To Do Remote Research Studies for Accessibility

Title slide

How to do remote research studies for accessibility by Rebecca Topps

In this talk I will be doing my best to answer the most common questions I have been asked about conducting moderated accessibility remote research, including:

How can I include accessibility in my research? What are e ective methods for nding participants? Which research tools should I use? How do I ensure my research sessions are inclusive? fi ff What do I need to consider in the analysis and reporting?

Atlassian

But before we get into answering those questions, you might be wondering why am I only talking about remote research today? Well, right at the start of the pandemic I started a new job at Atlassian in Sydney, Australia - Atlassian create work management software tools such as Jira, Confluence and Trello. My role at Atlassian was initially to lead a moderated accessibility research study which looked at the current state of accessibility of our core products. During the pandemic, Atlassian announced Team Anywhere, and as a result we were working remotely during lockdown, and all our research became fully remote. Although this was a difficult time and transition, doing my research remotely made me question why I had not been advocating for remote research sessions sooner and in previous roles. Or, at least providing the participants with a choice depending on what works best for them.

Remote research has a number of advantages for everyone, including those who are living with a disability. Including… - A person can stay within their own environment, no need for travel - A person can use their own device and potentially assistive tech or setup - mention example story around a user who had a specific version of Dragon with settings at home, but they could not replicate that in our research lab. This immediately impacts the findings as it will take longer for the user to adjust to Dragon without their usual, everyday settings. - A person is likely to feel more comfortable at home within their own environment than being recorded in a lab. Personally a lab can make me feel anxious and I’m a researcher - so I can imagine it can be the same experience for participants. Source (Mobility Works )

Benefits of accessibility research

In recent years I have noticed an increase in accessibility research roles and advocacy across the industry - including Google, Microsoft, Government departments, and now Atlassian. The reason more companies are prioritising accessibility within user research is because it There are a number of benefits, these include:

• It will contribute towards ensuring your work conforms to the accessibility legal standards (WCAG)
• It will improve your designs and the user experience for everyone - ensure everyone will have access to the essential tasks and key user journeys It ensures your research is fully inclusive and as a result that your product is inclusive for everyone. I have noticed it starts to make people think about other areas of diversity and inclusion - internationalisation, assisted digital, social economics, etc It can drive innovations, product improvements and improves accessibility understanding throughout your company

Quote

Whitney Quensenbery, a co-author of the book “A web for everyone” explains the importance of including accessibility to ensure our user research is fully inclusive, she says… (Say the quote)

• “…We can reach higher and aim to make the user experience delightful for everyone. That’s where user research comes in. You have to know the people you are designing for. Including people with disabilities.”- This quote is from her page called Better accessibility needs user research - she tells us that accessibility will improve our research and make sure it reaches everyone. Why would we not want to do that?

Things to consider

Before you start including accessibility within your research you should try to consider the following things:

• Think about team and company accessibility knowledge and training: Do you or your team members have a high level understanding of accessibility, user needs and assistive technologies? Can you provide in-depth training around how to correctly work and research with users who are living with disability?
• Check privacy laws: Ensure you speak with an expert on privacy or work with your companies legal team before conducting any accessibility research. There is something called special category data which we won’t get into today as it would take the whole 40 minutes! But it is important to understand with a legal team to ensure you are correctly handling and storing any users medical data, especially if you are working with users in EU and UK. \
• Research operations: If your company has a research operations team, you can work with them to assist with training, privacy and recruitment for accessibility. At Atlassian I am currently working with Kate Townsey and her team on all the areas mentioned here.
• Try to fix any major accessibility issues: If you want to research a digital product with users living with disabilities you should consider conducting a technical accessibility audit on the products before researching with users. In the past I have researched products too earlier with users, and due to the amount of technical accessibility issues users could not complete the majority of the key users journey’s on this product. It resulted in around 80% of the usability tasks being marked as a fail because the product was not ready for accessibility research. Iconography: https://www.flaticon.com/free-icons/instagram-like

How do I include accessibility?

You can do dedicated (moderated) accessibility research studies if you have the budget, however, you don’t need to conduct an accessibility-only focused study to do accessibility research.

How do I include accessibility?

A useful method is to include 2 users who identify as living disability for every 10 users. This results in 20% of your user group being users with an accessibility need - and worldwide around 20% of people identify as living with a disability. As a result you have a panel which also provides an inclusive representation to the population. This technique is practiced by Government Design Service in the UK, where it is a requirement to include accessibility in your research. However, it can be a useful method to include accessibility in your research, even if you start with a small percentage of users. Also, I view accessibility a as a big spectrum which many of us are experience at some point of our lives - whether that is due to mental health, an accident or the ageing process. Therefore, eventually you can use this technique to slowly start to include areas such as assisted digital, mental health and so on within your research participant recruitment screener.

Participant recruitment

The most common question I am asked is how to you find, source and recruit participants for accessibility research…

Participant recruitment

Some of the methods I recommend and have used are the following: Internal Panel - a good technique to build a panel is to advertise it online, Google and Microsoft do this to find users who want to be involved in QA testing and research. Build a panel or recruit in collaboration with a vendor (this removes some of the work around privacy and GDPR). Contacting charities and organisations - find local disability orgs, in the past I have focused on digital or creative workshops. You can also look for digital literacy workshops at libraries if you are doing assisted digital research & often you will find that a lot of those users are living with a disability. Advertising on social media and/or forums -you can advertise by saying you are looking for people who are living with a disability and who use your product to be involved in some research.

Iconography: https://www.flaticon.com/free-icons/instagram-like

Recruitment considerations

Recruitment considerations Plan for extra time to recruit Prepare for additional costs Check your vendor is accessible Adjust your recruitment screener

Pre-research questions

Pre-research questions Do you use any assistive technology or adaptive strategies when using our product? If yes: What assistive technologies or adaptive strategies do you use? Are there any accommodations, support or changes we can provide during the research session?

Pre-research questions

How would you prefer to receive the research questions or tasks? E.g. email, verbal, chat… Would you like to have breaks during the session? How often do you prefer a break? Are you okay with sharing your video and audio? (if possible offer the option to turn off video)

Who to include when

When planning to include accessibility in your research, often people ask me what user needs they should focus on for different research methods and stages of the product lifecycle…

Who to include when diagram

Interviews You can research all areas of accessibility and include all user needs. Paper prototyping / Mockups Cognitive; language; communication, and auditory. Prototypes Colour; language; communication; cognitive; mobility (e.g. keyboard only) and vision (e.g. ZoomText & magni cation). fi Production code Colour; vision (e.g. Screen readers); auditory (e.g. captions), and mobility (Speech to text control).

Accessible research tools

So once you have your strategy, recruitment and plan, you need to think about what tools you can use for accessibility research. However, the one blocker I have (and still) come across with accessibility research is that there are not many tooling options on the current market. The majority of research tools are available are only partially accessible, or they cater to providing design feedback, which doesn’t give the opportunity to do in-depth and longitudinal research.

Zoom photo

When it comes to doing remote accessibility research, because of the tooling limitation, so far I have used video communication software such as Zoom. If possible, I do prefer to adjust the remote video software to the participants preference and what they use to chat with family and friends or at work - this may be Google hangouts, Zoom or another option. With your video communication, make sure you ask participants who are using assistive technologies to share their computer audio, if you want to hear any potential issues. You should also remember to turn on live captions options in your zoom settings and turn on if a participant prefers it - they are not perfect but better than nothing! Source (Zoom Accessibility)

What to consider during a research session

There are many things to consider during a research study to ensure we create an enjoyable participant experience for each individual, but also ensuring we use the correct language, etiquette (eticate) and consider potential individual user needs throughout the study.

Inclusive design

A key point I would say to keep in mind during a research session and in your report writing is to make sure you put the person first and not their disability.

• Speak to a person living with a disability as you would speak to anyone else - using person-first language such as ‘a person living with a disability’ and ‘a person who Is colourblind’. Please note that this isn’t universal - some communities embrace different practices and all individuals may have different preferences. It is okay to ask the person if you are unsure. Also, Only mention disabilities when it is relevant - I practice this in my studies by asking the user if they use any assistive technology and adaptive strategies. So that way I am focusing on the technology and if a person chooses to explain their personal story, requirements or disability then I leave that to be their own choice during the session.

Source (Microsoft Inclusive Toolkit)

Guidelines

Some useful examples which I use for inclusive language guidance includes the National disability authority guidelines for effective consultation with people with disabilities. This guide was created many years ago but it is still useful as a guidance for inclusive language - but it is also very detailed. Source: nda.ie/nda-files/-Ask-Me-Guidelines-for-Effective-Consultation-with-People-with-Disabilities1.pdf

At Atlassian we use our own inclusive writing guidelines created by the design systems team. In the guidelines there are a selection of examples for speaking about accessibility and people who are living with disabilities. These guidelines are set up to help people within Atlassian understand the recommended inclusive language to use in meetings, at work and when conducting accessibility research. You can access this online https://atlassian.design/content/inclusive-writing

Source (NDA Guidelines) Source (Atlassian Guidelines)

Research methods

When conducting accessibility research, you should use the same methods and techniques as you would for all research.

• For example, try not to guide the participant straight away if they are unsure on how to complete a task and don’t ask leading questions - as you would with all research - give the user some time to try and find the solution before stepping in to ask open-ended questions. [tell story about keyboard or SR not accessible and you had to ask what they would do with you not there] In these situations I often ask if there are any “hacks” or work arounds they would try in this situation if I wasn’t watching them - this can give you insight into what a person would try when trying to use a product which is not accessible.

Analysis and reporting

Finally you have completed the sessions and now it is time for the analysis! It can seem daunting to write about accessibility and accessibility issues that may surface from your research - but it is actually very straight forward!

Conclusions and reporting

Conclusions and reporting Con rm that any categorisations are appropriate and useful Be careful about reporting comparisons, especially with small sample sizes fi Explain what accessibility is, if your audience may not understand

Surfacing accessibility issues

When reporting your findings, it is beneficial to pull out the accessibility issues for your devs and designers. One way I have done this in the past is to create a Jira ticket for each accessibility issue. Mention hearing Jira mentioned in axe con. The example ticket shown here is a template only which I created for others to use and Is only for decortative purposes. , the issue will usually have significantly more detail but this is just a template and example. In the heading I would normally say what the issue is… SAY heading In description Go through points and say content (Ticket image is decorative only)

Findings

Utilise quotes and video highlights - this quote is from a research session with a screen reader user talking about a drop down. They say… With this example I would then discuss with my team how the inaccessible drop down options impact the user, not just from using the tool but also how it impacts their team, working together and their productivity. As the team has to stop working to help their colleague use the drop down, but the product should be accessible & the person should be able to use the drop down independently.

Different presentations for product teams, managers and stakeholders - this one shown is more aimed towards a product team. I have found that quotes and videos where users speak about the impact the accessibility has on them at work or personally makes more impact with leadership and stakeholders. Especially if it potentially is a business risk and customers considering going elsewhere.

The end

Thank you for watching! Rebecca Topps Twitter: @beckytopps Email: hello@rebeccatopps.com