Auditing Design Systems for Accessibility

A presentation at Clarity in November 2022 in New Orleans, LA, USA by Anna E. Cook

Auditing Designs Systems for Accessibility — Auditing Designs Systems for Accessibility
Hi everyone, and thank you for coming to my session today on auditing design systems for accessibility. I’m so thrilled to be joining you all at my first Clarity conference!

Hi, I'm Anna! — Hi, I'm Anna!
My name is Anna E. Cook and I am a pale, short, white woman with blonde hair and blue eyes
I use she/her or they/them pronouns
I am a Senior Designer at Microsoft focusing on accessible and inclusive design and a Master’s student at the ATLAS Institute of CU-Boulder focusing on inclusive design.
I am planning to graduate this upcoming Spring

What we will discuss — What we will discuss
Today, we will discuss some of the most pivotal accessibility work product teams can do: creating design systems with baked-in accessibility conformance.
We will focus on finding accessibility issues in our design systems and documenting them in a way that makes them actionable for our team.
I have given talks about auditing design systems for accessibility before
But I’m going to approach it differently today, so we’ll talk about some new things

Design Systems & Accessibility — Design Systems & Accessibility
Let’s start with a conversation about design systems and their relationship to accessibility
When I talk about design systems and accessibility the first thing * I talk about is atomic design principles
This is a direct reference to Brad Frost’s Atomic Design

Atomic design principles — Atomic design principles
Essentially, each atom, molecule, and organism we create in our systems affects our ability to scale upwards and meet disabled people’s needs.
Atomic design means that our systems build on top of each other Each team has a slightly different understanding of jargon around this scale, but these principles’ fundamental elements tend to carry consistently through most digital design systems.
Starting with atoms, which can be combined to create molecules and can further be combined to form organisms then templates and pages.

Atomic design applied — Atomic design applied
I’m not going to go into depth on atomic scaling for design systems, because I want us to focus on the accessibility side of design systems today
Let’s dive into this discussion using an example from a sample design system
In this system, we’ve taken our atoms, put them into a molecule, bundled the molecule with other atoms and molecules to create an organism then a consuming team applied it to a page.

Let’s look at this system a little more closely… — Let’s look at this system a little more closely…
In this example, let’s say that the designers from a consuming product team have used the components built into our system to build this shipping address form.
The teams using our system use it thinking that it is accessible ”out of the box”

Checking the accessibility of our page… — Checking the accessibility of our page…
No icon button label provided (visually or programmatically)
None of the form fields have required indicators, so users wouldn’t know what is required or optional.
The form fields have low contrast borders, users may struggle to select the field (contrast ratio is 1.48 to 1).

Checking the accessibility of our page (part 2) — Checking the accessibility of our page (part 2)
The form fields have low contrast borders, users may struggle to select the field (contrast ratio is 1.48 to 1).
No input instructions are provided, how will users prevent errors if an input is not matching the desired format?
The color contrast on the save button is too low, people will struggle to read it (contrast ratio is 2.38 to 1).

Looking at our button atom — Looking at our button atom
Let’s break this down with one atom from our system: our button. Our example button design may have a few shortcomings
But let’s focus on how it didn’t meet color contrast conformance. Our button has used light blue in its background and white text on top.
Our text is 14px, semibold weight, and white in color.
When testing this button with a color contrast checker, we see that it doesn’t meet contrast conformance.
This combination of colors has a ratio of 3.22:1, but with a 14px semi-bold font style
The contrast should be 4.5:1 to pass the minimum required level.

Looking at our button atom — Looking at our button atom
But this combination of colors is going to affect more than just this component.
If it is used in other components in the system, suddenly this one instance will branch out Not only to all instances of that button used across products
But also, to all instances of components using that blue and white together similarly
So, we know there’s an issue with a component,
But we also know that it’s likely going affect other components because its problem is foundational, in our system.

Creating a more accessible design system empowers teams to build inclusive products — Creating a more accessible design system empowers teams to build inclusive products
That’s why we must make accessible design systems: we are the foundation for accessible products
Designs systems that fail to account for accessibility create products that fail accessibility conformance and most importantly fail disabled users
But alternatively, creating a more accessible design system empowers teams to build more inclusive products
So let’s talk about one way to improve our design system accessibility

Auditing your design system — Auditing your design system
By auditing a design system, we can make fixes that branch out across our components, and across products
So how can we review our system, log issues, and prioritize them to be addressed?
This is where our auditing starts to come in handy

Accessibility audits are a way to find and log issues in our designs so we can fix them — Accessibility audits are a way to find and log issues in our designs so we can fix them
Accessibility audits are a way to find and log issues in our design systems so we can fix them
Now auditing a design system can be applied at any point in our design system’s maturity
Regardless of whether we are just starting out or already have a robust system.
So how do we run an accessibility audit on a design system?

Log components in audit for review — Log components in audit for review
Our accessibility audit will begin like a regular component audit: by creating a log of components we will review
Making sure we know what exists and what we want to review in more detail
What exactly we review and how we review it will depend on our system, scale and maturity
So this step is about defining and logging what is in scope
Your team may have an early maturity system mostly in Figma, in which case you’d review designs only
Or if your system is more robust, you’ll maybe focus on reviewing both designs and code in Figma, system documentation, and component repos like Storybook
Review components most actively used to create products in both designs and code.

Start your audit documentation — Start your audit documentation
This component review is going to funnel directly into our itemized document
I tend to approach audits by creating a log of what we plan to review and placing links to files and pages in a single document
I recommend building our audit in a spreadsheet because the results tend to require a lot of details that can be helpful to filter and refine
We will insert component names into a sheet such as this and add to it as we find issues.
Each component can have multiple issues, so eventually, we might have multiple line items for a component, but we can start with just one
It’s also a good idea to make sure this document has a background page mentioning what was reviewed, who reviewed it and when it was reviewed.

Use WCAG to check designs and code — Use WCAG to check designs and code
Next, let’s discuss how we find accessibility issues our components
For our entire audit, no matter what we review, we will be using the Web Content Accessibility Guidelines (or WCAG).
These internationally and legally backed guidelines have been created with and by people with disabilities to ensure digital interactions are accessible. So we can trust them.
These guidelines include specifications for accessibility in content, visuals, interactions, and code so they apply to everything.
Our teams will want to include everything that is marked as A and AA in our review
A and double AA are levels of conformance
Meeting these tends to ensure that we are meeting the minimum levels required to create accessible components
AAA can be helpful but is not usually required

WCAG’s Quick Reference Guide can serve as a checklist for your review — WCAG’s Quick Reference Guide can serve as a checklist for your review
When it comes to using WCAG, I tend to personally keep WCAG’s quick reference guide handy for any reviews
I find this to be the most easily understandable version of WCAG and I use it like a checklist
The guide has every currently released WCAG Criteria listed, along with its A level and how to meet it with examples
So, get familiar with WCAG and keep it handy and use this guide

What to review in your audit — What to review in your audit
Our next few steps can happen in tandem
This is the bulk of our audit, reviewing designs, code, and docs
Each of these reviews tends to require different skillsets
So, you might choose to have a few people audit each of these to ensure everything is captured and to do so a little faster than one person might be able to do alone

Review designs — Review designs
First up, reviewing designs
When it comes to reviewing designs, you can do so in design files or even in component repos like Storybook
The thing to keep in mind about design review is that it can’t really capture a lot using automated testing tools like contrast checkers or dev tools alone
But many accessibility issues start in design with us and developers can’t out code inaccessible designs

67% of accessibility issues can originate in design — 67% of accessibility issues can originate in design
In fact, in a case study conducted by Deque, they found that 67% of accessibility issues audited related to design
The cost of fixing design mistakes can be amplified as issues move across a product cycle,
It can take 6 times longer to fix defects in development, 12 times longer to fix them in testing, and 30 times longer to fix them in production.
In our design systems and products alike, it is crucial we look at accessibility issues in design (Percentages of design issues may vary, this stat is not an exact representation of all audits)

What do we review in design? — What do we review in design?
Color contrast and usage
Content, copy, readability
Headings and Page Titles
Link purpose
Hover and focus states
Forms (errors, labels, etc.)
Layout (consistency, responsiveness)
Media (captions, alt text, etc.)
Tab order and bypass blocks
Timing
Typography

Look past styling… — Look past styling…
A lot of accessible design lives beyond our colors and typefaces so, remember that you must look past styling alone
Here is an example from our sample system
In the original info, success, warning, and error alerts, we relied on color alone to indicate that information to users
But relying on color alone is a WCAG issue, specifically WCAG 1.4.1 Use of Color
This would make it less accessible for folks who are colorblind, low vision, or blind
By finding this issue, we can update these to include icons and alt text for further meaning

Review Code — Review Code
Next, we should be reviewing code for accessibility.
While I understand coding and accessibility I’ll focus on this briefly and leave the best auditing code talks to experts like Marcy Sutton and Sara Soueidan
That said, there are many tools that can help get us to do this Tools like those built into our browsers as well as Deque’s axe extension and linter tools in your code editors can help
These tools can find automated issues easily and guide us through manual review
Just know that we cannot find all or even most of our code accessibility issues just doing automated checks
Combined our automated checks with manual checks using our WCAG Quick Reference and we can discover a lot

Review System Documentation — Review System Documentation
Lastly, we want to review any design system documentation we have
The reason we do this is even a perfectly accessible system can be broken if used incorrectly
Our consuming teams need to have references for how to use components with accessibility in mind
For example, we don’t want folks using an element with a button tag to link to a different page because buttons are designed to change the context of the current page not navigate elsewhere
IBMs Carbon Design System does a pretty good job of clarifying how to use components with accessibility in mind if you’re looking for a starting reference point

Add the issues you find to your doc — Add the issues you find to your doc
Audit docs should be actionable and easy to reference back to later.
With each review we do, we are going to add issues to the spreadsheet we started earlier
Everything we put in this sheet should be written to be actionable and easily referenceable for when our teams go to fix issues.
Here we should title the issue by our component name, then describe it There are a few other items I recommend doing in addition to this

Map issues to WCAG criteria — Map issues to WCAG criteria
In our sheet, we want to ensure we reference the WCAG criteria related to what we found
That way we can better prioritize issues, clarify what they are and why they are logged
We can anchor link directly to individual criteria in our WCAG quick reference guide so that folks can jump directly to it
This ensures that folks know there is legitimate backing to the issues we found

Group issues into common themes — Group issues into common themes
I also recommend grouping issues by common themes
So, from our earlier example, we could create a theme based on the usage of white and light blue in our components calling the theme “Color Contrast”
This helps us show how we can fix one issue and have the results of that fix branch across our components and products
It also helps us better understand how a type of issue is reoccurring
That way we can prioritize time, and help others understand high-level results from the review

With WCAG mapped and themes outlined, we can also prioritize issues
So what will have the biggest impact if left as is or fixed? *I tend to assess impact based on three factors: impact on users, business, and usage
Impact on users – how significant of a barrier is created with this issue?
If the issue is A level WCAG item its critical impact, AA is still high impact but maybe a little less
If users with disabilities are upset about this issue, it is a higher impact
Impact on business – does this component get used in essential interactions in our business (for example a log-in form to access products)?
If it is used a lot in essential, interactions it is a higher impact
Impact by usage – how often is this component used across products?
If we use a lot of that single component across products its higher usage
If a lot of components fit into one theme, it’s higher usage. More usage means a higher impact
For prioritization 3 to 5 rating scale of impact from minor to critical should communicate the impact well

Adjust audit as needed for the system, product, and organization — Adjust audit as needed for the system, product, and organization
This all stated, every audit is different because every system, product, and organization is different
So don’t feel like you have to use this exact approach in your review & Be sure to adapt it as best suits your circumstance

Auditing cannot replace testing and designing with disabled users. — Auditing cannot replace testing and designing with disabled users.
*I want to mention that auditing cannot replace testing and designing with disabled users
One of the most essential aspects of accessible design is designing with disabled people instead of for disabled people.
WCAG is written with and by disabled people, so it’s great to use in an audit. But is it everything? No.
Like all design, accessibility requires us to talk to people and learn about their unique circumstances, especially when using our sites and products.
It’s a key component, but we need to continually hear and respect the needs of disabled people.
I encourage you to take insights from your audits, find ways to ask users what they think, test ideas with them, and pay particular attention to your marginalized users.

Our job is to understand people, their needs, and their problems. *Similarly, we need to be listening to disabled people and want to encourage you to begin becoming comfortable with inclusive research strategies.
Collective teams can come together to learn from a diverse set of users, including users with disabilities.
Referencing guidelines like those put out by the Inclusive Research collective can be a great place to start or hiring folks like Fable to get feedback from folks with disabilities can be helpful too.

Acting on your accessibility audit — Acting on your accessibility audit
As we wrap up today, let me share some ways to act on your audit so its not just left sitting there in some Google Drive folder

Share out results of the audit with leaders and consuming teams. 35

Present a summary of audit findings — Present a summary of audit findings
A simple summary of your audit will help people understand what you found and what should come next.
We can use the categories we added to our individual results and show high-level data such as the amount of impact per issue, the themes we found, and more
This helps show our organizations what we are doing and why it matters
If our organization wants accessibility data, this is how we can start to show it

Share what issues exist with consuming teams using your system docs. — Share what issues exist with consuming teams using your system docs.
Secondly, do not hide the results of your audit from consuming teams – instead be transparent
This means showing them, component by component what we know the issues to be

Share known issues — Share known issues
Pretending our design system is perfect is a disservice to everyone.
Let teams know what issues are known in the documentation
Feel free to link to open tickets in your backlog so that teams ca
Create avenues for more feedback
Create opportunities to contribute and refine

If you leave those accessibility issues in your backlog forever,I will haunt you. — If you leave those accessibility issues in your backlog forever,I will haunt you.
Lastly, please, please create a plan to fix issues and integrate that into your team’s goals
I cannot tell you how frustrating it is to have an audit just left sitting for months or years without action
When your audit is done, get to fixing things
Create tickets, set OKR goals, and start pulling them into Sprints
So help me, if you leave accessibility issues in your backlog forever, I will haunt you.

Thank you! — Thank you!
Let’s Connect. www.AnnaECook.com or @AnnaECook

Resources & Credits

Tools to get started — Tools to get started
Auditing Design Systems for Accessibility: deque.com/blog/auditing-design-systems-for-accessibility/
Atomic Design by Brad Frost: atomicdesign.bradfrost.com
WCAG Quick Reference: w3.org/WAI/WCAG21/quickref
A11y Project Accessibility checklist: a11yproject.com/checklist
Stark plugin: getstark.co
Deque axe: deque.com/axe

Sources — Sources
Slide 7: Atomic Design by Brad Frost atomicdesign.bradfrost.com
Slide 8: WCAG Quick Reference w3.org/WAI/WCAG21/quickref
Slide 11: Stark plugin: getstark.co
Slide 22: Deque axe deque.com/axe
Slide 21: Deque Accessibility and Automation: Shift Left ROI
Slide 21: Deepsource The exponential cost of fixing bugs
Slide 23: Carbon Design System carbondesignsystem.com

Anna E. Cook
@annaecook

1 / 42

Design systems are a crucial part of building accessible experiences. Each atom, molecule, and organism we create in our systems affects our ability to scale across our product offerings and meet disabled people’s needs. In this session, learn how to audit components for accessibility issues from design to code using plugins, best practices, and testing tools. Walk away from this session empowered to make your design systems accessible sooner!