How Continuous Accessibility can boost your accessibility maturity

How Continuous Accessibility can boost your accessibility maturity

Presented at Accessibility Camp Bay Area, May 31, 2024 by Andrew Hedges, Assistiv Labs

Thanks for having me. I’m Andrew.

Co-founder of Assistiv Labs Longtime web dev, engineering leader, & accessibility advocate Found on the socials @segdeha

At the end of our time, you will understand…

1. …what is an Accessibility Maturity Model and how they’re helpful,
2. …the concept of Continuous Accessibility and how it can improve outcomes, and
3. …how Continuous Accessibility can contribute to advancing your organization’s accessibility maturity.

What is an Accessibility Maturity Model?

An accessibility maturity model is…

• A tool
• Self-reported
• A process that provides insight into the effectiveness and repeatability of an organization’s methods for achieving a range of accessibility-related outcomes

Dimensions covered by the W3C model

• Communications
• Knowledge and Skills
• Support
• ICT (Information & Communication Technology) Development Life Cycle
• Personnel
• Procurement
• Culture

How dimensions are assessed

“Each dimensional outcome has a range of suggested proof points, which includes any evidence or necessary measures that can be used to determine the maturity of each dimension. Progress towards achieving maturity is attained by creating the proof points described for each dimension.” – W3C

Example proof points

Procurement

• Accessibility requirements and other information communicated to vendors

Personnel

• Established goals for recruiting employees with disabilities

Proof points under “ICT Development Life Cycle”

Development

• Consistent approach to implementing accessibility features across products

Quality Review Through Release

• Consistent approach to testing and releasing products
• Testing process includes automated accessibility testing
• Accessibility identified as product release gate

What is Continuous Accessibility?

Continuous Accessibility, defined

“Continuous Accessibility is defined as the approach to ensuring that code intended to be displayed in browsers can be continuously checked and monitored for digital accessibility requirements through a novel application of existing software engineering concepts and Web Content Accessibility Guidelines (WCAG).” – ContinuousAccessibility.com

Why is Continuous Accessibility compelling?

Attention decay. The sooner after it’s made that an engineer finds out their change broke something, the more likely they are to fix it. This goes for any type of bug and helps explain why tests in CI/CD are so popular and valuable, because it means catching problems before they impact end users.

Tools for ensuring digital accessibility quality

1. Linters – e.g., axe Accessibility Linter
2. Scanners – e.g., WAVE Testing Engine
3. End-to-end tests – e.g., Cypress + cypress-keyboard-plugin
4. Manual QA
5. Periodic audits
6. AI

Tools for ensuring digital accessibility quality

1. Linters – e.g., axe Accessibility Linter
2. Scanners – e.g., WAVE Testing Engine
3. End-to-end tests – e.g., Cypress + cypress-keyboard-plugin
4. Manual QA
5. Periodic audits
6. AI (j/k)

Accessibility quality tools, grouped by when they typically happen

1. In dev

• Linters

2. In CI/CD

• Scanners
• End-to-end tests

3. After dev

• Manual QA

4. After deploy

• Periodic audits

No offense, humans, but Continuous Accessibility is about what the machines can do.

1. In dev

• Linters

2. In CI/CD

• Scanners
• End-to-end tests

Pros & cons of linters & scanners

1. Pros

• Fast
• Cheap

2. Cons

• Limited context around user goals
• Fairly hard ceiling on the types of issues this approach can surface

Pros & cons of end-to-end tests

1. Pros

• Fast-ish
• Cheap-ish
• Can be written to take context & user goals into account

2. Cons

• More expensive to run, so usually more limited in what gets covered
• Existing tools are not accessibility-first

Linters & scanners pick the low-hanging fruit

Use linters & scanners! They’re great for surfacing certain classes of accessibility bugs. End-to-end tests excel at ensuring that a user flow can be completed from start to finish, like validating the tree from root to fruit.

Pros & cons of end-to-end tests

1. Pros

• Fast-ish
• Cheap-ish
• Can be written to take context & user goals into account

2. Cons

• More expensive to run, so usually more limited in what gets covered
• Existing tools are not accessibility-first – Assistiv Labs is fixing this!

Let’s connect Maturity to Continuous Accessibility

Proof points under ICT Development Life Cycle

Development

• Consistent approach to implementing accessibility features across products

Quality Review Through Release

• Consistent approach to testing and releasing products
• Testing process includes automated accessibility testing
• Accessibility identified as product release gate

Consistent approach to implementing accessibility features across products

Meeting this proof point should involve shifting well left of the development phase into UX research and UI design. To the extent that automated tools (from linters through to end-to-end tests) can validate code before it’s released, we can enforce a consistent baseline of functionality at all times, across a company’s products.

Consistent approach to testing and releasing products

Implementing continuous accessibility tooling should be considered essential (but not sufficient) for meeting this one. I.e., manual validation by humans is still necessary for areas where the machines don’t give us 100% confidence.

Swiss cheese model

Stacked slices of Swiss cheese is a metaphor for using layers of testing to catch a greater percentage of accessibility issues than is possible using any single method.

Testing process includes automated accessibility testing

Amen. And, we need to expect more from the tools vendors provide.

Accessibility identified as product release gate

Some of our customers are very sophisticated with respect to their product accessibility programs. I don’t know of a single one that would fail a build due to an automated accessibility test failing. Yet.

Y’all increasing your accessibility maturity through Continuous Accessibility after this talk, probably

Questions for the group…

Tools for ensuring digital accessibility quality

1. Has your company/organization worked towards an accessibility maturity assessment?
2. How do you measure progress against your accessibility goals?
3. Do you think about Continuous Accessibility in the course of your work?
4. In what ways do you leverage automation to achieve your accessibility goals?

Photo credits

• Grass & sky photo on slide 1 by Dim Hou on Unsplash
• Selfie on slide 2 by yours truly
• Seedlings photo on slide 4 by Irina Krutova on Unsplash
• Watch gears photo on slide 10 by Lukas Tennie on Unsplash
• Fruit picking photo on slide 19 by Skylar Zilka on Unsplash
• Girl & robot photo on slide 21 by Andy Kelly on Unsplash
• Swiss cheese photo on slide 25 via healthline.com
• Ugly Windows sweater photo on slide 28 by Windows on Unsplash
• Answers 1km photo on slide 29 by Hadija on Unsplash