How Continuous Accessibility can boost your accessibility maturity

A presentation at Accessibility Camp Bay Area in May 2024 in San Francisco, CA, USA by Andrew Hedges

How Continuous Accessibility can boost your accessibility maturity — How Continuous Accessibility can boost your accessibility maturity
Presented at Accessibility Camp Bay Area, May 31, 2024 by Andrew Hedges, Assistiv Labs

Thanks for having me. I’m Andrew. — Thanks for having me. I’m Andrew.
Co-founder of Assistiv Labs Longtime web dev, engineering leader, & accessibility advocate Found on the socials @segdeha

At the end of our time, you will understand… — At the end of our time, you will understand…
…what is an Accessibility Maturity Model and how they’re helpful,
…the concept of Continuous Accessibility and how it can improve outcomes, and
…how Continuous Accessibility can contribute to advancing your organization’s accessibility maturity.

What is an Accessibility Maturity Model?

An accessibility maturity model is… — An accessibility maturity model is…
A tool
Self-reported
A process that provides insight into the effectiveness and repeatability of an organization’s methods for achieving a range of accessibility-related outcomes

Dimensions covered by the W3C model — Dimensions covered by the W3C model
Communications
Knowledge and Skills
Support
ICT (Information & Communication Technology) Development Life Cycle
Personnel
Procurement
Culture

How dimensions are assessed — How dimensions are assessed
“Each dimensional outcome has a range of suggested proof points, which includes any evidence or necessary measures that can be used to determine the maturity of each dimension. Progress towards achieving maturity is attained by creating the proof points described for each dimension.” – W3C

Example proof points — Example proof points
Procurement
Accessibility requirements and other information communicated to vendors
Personnel
Established goals for recruiting employees with disabilities

Proof points under “ICT Development Life Cycle” — Proof points under “ICT Development Life Cycle”
Development
Consistent approach to implementing accessibility features across products
Quality Review Through Release
Consistent approach to testing and releasing products
Testing process includes automated accessibility testing
Accessibility identified as product release gate

What is Continuous Accessibility?

Continuous Accessibility, defined — Continuous Accessibility, defined
“Continuous Accessibility is defined as the approach to ensuring that code intended to be displayed in browsers can be continuously checked and monitored for digital accessibility requirements through a novel application of existing software engineering concepts and Web Content Accessibility Guidelines (WCAG).” – ContinuousAccessibility.com

Why is Continuous Accessibility compelling? — Why is Continuous Accessibility compelling?
Attention decay. The sooner after it’s made that an engineer finds out their change broke something, the more likely they are to fix it. This goes for any type of bug and helps explain why tests in CI/CD are so popular and valuable, because it means catching problems before they impact end users.

Tools for ensuring digital accessibility quality — Tools for ensuring digital accessibility quality
Linters – e.g., axe Accessibility Linter
Scanners – e.g., WAVE Testing Engine
End-to-end tests – e.g., Cypress + cypress-keyboard-plugin
Manual QA
Periodic audits
AI

Tools for ensuring digital accessibility quality — Tools for ensuring digital accessibility quality
Linters – e.g., axe Accessibility Linter
Scanners – e.g., WAVE Testing Engine
End-to-end tests – e.g., Cypress + cypress-keyboard-plugin
Manual QA
Periodic audits
AI (j/k)

Accessibility quality tools, grouped by when they typically happen — Accessibility quality tools, grouped by when they typically happen
In dev
Linters
In CI/CD
Scanners
End-to-end tests
After dev
Manual QA
After deploy
Periodic audits

No offense, humans, but Continuous Accessibility is about what the machines can do. — No offense, humans, but Continuous Accessibility is about what the machines can do.
In dev
Linters
In CI/CD
Scanners
End-to-end tests

Pros & cons of linters & scanners — Pros & cons of linters & scanners
Pros
Fast
Cheap
Cons
Limited context around user goals
Fairly hard ceiling on the types of issues this approach can surface

Pros & cons of end-to-end tests — Pros & cons of end-to-end tests
Pros
Fast-ish
Cheap-ish
Can be written to take context & user goals into account
Cons
More expensive to run, so usually more limited in what gets covered
Existing tools are not accessibility-first

Linters & scanners pick the low-hanging fruit — Linters & scanners pick the low-hanging fruit
Use linters & scanners! They’re great for surfacing certain classes of accessibility bugs. End-to-end tests excel at ensuring that a user flow can be completed from start to finish, like validating the tree from root to fruit.

Pros & cons of end-to-end tests — Pros & cons of end-to-end tests
Pros
Fast-ish
Cheap-ish
Can be written to take context & user goals into account
Cons
More expensive to run, so usually more limited in what gets covered
Existing tools are not accessibility-first – Assistiv Labs is fixing this!

Let’s connect Maturity to Continuous Accessibility

Proof points under ICT Development Life Cycle — Proof points under ICT Development Life Cycle
Development
Consistent approach to implementing accessibility features across products
Quality Review Through Release
Consistent approach to testing and releasing products
Testing process includes automated accessibility testing
Accessibility identified as product release gate

Consistent approach to implementing accessibility features across products — Consistent approach to implementing accessibility features across products
Meeting this proof point should involve shifting well left of the development phase into UX research and UI design. To the extent that automated tools (from linters through to end-to-end tests) can validate code before it’s released, we can enforce a consistent baseline of functionality at all times, across a company’s products.

Consistent approach to testing and releasing products — Consistent approach to testing and releasing products
Implementing continuous accessibility tooling should be considered essential (but not sufficient) for meeting this one. I.e., manual validation by humans is still necessary for areas where the machines don’t give us 100% confidence.

Swiss cheese model — Swiss cheese model
Stacked slices of Swiss cheese is a metaphor for using layers of testing to catch a greater percentage of accessibility issues than is possible using any single method.

Testing process includes automated accessibility testing — Testing process includes automated accessibility testing
Amen. And, we need to expect more from the tools vendors provide.

Accessibility identified as product release gate — Accessibility identified as product release gate
Some of our customers are very sophisticated with respect to their product accessibility programs. I don’t know of a single one that would fail a build due to an automated accessibility test failing. Yet.

Y’all increasing your accessibility maturity through Continuous Accessibility after this talk, probably

Questions for the group…

Tools for ensuring digital accessibility quality — Tools for ensuring digital accessibility quality
Has your company/organization worked towards an accessibility maturity assessment?
How do you measure progress against your accessibility goals?
Do you think about Continuous Accessibility in the course of your work?
In what ways do you leverage automation to achieve your accessibility goals?

Photo credits — Photo credits
Grass & sky photo on slide 1 by Dim Hou on Unsplash
Selfie on slide 2 by yours truly
Seedlings photo on slide 4 by Irina Krutova on Unsplash
Watch gears photo on slide 10 by Lukas Tennie on Unsplash
Fruit picking photo on slide 19 by Skylar Zilka on Unsplash
Girl & robot photo on slide 21 by Andy Kelly on Unsplash
Swiss cheese photo on slide 25 via healthline.com
Ugly Windows sweater photo on slide 28 by Windows on Unsplash
Answers 1km photo on slide 29 by Hadija on Unsplash

Andrew Hedges
@segdeha

1 / 31

Continuous Accessibility is described by Melanie Sumner as “ensuring that code intended to be displayed in browsers can be continuously checked and monitored for digital accessibility requirements through a novel application of existing software engineering concepts and WCAG.” How does Melanie’s concept of Continuous Accessibility fit into digital accessibility maturity?