The intersection of performance and accessibility

The intersection of Performance & Accessibility, #PerfMatters 2019

So, let’s set the stage

We’re at a conference about performance,
With performance being the manner in which a mechanism behaves

I was conducting some accessibility auditing last year
Which is the process of performing manual and automated checks to see if a website can work with assistive technology
The client was a medical education app used to train caregivers

It was built using a single page application framework, using Google’s Material Design library to build the user interface

When I learned that, I thought:
Google made it? Oh sweet. I don’t have to worry as much.
So I fire up VoiceOver, macOS’s screen reader and start testing.
Things are going pretty well and then

VoiceOver crashes!
I try restarting Safari, I try clearing the cache, I try closing other tabs. Heck, I even try rebooting
Same result every time
So, I ask my boss what’s going on, and he says, “Oh, it’s probably problems with the Accessibility Tree.”

The what now?
Before I get more into this, here’s a question for you:
How do you describe an interface?
I’m not saying, “tell me what you see or hear after you turn the computer on.”
I’m saying how do you speak computer to a computer

The way we do that is with the Accessibility Tree
Fundamentally, it is is a collection of objects, all the atomic bits that make up a user interface
People who rely on assistive technology use the Accessibility Tree to navigate and take action on user interfaces
Without a functioning Accessibility Tree there is no way to describe the current state of what an interface is
And consequently no way for some people to be able to use a computer or smartphone
So suffice to say it’s a really important piece of technology!
So, how do you build an Accessibility Tree?

First, you have to give each individual object in an interface a Name
Names are a short identifiers used to describe purpose.

Then you give it a Role
Roles are predefined values that tell someone what they can do to the interface object
For example, a role of button means that someone can press it to activate some predefined action

Then we have Properties, which are the attributes of the object
Examples of this are its position in an interface, it’s state, or interactions that are allowed
For a button, that could mean a disabled state to prevent it from being clicked on

Finally, we have a Description which can provide further information about the object, if needed
These accessible objects are then programmatically grouped together to form more complicated interface components
So, how do you build these interface components?

Let’s say an operating system alert dialog, that’s a pretty commonplace thing for UI

Starting from the top down, we have a menubar
That’s the anchor that we’re going to attach everything to

And then we add a title for the menubar, so we know what it’s all about, and what contents we can expect to find
If you’re using a screen reader, titles are really helpful, as it saves you from having to dig around inside the component to figure out what it’s for

We might also need to close this dialog window once we learn what it’s for, so we include a close button in the menubar

Then we add the body of the dialog, the place where we can place the bulk of the dialog content

In the dialog’s body there’s another title, which asks us if we want to save the changes we made to the thing we were working on

We then add text to the body, which will provide more information about what happens if you choose not to save

Then we add a save button, to allow us to commit to saving the changes we made

And an accompanying cancel button, in case we don’t
And presto! We have a dialog component!

Because these collected objects have accessible names and are therefore present in the Accessibility Tree, we can speak computer to computer to get what we want

I can programatically say “go to the dialog

Then it’s body

and activate the save button
and it works!

-So, this might seem a little pedantic, but please bear with me for a bit

We do have this dialog component, but there’s more to it than just that

Interface components have to have a place to live, and that place is an operating system

Operating systems also include components that allow us to navigate to, and access the stuff you store on them

It’s also the place we install and run programs

And all the little doodads and widgets we can’t live without

And the browser, which is rapidly becoming an operating system in it’s own right

The browser contains the mechanisms we use to access the web
And the web is sort of the Wild West in that you can write more or less whatever you want and it’ll usually work, which is both a blessing and a curse

But regardless of how you write what you write, you can’t escape the fact that you have to ultimately create HTML

That HTML then has its appearance and behavior augmented by both JavaScript and CSS
Again, there’s many ways to go about doing this, but they are the other two requisite parts that make up the whole that is a website or web app

The HTML markup, augmented by both JavaScript and CSS creates the Document Object Model, or DOM tree, which is a programming interface for websites

Browsers then read the DOM, and all the information contained within it to draw an interface

Which is then shown to a user

The user can then take actions, which updates the DOM, which in turn updates what is visually rendered, allowing us to make our websites dynamic

Running in parallel is the Accessibility Tree, which is sampled from the generated DOM
I say sampled because the Accessibility Tree will use specialized heuristics to only surface things it deems necessary
Modern versions of the Accessibility Tree are generated after styles are computed, as certain CSS properties such as display and pseudo element content will affect it

This sampled version of the DOM is then read by various kinds of assistive technology, including, but not limited to screen readers
There’s two things I’d like to point out here:

First, using a visually rendered interface and assistive technology aren’t mutually exclusive
Not all screen reader users are blind
Many people choose to augment their visual browsing with devices that rely on the Accessibility Tree

Secondly, the Accessibility Tree relies on the user interacting with the DOM to update
It’s effectively a “read only” technology, in that we can’t directly work with it

Another thing you should be aware of is that it’s more an accessibility forest than an Accessibility Tree
There are different implementations of the Accessibility Tree, each depends on what operating system you’re using, and what version of said operating system is running
This is due to the different ways companies have built and updated their operating system’s underlying code

Just so we’re clear:

The DOM can be a part of the Accessibility Tree, but the Accessibility Tree is larger, and not limited to just the contents of your browser

Because of this, the accessibility tree is more brittle
It has more to pay attention to, and it’s technical architecture was developed before this whole internet thing really took off
Meaning it didn’t anticipate the sheer amount of information we’d be throwing at it.

Crashing the Accessibility Tree, and therefore breaking assistive tech support is bad, yes
But a large amount of information present in the DOM means that there’s an accompanying large amount of work the Accessibility Tree needs to do to figure out what’s what and report on it
Which slows. it. down.

This can create a lack of synchronization between the current state of the page and what is being reported by assistive tech
Meaning that a user may not be working with an accurate model of current screen, and be taking action on interactive components that are no longer present
This can create the confusing experience of activating wrong control or navigating to a place they didn’t intend to
So, how do we help prevent this?

Start with writing semantic HTML
Using things like the button element for buttons instead of ARIA slapped on a div helps lessen the amount of guesswork the Accessibility Tree has to do when it runs its heuristics to slot meaningful content into its description of the current state of things.
This serves to both speeds up its calculation and makes what it reports more reliable

And speaking of semantic HTML
Here’s the raw, living DOM tree of an actual website
In fact, it’s the website for this conference! Estelle, sorry not sorry

Even a static, performant website contains a lot of information a computer has to chew through

Again, utilizing semantic HTML, like the markup present here, helps reduce the effort it takes to generate the Accessibility Tree

And unfortunately, using semantic HTML is kind of a rare thing these days, which is partly why I’m here giving this talk

Okay, I think you get the gist of it

Of the code I just showed you, five slide’s worth,

We’ve only covered the top quarter of just the homepage
Again, it’s a lot of stuff to process

Keeping that DOM tree example in mind, we now have a simple form
It’s a pair of unstyled radio buttons asking you if you want the chicken or the fish as your meal preference

Here’s how that form might be translated into the Accessibility Tree
If we use semantic HTML, the names, roles, and properties are automatically filled in for us without any additional effort
When I was speaking about how semantic HTML slots in, this is a more granular example of what I’m getting at

Here’s a more high fidelity example, sampled again from the PerfMatters homepage
I’m using Firefox’s Accessibility Inspector to dig into a link
There’s a ton more information exposed, including attributes, states, actions, a child element count, relations, and available actions
All of this data is used by the Accessibility Tree to help it describe the shape of things

And here’s an even more high fidelity example, a raw text dump of the Accessibility Tree for this same page
This is an example of what the language might look like when we’re speaking machine directly to machine.

We don’t speak machine directly, though
Before we had Firefox’s Accessibility Inspector, we had to rely on more specialized tools to be the translators
The Paciello Group’s aViewer and Apple’s Accessibility Inspector are the two go-to resources
And you’ll still need them if you want to inspect anything other than websites, or do some really serious digging

So, let’s make the abstract immediate
What going on here, and why should you care?

With my auditing project, I narrowed the problem down to issues with Material Design’s radio inputs
Here’s how they appear visually

And here’s how they appear in code

To make a Material Design radio input, you need
Six HTML elements containing nine attributes, with a DOM depth of 3

You also need 66 CSS selectors containing 141 properties, which weighs in at 10k when minified

You will also need 2,374 lines of JavaScript, which weighs in at 30k minified

All that will get us a radio input

But you need more than one radio input to be able to use it properly

And oftentimes there’s a few options to select from

Sometimes there’s more than just a few options

And sometimes there’s even more than that

In the case of my audit, we had a ton of radio inputs conditionally being injected into the page

This all adds up

Google’s Lighthouse project, an open source tool that analyzes your website for performance problems, recommends an optimal website DOM tree has the following:
Less than 1,500 nodes
Max depth of 32 nodes
No parent node with more than 60 child nodes

What I’d like to call attention to is the max depth of 32 nodes bit
That might seem like a lot of wiggle room at first, but take a moment and think about the templates of the sites you’ve worked on.
There’s the frameset, wrapper divs, landmarks, components wrappers and if you’re doing it right, fieldsets on your forms
Each digging a little bit more inward

Another part of accessibility auditing is providing a fix to the problems you uncover
It’s tough work, but nobody likes to pay people money to tell them all the things that are wrong, but offer no solutions
We wound up recommending a radio input pattern that utilized three HTML elements

It’s a pattern developed by my friend Scott, taken from his excellent a11y_styled_form_controls project
It’s worth saying that Scott puts these patterns through their paces, and tests them with an incredibly robust set of assistive technologies
So a nice side benefit is knowing I can recommend them with confidence

Visually, it was completely indistinguishable from the original version

To compare the two solutions

We have a 50% reduction in HTML elements, and we cut the DOM depth down by a third

There’s also a 30% reduction in CSS selectors and properties, resulting in the CSS payload for this pattern being reduced by 90%

We’ve also completely removed 30k of JavasScript, which is 30k less blocking resource being served

Cobbling up a rough prototype, we were able to create an environment that mimicked the conditions of the site we were auditing, only using the new radio input code and guess what?

It worked! VoiceOver didn’t crash!

So why is this our problem?
Why should we care about the fragility of other people’s software?

Well, prioritizing developer ergonomics without considering the generated HTML can lead to bad things happening

And regardless of our setup, we tend to pile even more things on to our base experience

Things like ads

Analytics

Social marketing engagement tools

And development integration utilities

And then there’s the also cold war raging between websites and their users
Here is Facebook splitting a single word into 11 DOM elements to avoid ad blockers

This graph is generated from median values of how 1.3 million websites are generated, with JavaScript claiming the majority of the share.
This JavaScript majority means it’s that much more time spent blocking the browser from performing other other actions, including things like interfacing with assistive technology

When we throw out what the browser gives us for free, we oftentimes don’t realize there are very real, very important things we sacrifice in doing so

Here’s what Marco Zehe, Mozilla’s senior Accessibility QA Engineer has to say about all this:
Nibbling away at those milliseconds it takes for information to reach assistive technologies when requested. My current task is to reduce the number of accessible objects we create for html:div elements. This is to reduce the size of the accessibility tree.

This reduces the amount of data being transferred across process boundaries. Part of what makes Firefox accessibility slower on Windows since Firefox Quantum is the fact that assistive technologies now no longer have direct access to the original accessible tree of the document.

A lot of it needs to be fetched across a process boundary. So the amount of data that needs to be fetched actually matters a lot now. Reducing that by filtering out objects we don’t need is a good way to reduce the time it takes for information to reach assistive technologies.

This is particularly true in dynamically updating environments like Gmail, Twitter, or Mastodon. My current patch, slatted to land early in the Firefox 67 cycle, shaves off about 10 to 20% of the time it takes for certain calls to return.

Note that all this optimization is only for one browser, Firefox

And that there are a lot of browsers out there

It’s also not all about browsers, either
Here’s a refreshable braille display, one of many other kinds of assistive technology that interfaces with the accessibility tree

“Someone should do something!” The battle cry of bystanders everywhere
Let’s unpack this some, and figure out our available options

For screenreaders, the main ones are JAWS

And NVDA
Both are Windows screen readers

VoiceOver for macOS and iOS

And Talkback for Android
These are the big four screen readers you’re going to hear about
It’s sort of analogous to Chrome, Safari, Firefox, and IE, in that there’s more screen readers out there, but these cover the main use cases you’re most likely to deal with
While not all assistive technology are screen readers, if your site works well for them, chances are good it’ll work well for other assistive tech

Of them, all but one have an open issue tracker

But half have closed source code
This means that while we can file issues, we aren’t really empowered to do much more about it on the assistive technology layer
It’s also completely unrealistic to expect people to submit and follow issues or code pull requests across all trackers in addition to working a full-time job

Our only other realistic option is to keep the DOM trees on our sites nice and shallow
There are actual, tangible benefits to doing this

Here’s Marco again weighing in again on his optimization efforts:
Reducing the number of milliseconds it takes for a screen reader to start speaking after a key press from about 140 to 100 here, or from 120 to 100 there, doesn’t matter much on a fast machine. On a slow machine, that reduction is from about 230 to 250 down to 200 or 190.

Let’s talk about what slow machines means
If you are disabled and/or underserved, you face significant barriers to entering and staying in the workforce
This means you may have less access to current technology, especially the more expensive, higher quality versions of it
Another factor is some people who rely on assistive technology are reluctant to upgrade it, for a very justified fear of breaking the way they use to interact with the world

Slow machines may also not mean an ancient computer
Inexpensive Android smartphones are a common entry point for emerging markets, and with Android comes Talkback

A slow machine might also come from a place you may not be expecting
By which I mean you might have a state-of-the-art computer and are doing state-of-the-art things on it
This requires a ton of computational resources, which makes fast things slow
With less computational resources to go around, we may have unintended consequences, possibly re-creating a situation similar to our too many radio inputs problem

Think Electron apps, desktop programs that are built with web technologies

Accessibility auditing isn’t something people normally do out of the goodness of their own heart
It’s typically performed after a lawsuit has been levied against an organization
Let me be clear: when you create inaccessible experiences, you are denying people their civil rights
The Americans With Disabilities Act guarantees that people cannot be discriminated against based on their disability conditions, and this extends to both private organizations and the digital space
This is a good thing, as it guarantees protections as more and more of the services necessary to living life go online

You need to work with what you have, not what you hope will be
Part of this means understanding that when we want things to be better, we need to understand that these kinds of changes are really technically complicated under the hood and spread across multiple concerns
On top of that, accessibility fixes are often viewed as unglamorous and deprioritized compared to other features
Don’t believe me?

Here’s the support matrix for the title attribute, incorporated into the W3C’s HTML spec in 1993.
It’s 26 years later and we still have a ton of interoperability problems.

I don’t mean to bum you out, and I don’t expect you to become accessibility experts
However, as people who are interested in more than just the surface level of things, you are uniquely empowered to affect positive change
What I ask of you is to at least incorporate basic accessibility testing into your workflow
If anything, just check to see if a screen reader crashes:
A bad assistive technology experience is better than none at all

Whoa!
The slide background is yellow now!

This is what we call a great segue

I’m a designer by trade,
Part of that job means coming up with alternate strategies for allowing people to accomplish their goals
because sometimes the most effective solution isn’t necessarily the most obvious one

With that in mind, we’re going to talk about another definition of performance, which is the ability to actually accomplish something
All of these little surgical tweaks and optimizations don’t mean squat if people don’t understand how to operate the really fast thing you give them

Another aspect of dynamically injecting a ton of radio inputs into a page is that it adds more things for a person to think about
This is called cognitive load, and it’s a huge problem
It affects our ability to accomplish tasks, and accomplish tasks accurately

Namely it inhibits our memory

Problem-solving skills

Attention

Reading comprehension level

Math comprehension level

And shockingly, our ability to actually understand what we see

It’s such an important problem that NASA developed the Task Load Index, an assessment tool to help quantify it
This isn’t warm fuzzy feelings about empathy, this is a serious attempt by a government agency to refine efficiencies and prevent errors
One of the most interesting things about the existence of the Index is that it’s an admission that disability conditions can be conditional
Think about it: when’s the last time you were tired, distracted, sick, or learning a new skill?
Heck, when’s the last time you were drunk?
Cognitive load an important thing to track for building rockets, sure
But it also translates to other complicated things like making and using software

One of the things we can do to lessen cognitive load is to lessen the amount of information someone has to parse at any given moment

For the medical education app, we could have added an additional step into the user flow that asks a high level segmenting question
It’s a little more friction, sure
but it’s being used strategically as a focused choice, to keep the cognitive load lower

This would help filter down the results you get
Which makes it both easier for the person and the browser to parse

The other big-picture question we need to ask is if all this work is even necessary?
The most performant page is the one you never have to use, by which I mean how often can we side-step these issues by using other resources and information made available to us?

If you’re interested in this sort of thing, the 2018 Design in Tech report is a must-read piece
One of the things it was revealed is that surprisingly very few companies conduct qualitative user research
Which is the practice of asking people how they’d use a feature, and how they feel while they do it

I’m not a numbers person, but there does seem to be a trend going on here
Another interesting thing is barely any companies conduct qualitative testing for features after launching them
So we’re all just throwing features into the larger, ever-evolving ecosystem that is your product and not checking to see how it’ll affect things

We also need to remember that we’re only seeing those companies that beat the odds
The market is built on top of the corpses of failed businesses who poured all their cash and resources into the wrong features

The second big ask from this talk is really just repeating the first one
It’s one thing to read about something and believe it to be true, but it’s another thing entirely to put it out into the world without verifying it works

The web, and more importantly, the people who use it, are too important not to

Thank you to our captioner,
Michelle and the other staff
Mina for being a great emcee
Estelle for organizing such a wonderful conference
And most importantly, you, our audience
The slides for this talk are available online on Notist
And I’ll be both tweeting out a link to it and linking to it on my personal website

Selection Controls: Usage - Material Design
WAI-ARIA 1.0 User Agent Implementation Guide
The Accessibility Tree: Whatsock
The Browser Accessibility Tree - The Paciello Group
Semantics to Screen Readers

Accessibility APIs: A Key To Web Accessibility
Uses An Excessive DOM Size | Google Developers
Accessibility Viewer (aViewer) - The Paciello Group
Testing for Accessibility on OS X

patrickhulce/third-party-web
Accessibility and Performance - MarcySutton.com
There’s more to performance than meets the eye - Notist
JavaScript dominates browser CPU
The Ethics of Web Performance - TimKadlec.com

Mozilla accessibility architecture | MDN
Table A-6. Employment status of the civilian population by sex, age, and disability status, not seasonally adjusted - United States Bureau of Labor Statistics
HTML attributes Screen reader compatibility - PowerMapper
How and Where To Report Accessibility Bugs

WebAIM: Cognitive - Introduction
Cognitive difficulties: an overlooked aspect of accessible design
Cognitive Accessibility 101 - Part 1: What is Cognitive Accessibility
Cognitive Accessibility User Research - W3C First Public Working Draft 15 January 2015
2018 Design In Tech report