Be the Villain

A presentation at axe-con 2021 by Eric Bailey

Everything is awesome when you’re online!

Before we get started, there are a two things I’d like to cover
First is a content warning for this talk

The nature of this talk’s content means I will be discussing some pretty terrible things
The list is:
Domestic abuse, drug abuse, genocide, harassment, racism, stalking school shooting, and swatting

I will only be touching on the concepts described in this list lightly to prove a point, with the exception of school shootings, genocide, and harassment, in that order
I’ll need to provide more detail for these three topics in order to better articulate points I want to make about how to avoid them
If this talk’s content is something you don’t think you’re in a good place to listen to, I completely understand
The talks by Preety, Plum and Jack, and Wilco and Mary Jo are also happening at this time slot, and they both look great
I’ll also be posting this talk, including annotations and links, after it is concluded, if you feel like this is something you want to consume at your own pace

The second thing I’d like to call attention to is that I’m not the first person to be discussing the concept of subverted design, and hopefully not the last
A lot of the research for this talk is built from the painstakingly-assembled information created by women working in our industry, or web-adjacent technology industries

I would also like to say that finding people who have discussed this topic was difficult
I attribute part of that to survivor bias, the logical error that teaches us that we overlook what isn’t present as part of our selection process
It is my belief that many who could talk about this are no longer here, having left, or having been forced out of the industry

The goal of today’s talk is to equip you, the audience, on how to identify subverted design and guard against it
The beats I’ll be hitting on for today’s talk are:
What isn’t subverted design
How subverted design can ruin a product
Subverted design in the headlines
How subverted design is our responsibility, and
How to identify where subverted design could occur

I might be biased, but I think Inclusive Design is a really great framework for a lot of design-related things
The reason for this is that the nature of inclusivity requires thinking about how people can get left out and be discriminated against
Things like equality, accessibility, and fairness are really important, but we need to acknowledge the reason Inclusive Design even exists in the first place is because we oftentimes accidentally, or intentionally create the opposite
As a methodology, Inclusive Design allows us to take an idea and see how we can facilitate, or hinder access And this talk is very much in the spirit of that

So now let’s describe what subverted design is not
Defining these more commonly-understood parameters helps to give us a framework for what does, and what does not qualify as subverted design

First off, subverted design is not bad design
On this slide is a web page with a bright rainbow background and a marquee banner that reads Accept Jesus Forever Forgiven church
It inexplicably features: a passage from Romans 6:23, a background MIDI, a hot air balloon, a World War II fighter plane, a dove, some arches, a step ladder, and a banner ad for Maverick Christians.com
This prismatic vomit is a rather extreme example, but it is honestly pretty direct about its intentions, and there is little room for doing anything other than the six or so calls to action it wants you to undertake

Subverted design is also not what I like to call Beholden Design
Remember in the Before Times where we could go to the theater and spend our hard-earned money to watch a movie?
And that how, despite paying for this service, you would still be forced to watch some advertisements before the thing you actually wanted to watch was shown to you?
That’s an example of Beholden Design
It’s where a person enters into a system and is forced to endure something that benefits the system to ultimately get what they want

Speaking of systems, subverted design isn’t hacking
This is a little more subtle than the previous examples
Hacking is where someone breaks a system on a deeper level, typically by finding an exploit in code and using that exploit to gain access and exert control on the underlying systems that power the experience

Subverted design also isn’t a Dark Pattern, although they’re close cousins
If you’re not familiar, a Dark Pattern is a technique where a website or app deliberately uses a confusing user experience to trick you into doing things you didn’t intend to

For the example displayed here, SportsDirect has automatically added a product to your shopping cart and charges you for it, hoping that you won’t notice when you check out
Gross

So now you might be thinking that this is just a one-off, and that I’m really grasping at straws here
What I say back is that we should not be so quick to underestimate what people are capable of doing with the resources we give them
In design, there is the bias that our users are simple, and require straightforward instruction, oftentimes to the point where we infantilize them

What I am telling you is that even a less digitally-literate user may be able to subvert a system, provided there is enough motivation on their part
Here is an example of people figuring out that they could get free burgers from a McDonalds self-serve kiosk by customizing an order to a ridiculous degree -It’s a funny example, but the real fault here is the designers and developers who didn’t anticipate the logic behind their rules for order customization being taken to an extreme.

So what exactly is subverted design, then?
Now that we have some solid examples of what it isn’t, I’d like to tell you a story that illustrates an example of what it could be
This isn’t something I’ve made up, it was a potential client we were considering working with at my job
What I’m going to discuss here is how a poorly-thought out system can be taken advantage of by someone with malicious intentions
To do this, we’ll be breaking this system down into discrete components, to show how the components relate and rely on each other, and where that can fall apart under stress

This is a school
Schools are effectively a collection of classrooms, each classroom containing a teacher and their students

In a school’s organization, there is admin staff that the teachers answer to.
This is a simplified version of things, but for the sake of this talk, let’s say this is the principal

This is a school shooter
If you are not from the United States, this is a horrifically common thing that we’ve normalized
A school shooter could be a student, or an adult either working in the school, or who enters into it

In a school shooting scenario, a shooter may target a classroom
Nearby teachers will become aware of this once the shooting starts

These teachers will reach out to emergency response services, typically via a text or phone call from their smartphone
Because again, this is America, and we’ve normalized this horror to the point where we need to train our teachers for this kind of formerly unthinkable scenario

Each emergency response service can then take this reported information and push it out to school staff, including other teachers in the school, as well as other associated emergency response services
This allows both teachers and school admin, as well as emergency response services, to become aware of the situation, and act on information as the crisis unfolds

However, one big problem is that it is this is real life, and real life is messy
Things aren’t as straightforward as the scenario I just described
In addition to reaching out to emergency services, teachers will reach out to family members, fellow coworkers, friends, and loved ones
And to add to this complexity, emergency services are constantly communicating with each other as new information is made known

Friends, family, and loved ones may also try to reach out to teachers, students, and school admin when they learn about what is happening
In addition, local and national media may also try to butt in
This creates a ton of chaos in an already chaotic, high stakes situation
With it comes increased chances for out-of-date information, or bad information to be disseminated

To add to this, too much communication happening too quickly in a small area can cause the whole thing to crash

To address this issue, the potential client had the idea that they could focus their efforts on a single individual

In this case, it would be the principal, or other high-level school official

In the event of a shooting, teachers would be instructed to not pass information along to emergency services, and instead all notify a central figure

The idea being that the central figure could communicate with emergency officials on their behalf

Emergency services would then coordinate with each other, as well as disseminate information back to the school in one single consolidated transmission
In theory, this would streamline communication to keep bad information down, as well as helping to keep the network up
In practice, however, they failed to think like a villain

An attacker who gets control of a teacher’s phone now has a one-stop shop for accessing information about other teachers, students, and emergency services
It lets them know all sorts of information that would make an already terrible situation worse

If they get control of the school administrator’s phone, it gets even more dangerous
Using the client’s proposed feature set, they could send messaging to both emergency services and school staff
Messages that could, for example, corral staff, declare false alerts and all clears, and facilitate taunting, goading, and other forms of psychological manipulation

It also failed to anticipate a scenario where there were multiple coordinated attackers, a situation that has tragically happened before on multiple occasions
After a review of the full proposal, we were able to determine that a school shooter could potentially get access to:

Identifying the various ways this product could be subverted translated into the very real risk for harm to be amplified
This discovery represented a hard stop on the progress of a product that had already had tens, if not hundreds of thousands of dollars of development time poured into it
On this slide is a panel from a golden age comic depicting a sportscar that’s crashed into a tree placed at a fork in the road

-= The school shooting reporting app is one example, but this subversion of technology for personal purposes has been happening for quite some time now, and will continue to happen
If you start to look for this kind of thing, you will see it everywhere

In The Washington Post:
Women face new form of harassment on metro: Airdrop

In The New York Times:
Thermostats, locks and lights: digital tools of domestic abuse

In NewsHub:
Auckland woman ‘creeped out’ after restaurant worker uses her contact tracing details to hit on her

In The Verge:
Twitter bans animated png files after online attackers targeted users with epilepsy

In The Daily Dot:
Facebook is helping husbands ‘brainwash’ their wives with targeted ads

In Buzzfeed News:
Instagram’s opioid recovery hashtags are full of drug dealers

In The Verge:
Swatting over Call of Duty game results in deadly police shooting of Kansas man

I keep a list of these headlines as I run across them

And part of creating this talk was whittling this list down to a length that could prove my point

Or completely shutting you, the audience, down
But I do need to stress that this is happening, and it is widespread
As technology enters into more and more of our daily lives, there are more and more chances for this subversion happen
It also means the scope of harm gets exponentially worse
Culminating in this:

Mynmar military personnel used Facebook’s features to orchestrate an ethnic cleansing of an minoritized population
This is genocide, facilitated by technological subversion.
Don’t look away

We need to stop being naive or this will continue to happen

Design finally got a seat at the table, and you know what? That’s great
However, this also means the scope of our job has expanded
We’re no longer people who just make pretty things, we now help shape how businesses operate in the world

With this new set of responsibilities comes the uncomfortable understanding that we may not be the heroes we thought we were
Historically there’s been this notion that we fight for the user’s best interests and be the first line of defense for guarding them online
But realistically, design is now mainly a profession that uses aesthetic and usability best practices to help businesses more efficiently generate profit
This shift is worth acknowledging for many reasons, but the one I’d like to focus on for this talk is the vacuum that this creates
And how one of the many things that can enter into this gap is a higher likelihood for harm
On this slide is a comic panel depicting an ecstatic man in a suit half-submerged in a bag of money, completely oblivious to the sinister hand reaching for him

We need to understand that we’re still pretty new at thinking about how to prevent harm, but others in the tech industry have been doing it for a long time
We can look to the field of information security for example, which has been practicing threat modeling for quite some time now

Threat modeling is the process of identifying all the parts that make up a system…

With the knowledge of what the parts that make up the system are, and how they operate,

We are then able to compartmentalize them, and control who has access to what
This proactively limits the amount of harm an attacker can inflict on a system

Any feature can be abused.
The gist of this talk is wisdom from Kat Fukui, Senior product designer for communities at GitHub
I greatly admire, and have benefitted from Kat’s work, and you’d be smart to read more of what she’s written

So, we now know that harm happens because of a naive view of technology, that people can exploit systems if sufficiently motivated, and that we need to consider both the systems we work on, and the component pieces they’re made from

In order to fight monsters, we must become monsters
It’s time to be the villain
This is our origin moment, our tragic backstory,
Our chance to seize control and show the world what we’re made of

Threat modeling is great starting point for thinking about how to prevent subverted design
But we’re villains now. Resourceful, relentless, radical
We need to formulate bigger, bolder plans

I think that turning to the Inclusive Design principles is a great way to start
If you are unfamiliar, they are seven principles that, if followed, help to make accessible digital experiences
Remember: It’s not all sunshine and rainbows and kittens, it’s a methodology to take an idea and see how it can facilitate, or hinder access

The first Inclusive Design principle of Providing a Comparable Experience teaches us that content can be accessed in multiple ways
That people can accomplish tasks in a way that suits their needs without undermining the quality of the content
Examples of this include alternate image content and synchronized captions

As villains, we can flip this
My first flipped Inclusive Design Principle is to Force a Single Path
This opens up opportunities for exploitation
For example, needing to rely on someone else to read the content for you requires a huge extension of trust
Think banking, medical records, mental health services, legal information, and all other kinds sensitive data
What happens when this kind of trust gets broken?

The second Inclusive Design Principle of Considering Situation tells us to ensure our content works in a variety of scenarios:
First time and experienced users, outdoors and indoors, wifi and a metered data plan, bright light and low light, etc.

Flipping this, my second inverted principle is to Ignore Circumstance
Demand specific conditions to work, and restrict access until they’re met,
An example of this is requiring someone to be in a specific location in order to work
In controlling the conditions, the system can control how its users behave
This is predatory, with the side-effect of making users more predictable, which in turn makes them more exploitable

The third Inclusive Design principle is to Be Consistent,
To use familiar conventions and apply them consistently
Examples of this are using buttons that look like the buttons on other websites, and links that look like the links on other websites, then making sure they appear consistently on your own site

When I invert it, we Act Capriciously and create sudden, unannounced and unanticipated changes in how our experience works
This allows a bad actor to easily trick and trap the unsuspecting
For example, tricking people into dropping out of queues, misdirecting payment, or sending shock content

Giving Control places people’s needs first, and ensures that they are able to interact with content in their preferred way
Examples of this include the ability to scroll via keyboard, stop animation, and zoom in and out of content

The opposite would be Removing Autonomy
Here we downplay voluntary interaction, doing things like ignoring expressed preferences or automatically triggering actions without consent

The fifth Inclusive Design principle is to Offer Choice
This is providing different ways for people to complete tasks, especially tasks that are complex or non-standard
This principle is reflected with things like individual and bulk actions, expanded and condensed data views, and graph and chart views of tabular data

Its opposite would be to Limit Options, forcing seemingly-arbitrary conditions on your users
Here, you restrict things to a single, narrow view, and user flows that make no external sense

The sixth Inclusive Design principle is to Prioritize Content, to help users focus on core tasks, features, and information
Here, you do things like place the main information prominently, anticipate information before it is needed, and ensure content loads quickly

When I invert it, it becomes Obfuscate Purpose, the act of misdirecting and downplaying the actual goals and consequences of action
Examples of this are hiding fees, legal gotchas, and other schemes

The final Inclusive Design Principle is to Add Value.
This is the act of weighing the positive impact of new and existing features, and how they interrelate
An example of this from the web platform is how new features such as CSS grid can greatly enhance a layout, but still gracefully degrade into a perfectly readable single-column layout for browsers that don’t support the feature

The flipped version of this final principle is to Fill with Gibberish
Hide important and consequential information behind complicated language, jargon, distracting decoration, and the like
A really good example of this is trying to buy a car

To recap, my 7 inverted Inclusive Design principles are:
Force a Single Path, Ignore Circumstance, Act Capriciously,
Remove Autonomy, Limit Options, Obfuscate Purpose, and Fill with Gibberish
Each of the seven inverted Inclusive Design Principles I just described weaken a quality digital experience, and each one presents an opportunity for someone to capitalize on a system to cause harm
Combined, they create a system that is ripe for for subversion
And as villains, we’d do good to remember this when taking another look at the products we work on

So let’s take this out of the abstract and work through an example of how we can identify areas where bad behavior could occur,
And importantly, what can be done about it

It’s a joke and the joke is that it is the truth that on a long enough timeline every product eventually gets a messaging service
In this example, Christina is using a chat widget to ask for help with her service on a cable provider website

The other, sadder joke is that on a long enough timeline messaging services becomes dating apps
Here, the responding agent says “you’re hot, can I call you?”
While he can’t see Christina, he can access her account, which includes information such as her address
He can also search around and find her social media accounts, where she works, and possibly even where she likes to hang out
As to why, maybe he’s bored, or angry, or desparate
You can never know.
But we can know that in this scenario, he’s abusing position, power, and circumstance to put Christina in an incredibly uncomfortable position

When we consider a simplified version of this chat support widget’s user flow, only the happy path is considered:
A customer asks a question and either it is answered, or clarifying information is requested and delivered, leading to an eventual resolution of the issue

However, when we think like a villain, we can identify areas where both the agent and the customer can act inappropriately, and what can be done about it
It’s important to note here that issue resolution may not be the end goal in this circumstance, and may in fact never come into play
A successful user flow in this context can be where harm is removed

All of this translates into user interface decisions

Adding additional functionality for secondary actions allows us to empower the chat widget’s users

Reporting, blocking, and exporting all can help prevent and minimize harm:
Reporting logs the incident and notifies someone charged with reviewing the reports it generates
Blocking breaks contact and programatically prevents this agent from being assigned to this customer in the future, and
Exporting provides an artifact for the user that can persist after the session is ended and the browser tab has been closed

Note that each one of these features represents additional work, especially if things aren’t set up on the backend to accommodate these new changes,
So it is important to identify and codify these features as early in the process as you can
On this slide is a comic panel featuring astronauts frantically repairing a silver rocketship that is already in-flight

Another really important thing you can do is slowly, gently introduce the idea of subverted design to the organization you work for
By which I mean tackling subverted design head-on might not work the way you think it will

People don’t like to hear bad news, especially bad news about things they failed to consider on the things they worked on, and especially bad news about how that might hurt people
And then you factor in things like institutional culture, pressure from management, and toxic positivity.
On this slide is an illustration showing an ostrich with its head buried in the sand
If you don’t ease people into the idea of their work having the potential to create harm, your efforts might blow up in your face
Here’s a Twitter conversation between designers Aaron Z. Lewis and Scott Perket that talks about just that

Aaron says, quote:
In light of the latest FB scandal, here’s my proposal for replacing Design Sprints: Black Mirror Brainstorms
A workshop in which you create a Black Mirror episode. The plot must revolve around misuse of your team’s product.
Pair with @brownorama’s idea of “abusability testing.” Endquote.
Here, Aaron is suggesting a roleplaying exercise where you take an existing feature that people like and try and subvert it
Scott replies, quote:
I frequently asked genuinely earnest questions of teams along these lines while working on ads measurement at Facebook and was considered negative and not a cultural fit.
I did so from a stance of experience. Reminder: blind optimism usually leads to bad outcomes. Endquote
With his reply, Scott warns about how this kind of proposal has been received in the real world

Scott wasn’t wrong despite this alleged negative cultural fit.
Of its many, many issues Facebook has enabled people subverting its ad platform to discriminate based on age, gender, race, religion, and other characteristics
Since Facebook has effectively become digital infrastructure, this means it may prevent access at scale to things like jobs, housing, and credit
This redlining is a repetition of racist segregation policies that occurs in the physical world

You see, easing people into the idea of subverted design is a matter of framing
On this slide I’ve taken the screenshotted conversation between Aaron and Scott and placed a gigantic ornate golden frame around it

Heroes charging in full of righteousness and declaring that someone could abuse something will shut things down
Especially if this accusal is hypothetical
The act immediately challenges personal and organizational ego
And in fact, someone in the organization who has decision-making power could even be actively exploiting the services you’re calling attention to
It’s a bad thought, but remember: we’re the villains now, so we need to think this way

Instead, gather decision makers in a room
The more diverse the better, the more experience and perspectives the better
Once assembled, ask them, “How could we make this feature terrible?”
It’s a fun exercise. The prompt creates a space where it’s safe to come up with ideas about what bad design is without fear of punishment or retaliation

Villains are sneaky.
Start with obvious things like ugly design and bad inputs.
Then maybe float the idea of poor performance and cryptic error messages

Slowly nudge them them towards worse and worse things.
Chances are good they’ll start to move into exploitative behavior territory on their own, but if they don’t, subtly and sneakily suggest it a casual manner to help guide them towards where you want them to go

With all the suggestions written down on sticky notes, have them sort the ideas into groups, and then have them label the groups

And tada! Subversion is now one of a few concerns you’ve all worked together to uncover

Now the dynamic has shifted
You—the manipulative supervillain puppeteer that you are—have just created a superhero team
This new team has defined what evil is, and now knows what that evil will look like—including subversion
With this newfound knowledge the superhero team will now rally around protecting their users and feel good about doing so, because now they’re the ones feeling empowered to set things right
On this slide is a part of a golden age comic showing a 5 man superhero team who have very serious-looking expressions, but who are also ridiculously-dressed

Another really important thing you can do is create a code of conduct for your companies, teams, and projects

Codes of Conduct do a really important job: they outline expectations of what is, and what is not acceptable behavior
This creates a shared understanding of community norms
It also outlines responsibilities, and provides benchmarks for behavioral and ethical standards, as well as methods to report bad behavior in a safe and secure way

On this slide is Multiple Man fighting some superheroes.
Multiple Man is a supervillain that has the ability to instantly create an infinite amount of copies of himself

The Contributor Covenant Code of Conduct is a code of conduct you’ll frequently see on GitHub and many digital projects
The Covenant itself makes for a good starting point, but the problem is it is presented as a turnkey solution
You can click a button to drop a copy of the Covenant into a project without really knowing what it fully is or why it covers what it covers
Each community has its own specific needs, and the Code of Conduct should be manually reviewed and updated to reflect that

The other important bit about Codes of Conduct is adopting one isn’t the same as internalizing it
We all know we don’t read Codes of Conduct, but we really should
And we should encourage others to do so by showing the behavior we want modeled!

Harm is also inevitable
Sadly, it is impossible to predict every possible way a system could be misused
But we can preemptively set up guidelines about what to do when harm does happen
For this, you’ll want an Incident Response Guide

The A11Y Project, a project I proudly help maintain, has one
Much like a Code of Conduct, it creates a shared set of expectations around what to do if an incident occurs
We require each maintainer to read through and sign it, as well as read through a sample scenario,
This helps maintainers understand how abstract guidelines can be applied in a real-life situation.
And this really isn’t about us as maintainers, either
It’s about the community we serve, and prioritizing their health and safety in a fair and equitable way

Speaking of community, one last little bit about villains:
We don’t have to be miserable islands of fear and resentment
We can team up
On this slide is the top half of issue number 2 of Marvel’s Giant-Size Super-villain Team-up, featuring Dr. Doom and the Sub-Mariner

And on this slide is a villain addressing a room full of other villains from a podium
Hear my words, fellow villains: Alone, we may struggle in vain,
Isolated, we may face stigma for daring to speak the truth
But if we band together, we can leave our mark on the world, and show all those fools who dared doubt us
You’ve all bore witness to my plan, now it’s time to take what you’ve learned here today, go forth, and be the villain!

I know this wasn’t exactly the most fun talk, but I think it’s an important one, and hopefully you did to
Thank you for your time and attention, and
Thank you also to the captioning and ASL interpretation services, and to Deque for allowing me to have this great opportunity

The foundation for this talk was sourced from an article I wrote a couple of years ago for 24ways
I’ll have a link to it, as well as links to the other resources I used to make this talk included on the slides that are now live on Notist
I’ll also be linking to this presentation from both my personal website, ericwbailey.design
And from my Twitter account, @ ericwbailey, if you do the Twitter thing
And with that, I’ll happily try to answer any questions you have, thank you!

Eric Bailey
@ericwbailey

1 / 132

Historically, design’s goal was to make things as easy to use for as many people as possible. However, as software matures and takes over more aspects of everyday life, designers must learn to be less naive. In flipping the inclusive design principles, we are able to become the villain and better understand and guard against bad actors who subvert design for their own malign purposes.

Resources

The following resources were mentioned during the presentation or are useful additional information.

Buzz and feedback

Here’s what was said about this presentation on social media.

I am loving this talk. What a wondering way of framing the topic.
— Jamie and Lion (@spacedoutsmiles) March 11, 2021
“On a long enough timeline, every app gets a messaging service. Also on a long enough timeline, all messaging services become dating apps.” -@ericwbailey

Gross gross gross. Technology is good for letting humans act on our worst impulses. Work hard to counter that.
— Eric Wright (@ewaccess) March 11, 2021
The hauntingly long pause @ericwbailey took on the slide of @Facebook's features being used by Myanmar's military to perpetrate ethnic cleansing, genocide, will stick with me for a very long time.

Especially Eric's prompt to 'not look away', not shy away from the fact.
— Shell Little 🧜‍♀️#AxeCon21 (@ShellELittle) March 11, 2021
Wow. This is a very impressive talk. Totally not what I expected. Webdesign meets infosec (kinda).

thanks @ericwbailey !
— Sophie (@sophies_choice) March 11, 2021
This was a real good, slightly painful talk. Watch it if you build anything where UX is a thing. https://t.co/1kzKGDytXr
— Bigotry is bad, actually (@Woebin) March 11, 2021
loving @ericwbailey 's anti-inclusion design patterns!! #axecon @dequesystems

1. force a single path
2. ignore circumstance
3. act capriciously
4. remove autonomy
5. limit options
6. obfuscate purpose
7. fill with gibberish pic.twitter.com/lcq4j8zgRU
— Alex they/them 🤓 (@access_guide_) March 11, 2021
This talk was great and will be available as a recording in the next 48 hours. You can sign up for free to be notified and see other excellent speakers by following that link 👇🏻

Nice work @ericwbailey https://t.co/Yi0tidWmCc
— AT (@goldenmeanie) March 11, 2021
I love feeling included in the presentation. Cannot emphasize enough how important descriptive presentations are when leaning on visual props. Plus1 to the incredible importance of @ericwbailey 's talk is in conjunction with the inclusiveness. Going to rewatch for sure. #axecon
— mike (@GeauxEnder) March 11, 2021
"What could possibly go wrong with what I’m designing?" Great talk by @ericwbailey at #axecon Consider any cause of harm people can do with your product.
— Marie van Driessche (@marievandries) March 11, 2021
This is a presentation I really enjoyed. Be aware Eric gives a content warning at the start. There may be some content triggers.

However, Eric's scenarios have me reconsidering my design decisions. I need to take a closer look at my projects.
— Deborah Edwards-Oñoro (@redcrew) March 11, 2021
.@ericwbailey also brought up the @A11YProject 's Code of Conduct is a good example of anticipating and responding to those causing harm https://t.co/oR9mrG6gKO #axecon @dequesystems
— Alex they/them 🤓 (@access_guide_) March 11, 2021
Thanks to @ericwbailey for an insightful presentation on thinking like a villain when it comes to web design at #axecon. If we don't plan for gaps in our systems, someone will exploit them.
— Genevieve Nelson (@GenevieveNelson) March 11, 2021
Great talk by @ericwbailey on being a villain to understand subversion of designs. As someone who does both web dev and digital security, I can definitely see the relation to threat modelling. Also, learned a lot about making slides and presentations in general #axecon
— Saptak Sengupta (@Saptak013) March 11, 2021
"Abusability testing" that's a keeper.

I think I just about grasped a basic level of understanding about what subverted design is, but will have to follow up more on this.

Thank you Eric for the extremely powerful talk! https://t.co/cGu9grdcqj
— Grunet (@__grunet) March 11, 2021
Overall a fantastic talk from @ericwbailey that reminds us we design real products, for real people, in the real world & it's time to stop sticking our heads in the sand thinking 'humanity will behave' when given opportunities to abuse a system. Think evil to prevent it! #AxeCon
— Shell Little 🧜‍♀️#AxeCon21 (@ShellELittle) March 11, 2021
Just listened to @ericwbailey for the first time today at #axecon: „Be the Villain“ was an excellent talk about how we must start thinking more beyond the happy paths and ideal scenarios and account for ways people will do harm with your products
— Taner (@saryekad) March 11, 2021
We have to think like the villains, the users who will use our products in malicious ways, in order to reduce/prevent harm when designing features. Loved @ericwbailey's talk at #AxeCon https://t.co/EQDxPnV7OL
— Margie (@margarinemargie) March 11, 2021
"We are no longer people who just make pretty things" - @ericwbailey
— Margie (@margarinemargie) March 11, 2021
having a wonderful second day at #axecon, i got to shoutout @epenzeymoog and @CoralineAda for their work that helps fight subversive design and promotes inclusive design during a session titled "Be The Villan" by @ericwbailey
— Kara at #AxeCon (@KaraAJC) March 11, 2021
Inclusive presentation is a skill and an art. When done well, the viewers/listeners aren't noting the relay of visual content as an add-on, but as part of the conversation and presentation format. I love seeing/hearing this in action. Kudos to @ericwbailey https://t.co/SZ8CbrPML9
— Sam Evans, CAE (she/her) (@samspearsevans) March 11, 2021
Another amazing day #axecon! Really appreciated the talks by @ericwbailey and @arhayward. Super proud of my colleagues @JamieKnight and @GarethFW for their awesome talks. And looking forward to @ShellELittle talk. #accessibility people are awesome!
— Emma Pratt Richens (@EmmaJ_PR) March 11, 2021
Hanging out in @ericwbailey talk, “Be the Villian”. Super pumped for this. Also dig the Dr. Doom opening slide. pic.twitter.com/RIbYvUzcTS
— Mark Steadman (@Steady5063) March 11, 2021
"Be The Villain" by @ericwbailey now live at #axecon pic.twitter.com/Z1ZEebBt2b
— de Oostfreese (@deoostfreese) March 11, 2021
I appreciate the content warning at the top of @ericwbailey 's topic. psychological safety is important. #axecon
— mike (@GeauxEnder) March 11, 2021
I think inclusive design is a great framework for a lot of design related things. Things likes equality, #a11y + fairness are important. We need to acknowledge the reason inclusive design exists is because we accidentally or intentionally create the opposite. @ericwbailey #axecon
— Julianna Rowsell (@JuliannaRowsell) March 11, 2021
I didn't expect to hear the phrase "prismatic vomit" at #axecon but boy did @ericwbailey deliver in his talk on #SubvertedDesign.#axecon21 #a11y
— Tori | #BlackLivesMatter #A11y #CymruAmByth (@cymbrygirl) March 11, 2021
@ericwbailey 's descriptions of visual content in their presentation is spot-on. Inclusive presentations amplify the intended message. Thank you. #axecon
— mike (@GeauxEnder) March 11, 2021
Here is what to expect from @ericwbailey today in his session about guarding against subverted design. #axecon pic.twitter.com/vCbdeDNL8y
— Deque Systems (@dequesystems) March 11, 2021
“Subverted design isn't a dark pattern, although they are close cousins. A dark pattern is a technique where a website or app uses a confusing user experience to trick you into doing things you didn't intend to.” – @ericwbailey #AxeCon #A11y #Accessibility
— Todd 🦞 (@toddlibby) March 11, 2021
Stop being naive.

Design has a seat at the table.

We're no longer people who shape pretty things.

We help shape how businesses operate in the world.

We may not be the heroes we thought we were.

~ @ericwbailey

#AxeCon #a11y #accessibility
— Deborah Edwards-Oñoro (@redcrew) March 11, 2021
Inclusive design requires thinking about how people can be left out and discriminated against. It forces us to think about how we can facilitate or hinder access. @ericwbailey #axecon #accessibility
— katiedel (@katiedel) March 11, 2021
.@ericwbailey demonstrating how technology can have unintended, awful uses. #axecon

Makes me think about how Ring doorbells are being used by US police departments to create mass surveillance networks.
— Rachele DiTullio 🐱 (@racheleditullio) March 11, 2021
@ericwbailey's #AxeCon talk "Be the Villain" is critical material for anyone that considers themself to be a designer or technologist.
— Caitlyn Mayers (@caitlynmayers) March 11, 2021
“But design is now a profession that uses esthetic best practices to help business generate profit. The one I would like to focus on for this talk is the vacuum and how one of the many things that can enter into the gap is the higher likelihood for harm.” – @ericwbailey #AxeCon
— Todd 🦞 (@toddlibby) March 11, 2021
Inclusive design principles @ericwbailey mentioned in his axe-con session.

My favorite: Principle 6: Prioritize content

How to: place main info prominently. Ensure content loads quickly.

It's opposite: misdirect the user. Hide legal fees.https://t.co/RwiF8L2sge #AxeCon
— Deborah Edwards-Oñoro (@redcrew) March 11, 2021
Well @ericwbailey is crushing it at #AxeCon during his talk "Be The Villan". Talking about the subjects we prefer to look away from. We need to be villains to try and design around bad behavior, and in some cases, even evil behavior
— Shell Little 🧜‍♀️#AxeCon21 (@ShellELittle) March 11, 2021
@ericwbailey I am 10 mins into your talk and I am in tears. Thank you for your talk today. https://t.co/7wDI6RfTgI
— Chaoyue (Chao) Zhao 😷 (@ChaoyueZ) March 11, 2021

Be the Villain

Link for this presentation:

HTML code for embedding:

Share on social media:

Resources

Be the Villain

Survivorship bias

Equifax Data Breach, Credit Freezes, and Beholden Design

These Dudes Tricked a McDonald’s Self-Order Kiosk Into Giving up Free Burgers

Tweet thread for subverted design links

Designers Finally Have A Seat At The Table. Now What?

Creating Users, Not Addicts

An introduction to approachable threat modeling

Designing for Safety: Principles & Practices

Inclusive Design Principles

Why I don’t use my real photo when messaging with customers on my website

Despicable Design – When “Going Evil” is the Perfect Technique

Guide to writing a code of conduct

A Code of Conduct Is Not Enough

Using Dark Patterns for Good

Consensual Software

Subverted Design

Technically Wrong: Sexist Apps, Biased Algorithms, and Other Threats of Toxic Tech

The Internet of Garbage

Using Dark Patterns for Good

Buzz and feedback