A presentation at Design & Content in in Vancouver, BC, Canada by Lauren Isaacson
1 Boosting Your Bias Immunity LAUREN ISAACSON Market & UX Research Consultant Curio Research @curio_research www.curioresearch.net
Character Reference 2 You’re who? You do what? @curio_research Lawful Good Neutral Good ChaoAc Good Lawful Neutral True Neutral ChaoAc Neutral Lawful Evil Neutral Evil ChaoAc Evil www.curioresearch.net
Character Reference 3 You’re who? You do what? @curio_research Lawful Good Neutral Good ChaoAc Good Lawful Neutral True Neutral ChaoAc Neutral Lawful Evil Neutral Evil ChaoAc Evil www.curioresearch.net
Research Polymath 4 Numerating and Talking Quantitative Research Understanding what and how much @curio_research Types of Primary Research Qualitative Research Understanding what and why www.curioresearch.net
5 WHAT IS BIAS WHY WE ARE BIASED AVOIDING BIASED QUALITATIVE RESEARCH AVOIDING BIASED QUANTITATIVE RESEARCH AVOIDING BIASED DECISIONS @curio_research www.curioresearch.net
6 What Is Bias? Seeing is not necessarily believing @curio_research www.curioresearch.net
7 “A cognitive bias refers to the systematic pattern of deviation from the norm or rationality in judgement, whereby inferences about other people and situations may be drawn in an illogical fashion. Individuals create their own ‘subjective social reality’ from their perception of input.” —Wikipedia @curio_research Photo by Ayo Ogunseinde on Unsplash www.curioresearch.net
Thinking Fast and Slow 8 The foundation of behavioural economics Intuition s l l i k S e t na In Effortlessness System 1 Speed Reflex Involuntary Control @curio_research Patience Concentration An aly tic s System 2 Reason Effort Constructed Thoughts www.curioresearch.net
Photo by Daniel Tafjord on Unsplash 9 Move Fast and Break Things! And Fix Them Only If It’s In Our Financial Best Interest @curio_research www.curioresearch.net
Disrupting Hotels 10 When is a house not a home? Sharing Living Spaces Allow people to rent out spare bedrooms or entire homes to tourists. NEGATIVE IDEA Income & Ease of Travel Hosts make extra money and travellers can stay somewhere cheaper and often more authentic than a hotel. Win Win. POSITIVE Mass Gentrification What happens when a place to live has more economic value as a place for tourists to stay? @curio_research www.curioresearch.net
Disrupting Urban Transit 11 When taxis compete with mass transit, who wins? On-Demand Ride Hailing Turning under-utilized vehicles into income generators. Bringing mobile tech to the often frustrating taxi experience. NEGATIVE IDEA New Income & Ease of Intercity Movement Drivers have a new source of extra or primary income. Riders can get from place to place with the simplicity of a well designed app. POSITIVE Increased Traffic & Imperfect AI More single person car trips for higher traffic and letting AI choose between killing pedestrians or riders @curio_research www.curioresearch.net
Disrupting Newspaper Ads 12 No local ads means no local journalism Offer Free/Cheap Ads Offer online ads and means of communication for people and businesses to communicate directly with each other. NEGATIVE IDEA Peer2Peer & SMB Ads Get Direct It’s easier for individuals to advertise to others. SMBs can target and communicate with their customers directly. POSITIVE Media Atrophy & Distrust Local journalism is no longer viable. The public distrusts a media they no longer interact with. @curio_research www.curioresearch.net
Disrupting Justice 13 Do robots get hangry? Objective Robot Judges Use prior sentencing data to train computers to make objective judgements in criminal cases. IDEA Objective Consistency Judgements don’t fluctuate with the time of day or whether the judge just ate. Machines don’t bring emotion into decisions. NEGATIVE POSITIVE Uneven Fairness Baseline The system is defined as fair, but the definition of what is fair differs. We see the system the algorithm is based off as fair, but what if it’s not? @curio_research www.curioresearch.net
14 Why Are We Biased To protect and serve @curio_research www.curioresearch.net
Primary Sources of Bias 15 Efficiency and Defence Self Protection Avoid bodily and ego harm @curio_research Efficient Decision Making System 1 www.curioresearch.net
16 Bias is like polluted air • Impossible to avoid, but filterable. • Filtering requires identification of potential influencing biases. @curio_research www.curioresearch.net
17 How researchers leverage bias Sometimes bias serves a purpose. For researchers, it can maximize the ratio of insights to data. @curio_research • We can’t collect and analyze all data • from everywhere. We intentionally bias our methodology, our sample selection, and the analysis to get most insight value from our research. www.curioresearch.net
18 Types of Bias Name Thy Enemies @curio_research www.curioresearch.net \
The World of Biases 19 Using bias to understand bias There are over 170 types of cognitive biases @curio_research www.curioresearch.net
The World of Biases 20 Using bias to understand bias There are over 170 types of cognitive biases ABUNDANT INFORMATION Selective filtering We can’t take it all in @curio_research LIMITED CONTEXT We can’t know everything, so we fill in the gaps LIMITED TIME LIMITED MEMORY We need to act fast We can’t remember everything, so what sticks? www.curioresearch.net
Abundant Information 21 Selective filtering, we can’t take it all in AVAILABILITY HEURISTIC We notice what is already primed in our brains from repeated past experience ANCHORING The order receive information influences our judgement CONFIRMATION BIAS We tend to notice and agree with information that fits what we already believe BIZARRENESS EFFECT @curio_research We tend to have better memory for things that are odd than we do things that are common www.curioresearch.net
Limited Context 22 We can’t know everything, so we fill in the gaps FRAMING EFFECT We judge items and subjects by how they are presented PLACEBO EFFECT Believing something works can be just as powerful as something that actually works FUNDAMENTAL ATTRIBUTION ERROR We judge others by what we see in the moment, but we judge ourselves based on the situation IN-GROUP BIAS @curio_research We favour people who belong to the same group as ourselves www.curioresearch.net
Limited Time 23 We need to act fast OPTIMISM BIAS We overestimate the probability of a positive outcome BARNUM EFFECT We connect things leaps through leaps of imagination because we don’t have all the facts DUNNING-KRUGER When you’re not an expert, there is a tendency to think layperson’s knowledge is all there is GROUPTHINK/BANDWAGON EFFECT @curio_research Opinions are based on fitting in with the group rather than evidence. www.curioresearch.net
Limited Memory 24 We can’t remember everything, so what sticks NEGATIVITY BIAS Negative events and feelings stay with us longer than positive ones SPACING EFFECT We learn better in bits and chunks over time rather than all at once IMPLICIT STEREOTYPE Learned associations between various qualities and social categories PEAK-END RULE @curio_research People judge an experience based in how they felt at its most intense point rather than overall www.curioresearch.net
25 Avoiding Biased Qualitative Research That’s interesting. Tell me more about that. @curio_research www.curioresearch.net
26 Confirmation Bias • Don’t test your own designs. • Involve outsiders in the discussion guide approval process. • Hire research agency or contractor for the project. @curio_research www.curioresearch.net
27 Research Bias • Try to begin every questions with Who, What, When, Where, Why, and How • Avoid Should, Would, Is, Are and Do You Think, unless you plan to have a follow up. • Let the silence sit there. @curio_research www.curioresearch.net
28 Anchoring Heuristic • Randomize the order of test stimuli • Don’t offer multiple choice questions • Don’t set baselines within questions. @curio_research www.curioresearch.net
29 Availability Heuristic • Debrief. Debrief. Debrief. Make sure stakeholders are seeing the same things you are. • Get transcripts of your interviews and analyze them thoroughly. @curio_research www.curioresearch.net
30 Interpretive Bias • Restate ambiguous answers in your own words, but make sure the participant knows they have permission to contradict you. • Don’t let ignorance got in the way of understanding. Ask for clarification. @curio_research www.curioresearch.net
Photo by Caleb Woods on Unsplash 31 Be Self Critical, But Not Self Flagellating We ’ r e a l l i n a c o n s t a n t s t a t e o f l e a r n i n g a n d i m p r o v e m e n t @curio_research www.curioresearch.net
32 Avoiding Biased Quantitative Research Random probability sampling @curio_research www.curioresearch.net
@curio_research Yes/No Agree/Disagree True/False www.curioresearch.net
Get the by monitoring data in the moment or from a neutral third party @curio_research www.curioresearch.net
Random number generators Dice • Use quotas to make the sample reflect the population. @curio_research www.curioresearch.net
36 Research Bias • Word questions carefully. - Avoid leading, framing, and telegraphing • Use red herring questions to increase your data quality. @curio_research www.curioresearch.net
37 Example Time! Debiasing a Question Photo by Ben White on Unsplash www.curioresearch.net
Debiasing a Question 38 What is your opinion on the following statement: Jean Luc Picard was (will be) the best captain in Starfleet. Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree @curio_research www.curioresearch.net
Debiasing a Question 39 What is your opinion on the following statement: Jean Luc Picard was (will be) the best captain in Starfleet. Strongly agree Agree Neither agree nor disagree Disagree Strongly disagree @curio_research www.curioresearch.net
Debiasing a Question 40 Who was (will be) the best captain in Starfleet? James T. Kirk John Archer Benjamin Sisko Kathryn Janeway Jean Luc Picard @curio_research www.curioresearch.net
Debiasing a Question 41 Who was (will be) the best captain in Starfleet? John Archer Kathryn Janeway Jean Luc Picard Randomize James T. Kirk Benjamin Sisko @curio_research www.curioresearch.net
Debiasing a Question 42 Who was (will be) the best captain in Starfleet? John Archer Kathryn Janeway Jean Luc Picard James T. Kirk Han Solo Benjamin Sisko @curio_research www.curioresearch.net
Debiasing a Question 43 Who was (will be) the best captain in Starfleet? John Archer Kathryn Janeway Jean Luc Picard James T. Kirk Han Solo Benjamin Sisko @curio_research www.curioresearch.net
Photo by David Travis on Unsplash 44 TEST YOUR SURVEYS A s u r v e y i s a d e s i g n e d i n t e r f a c e , m a ke s u r e i t w o r k s @curio_research www.curioresearch.net
45 Avoiding Biased Decisions Move at a steady pace and make good choices @curio_research www.curioresearch.net
46 @curio_research Photo by Vidar Nordli-Mathisen on Unsplash www.curioresearch.net
47 REMEMBER Bias is unavoidable, but it is manageable @curio_research www.curioresearch.net
48 Think Like a Researcher Disprove your hypothesis @curio_research Photo by João Silas on Unsplash www.curioresearch.net
Process Matters 49 I love it when a plan comes together Slow Down @curio_research Bias Identification Bias Avoidance Reevaluation www.curioresearch.net
50 B i a s e s A r e Av o i d e d o n P u r p o s e , Not By Accident @curio_research www.curioresearch.net
Take Good Notes 51 It works for Comey, it will work for you Write it down or it never happened Get agreement on goals What is the understanding of risk? Revisit after the decision is in action @curio_research www.curioresearch.net
Don’t Try to Control Everything 52 How do you reduce individual or group influence? Secret Ballots Outside Input Uncomfortable Questions Individual Opinions Reverse Brainstorming @curio_research No HiPPOs www.curioresearch.net
53 If we are all in agreement on the decision - then I propose we postpone further discussion of this matter until our next meeting to give ourselves time to develop disagreement and perhaps gain some understanding of what the decision is all about. A L F R E D P. S LOA N CEO of GM @curio_research www.curioresearch.net
54 Hypotheses were made to be disproved •Researchers don’t try to prove their theories. They try to disprove them. •If an idea can withstand counter evidence, it’s probably a good idea. @curio_research www.curioresearch.net
Think Like a Researcher 55 Yes, take a photo of this slide What are the biases at play? Abundant Information Limited Context Limited Time Limited Memory @curio_research Have a process Take good notes Slow your roll Identify potential biases Avoid biases Revaluate Record consensuses Revisit after implementation Be open to disagreement Break echo chambers Strong ideas withstand contrary arguments www.curioresearch.net
56 Parting Thought Just World Hypothesis @curio_research www.curioresearch.net
57 @curio_research www.curioresearch.net Photo by NeONBRAND on Unsplash
58 @curio_research www.curioresearch.net Photo by Søren Astrup Jørgensen on Unsplash
Photo by “My Life Through A Lens” on Unsplash 59 @curio_research www.curioresearch.net
60 THANK YOU LAUREN ISAACSON @CURIO_RESEARCH WWW.CURIORESEARCH.NET @curio_research www.curioresearch.net
Bias is everywhere. It affects design choices, product decisions, whom we hire, how we live. It’s like polluted air - impossible to avoid, but possible to identify and filter. By becoming more aware of different types of bias, we can make better decisions about the qualitative and quantitative research we conduct and analyze. Learn how researchers can avoid different biases and how everyone can avoid biased decision making.
Here’s what was said about this presentation on social media.
for free. You
can too.