Art vs Science

A presentation at Web Directions Design in April 2019 in Melbourne VIC, Australia by Laura Summers

In tech in recent years, it feels a bit like everybody and their dog has hopped on the ‘scientific method’ bandwagon. Design Thinking, Agile and UX Research methodologies are all modelled on this approach. But science itself is in the grips of an existential crisis, as embodied by a series of failed attempts to reproduce existing research. (See ‘The Reproducibility Project’ for context). Against this backdrop, it’s a prescient time to ask the question: should UX Research be reproducible?

Resources

The following resources were mentioned during the presentation or are useful additional information.

Buzz and feedback

Here’s what was said about this presentation on social media.

If you want to read more, slides are here: https://t.co/v9Egnxs9KV
(Highly recommend checking out the John Oliver talk on science reporting even if you don't want to read any of the heavy duty stuff). Thanks #design19 for being such a great audience!! 🙌
— Laura Summers (@summerscope) April 11, 2019
Laura Summers @summerscope giving us a thought provoking talk and asking "Does reproducibility matter in UXR?" #design19 pic.twitter.com/JVv6xD0dfq
— LadiesThatUX / Melb (@LadiesThatUXMEL) April 11, 2019
Timely advice not just for designers and researchers, but anyone consuming news media today: always fact check! @summerscope #design19 pic.twitter.com/zbts78P76U
— Rachael Mullins (@rachaelamullins) April 11, 2019
Pick a good question and it can be answered in many ways! great insights by @summerscope #design19 #melbourne #webdirections #UXResearch #firsttweetever pic.twitter.com/vPDYZ58s3b
— sunethra de mel (@SunethraDe) April 11, 2019
Share your code, share your research@summerscope #Design19 pic.twitter.com/iN7uumW4f6
— Chris Lienert (@cliener) April 11, 2019
Replications studies in UX research are important to maintain understanding about the market and biz model for an org’s long term health - @summerscope #design19
— Jason Pamental (@jpamental) April 11, 2019
Interesting question for me coming from a science background: Should #UXResearch be reproducible? In science it's a hard yes, but in UXR @summerscope says sometimes(!) depending on whether it's qual or quant research 🤷🏻‍♀️#design19
— Vivian Li (@vivzli) April 11, 2019
Reward your people for saying:
“I don’t know”.
Especially when they are higher up inside the company.

This is so important. I can’t agree more via @summerscope #design19
— Dominik Wilkowski (@wilkowskidom) April 11, 2019
“Reward ‘I don’t know’” @summerscope
All the more important for seniors/leaders to admit when they don’t know something#Design19
— Chris Lienert (@cliener) April 11, 2019
Here's the thread on outdated/debunked scientific ideas that @summerscope mentioned in her talk on UX research in the age of the reproducibility crisis. #design19 https://t.co/on6SKwiaZB
— Rachael Mullins (@rachaelamullins) April 11, 2019
Usability testing tells us if a user can use the product/interface, not whether they want to or if they like it. It's wrong to use the method to try and ascertain market desirability. @summerscope #design19
— Katja Forbes (@luckykat) April 11, 2019
“Myers-Briggs is astrology for people who should be too smart for astrology.” @summerscope #design19 😂
— crazy witch asian (@jina) April 11, 2019
“Do we need open UX? Let’s all look at the data & results across dif orgs and industries?” Super interesting notion from @summerscope #design19
— Jason Pamental (@jpamental) April 11, 2019
"are we doing enough to double check our results?" - I'm so here for this excellent talk on the reproducibility crisis, bias, bad science & replication studies by @summerscope 💪 #Design19 pic.twitter.com/Dj848ul2c8
— Patima™ 💁🏻✨ (@the_patima) April 11, 2019
Brilliant: “Myers-Briggs is like astrology for people who should be too smart for astrology” @summerscope #design19
— Jason Pamental (@jpamental) April 11, 2019