From Terminator to Star Trek… Using AI in day to day interfaces

From Terminator to Star Trek… Using AI in day to day interface s Chris Heilmann (@codepo8) June 2018

All resources: aka.ms/human

ai @codepo8

Let’s talk about “Artificial Intelligence” @codepo8

Artificial Intelligence ▪ Is nothing new – the concepts go back to the 50ies @codepo8

Artificial Intelligence ▪ Is nothing new – the concepts go back to the 50ies ▪ Is quite the hype and ver y often misattributed @codepo8

Artificial Intelligence ▪ Is nothing new – the concepts go back to the 50ies ▪ Is quite the hype and ver y often misattributed ▪ Is an umbrella term for a lot of math and science around repetition, pattern recognition and machine learning @codepo8

Artificial Intelligence ▪ Is nothing new – the concepts go back to the 50ies ▪ Is quite the hype and ver y often misattributed ▪ Is an umbrella term for a lot of math and science around repetition, pattern recognition and machine learning ▪ Got a huge boost because of availability of hardware @codepo8

Florian Ziegler flickr.com/photos/damndirty/41263240134 The machines are watching…

Big brother is redundant… ▪ Ever ything we do online is monitored and recorded @codepo8

Big brother is redundant… ▪ Ever ything we do online is monitored and recorded ▪ We often don’t realise that our data is how we pay for “free” ser vices @codepo8

Big brother is redundant… ▪ Ever ything we do online is monitored and recorded ▪ We often don’t realise that our data is how we pay for “free” ser vices ▪ We’re happy to use systems that record all the time in exchange for convenience @codepo8

Big brother is redundant… ▪ Ever ything we do online is monitored and recorded ▪ We often don’t realise that our data is how we pay for “free” ser vices ▪ We’re happy to use systems that record all the time in exchange for convenience ▪ Often people don’t realise just how dangerous this can be in the wrong hands . @codepo8

Leaving invisible marks… @codepo8

Leaving invisible marks… ▪ By using other people’s machines and infrastructure , we leave traces @codepo8

Leaving invisible marks… ▪ By using other people’s machines and infrastructure , we leave traces ▪ This allows companies to recognise us, and accumulates a usage histor y @codepo8

Leaving invisible marks… ▪ By using other people’s machines and infrastructure , we leave traces ▪ This allows companies to recognise us, and accumulates a usage histor y ▪ This leads to better results , but can leaks data @codepo8

Leaving invisible marks… ▪ By using other people’s machines and infrastructure , we leave traces ▪ This allows companies to recognise us, and accumulates a usage histor y ▪ This leads to better results , but can leaks data ▪ We should have more transparency about what digital legacy we left behind. @codepo8

Ever ything counts in large amounts @codepo8

Ever ything counts in large amounts ▪ We create a massive amount of information – actively and without our knowledge. @codepo8

Ever ything counts in large amounts ▪ We create a massive amount of information – actively and without our knowledge. ▪ It is tough to make that amount of information consumable again. @codepo8

Ever ything counts in large amounts ▪ We create a massive amount of information – actively and without our knowledge. ▪ It is tough to make that amount of information consumable again. ▪ That’s why we have computers @codepo8

Ever ything counts in large amounts ▪ We create a massive amount of information – actively and without our knowledge. ▪ It is tough to make that amount of information consumable again. ▪ That’s why we have computers ▪ With cloud computing, on demand processing and advances in hardware we’re faster than ever . @codepo8

Florian Ziegler flickr.com/photos/damndirty/40153024740/ Are machines friend or foe?

Artificial Intelligence Myths ▪ AI can’t replace a thinking, creative human @codepo8

Artificial Intelligence Myths ▪ AI can’t replace a thinking, creative human ▪ AI can not magically fill gaps with perfect information – it can only compare and assume @codepo8

Artificial Intelligence Myths ▪ AI can’t replace a thinking, creative human ▪ AI can not magically fill gaps with perfect information – it can only compare and assume ▪ AI doesn’t learn in a creative fashion. It does not transfer. @codepo8

Artificial Intelligence Myths ▪ AI can’t replace a thinking, creative human ▪ AI can not magically fill gaps with perfect information – it can only compare and assume ▪ AI doesn’t learn in a creative fashion. It does not transfer. ▪ AI has no morals and ethics , but – used wrongly – it can amplify our biases @codepo8

Machines can be great tools or weapons… ▪ Machine Learning is all about returning assumptions @codepo8

Machines can be great tools or weapons… ▪ Machine Learning is all about returning assumptions ▪ We don’t get any definitive truth from algorithms , we get answers to our questions @codepo8

Machines can be great tools or weapons… ▪ Machine Learning is all about returning assumptions ▪ We don’t get any definitive truth from algorithms , we get answers to our questions ▪ AI can answer questions, but it is up to you to ask good questions – generic questions yield assumed results. @codepo8

Machines can be great tools or weapons… ▪ Untrained and limited data leads to terrible and biased AI results @codepo8

Machines can be great tools or weapons… ▪ Untrained and limited data leads to terrible and biased AI results ▪ It is ver y easy to get either wrong deductions or false positives @codepo8

Machines can be great tools or weapons… ▪ Untrained and limited data leads to terrible and biased AI results ▪ It is ver y easy to get either wrong deductions or false positives ▪ AI is as intelligent and good as the people who apply it @codepo8

About face… aka.ms/face

api @codepo8

About face… aka.ms/face

api ▪ Face rectangle / Landmarks ▪ Pose (pitch/roll/yaw) ▪ Smile ▪ Gender/Age ▪ Type of glasses ▪ Makeup (lips/eye) ▪ Emotion (anger, contempt, disgust, fear, happiness, neutral, sadness, surprise) ▪ Occlusion (forehead/eye/mouth) ▪ Facial hair (moustache/beard/sideburns) ▪ Attributes: Hair (invisible, bald, colour ) @codepo8

Is this you? Are those also you? aka.ms/face

api @codepo8

Is this your driver? youtube.com/watch?v=aEBi4OpXU4Q @codepo8

Taking it too far? ntechlab.com @codepo8

Photo by Florian Ziegler flickr.com/photos/damndirty/40153024740/ Andreas Dantz flickr.com/photos/szene/40193567250 AI for humans

How AI can help humans… aka.ms/ai

for

good @codepo8

Good interfaces allow for fuzzy entries. ▪ Is nothing new – the concepts go back to the 50ies @codepo8

Bots and computers… Humans ▪ Messy and prone to mistakes @codepo8

Bots and computers… Humans ▪ Messy and prone to mistakes ▪ Make no mistakes, other than physical fatigue @codepo8

Bots and computers… Humans ▪ Messy and prone to mistakes ▪ Forget things and filter them by their biases ▪ Make no mistakes, other than physical fatigue @codepo8

Bots and computers… Humans ▪ Messy and prone to mistakes ▪ Forget things and filter them by their biases ▪ Make no mistakes, other than physical fatigue ▪ Never forget, don’t judge @codepo8

Bots and computers… Humans ▪ Messy and prone to mistakes ▪ Forget things and filter them by their biases ▪ Bored when doing repetitive tasks ▪ Make no mistakes, other than physical fatigue ▪ Never forget, don’t judge @codepo8

Bots and computers… Humans ▪ Messy and prone to mistakes ▪ Forget things and filter them by their biases ▪ Bored when doing repetitive tasks ▪ Make no mistakes, other than physical fatigue ▪ Never forget, don’t judge ▪ Great at tedious, boring tasks @codepo8

Bots and computers… Humans ▪ Messy and prone to mistakes ▪ Forget things and filter them by their biases ▪ Bored when doing repetitive tasks ▪ When bored create more errors ▪ Make no mistakes, other than physical fatigue ▪ Never forget, don’t judge ▪ Great at tedious, boring tasks @codepo8

Bots and computers… Humans ▪ Messy and prone to mistakes ▪ Forget things and filter them by their biases ▪ Bored when doing repetitive tasks ▪ When bored create more errors ▪ Make no mistakes, other than physical fatigue ▪ Never forget, don’t judge ▪ Great at tedious, boring tasks ▪ Repeat things with minor changes on iterations till a result is met @codepo8

Bots and computers… Humans ▪ Messy and prone to mistakes ▪ Forget things and filter them by their biases ▪ Bored when doing repetitive tasks ▪ When bored create more errors ▪ Non

optimised communication, lots of nuances and misunderstanding ▪ Make no mistakes, other than physical fatigue ▪ Never forget, don’t judge ▪ Great at tedious, boring tasks ▪ Repeat things with minor changes on iterations till a result is met @codepo8

Bots and computers… Humans ▪ Messy and prone to mistakes ▪ Forget things and filter them by their biases ▪ Bored when doing repetitive tasks ▪ When bored create more errors ▪ Non

optimised communication, lots of nuances and misunderstanding ▪ Make no mistakes, other than physical fatigue ▪ Never forget, don’t judge ▪ Great at tedious, boring tasks ▪ Repeat things with minor changes on iterations till a result is met ▪ Highly optimised , non

nuanced communication. @codepo8

Bots and computers… Humans ▪ Messy and prone to mistakes ▪ Forget things and filter them by their biases ▪ Bored when doing repetitive tasks ▪ When bored create more errors ▪ Non

optimised communication, lots of nuances and misunderstanding ▪ Make no mistakes, other than physical fatigue ▪ Never forget, don’t judge ▪ Great at tedious, boring tasks ▪ Repeat things with minor changes on iterations till a result is met ▪ Highly optimised , non

nuanced communication. @codepo8

Bots and computers… Humans ▪ Messy and prone to mistakes ▪ Forget things and filter them by their biases ▪ Bored when doing repetitive tasks ▪ When bored create more errors ▪ Non

optimised communication, lots of nuances and misunderstanding ▪ Make no mistakes, other than physical fatigue ▪ Never forget, don’t judge ▪ Great at tedious, boring tasks ▪ Repeat things with minor changes on iterations till a result is met ▪ Highly optimised , non

nuanced communication. Data Insights Patterns @codepo8

Humans and Bots/Computers autodraw.com @codepo8

Humans and Bots/Computers quickdraw.withgoogle.com @codepo8

Humans and Bots/Computers google.com/recaptcha/intro @codepo8

Intelligent, responsive systems ▪ AI ser vices offer us lots of data to compare our users’ input with Google : cloud.google.com/products/machine

learning Amazon : aws.amazon.com/machine

learning Microsoft : azure.microsoft.com/en

us/ser vices/cognitive

ser vices @codepo8

Intelligent, responsive systems ▪ AI ser vices offer us lots of data to compare our users’ input with ▪ Thus our users don’t need to speak computer but be human instead Google : cloud.google.com/products/machine

learning Amazon : aws.amazon.com/machine

learning Microsoft : azure.microsoft.com/en

us/ser vices/cognitive

ser vices @codepo8

Intelligent, responsive systems ▪ AI ser vices offer us lots of data to compare our users’ input with ▪ Thus our users don’t need to speak computer but be human instead ▪ We can prevent them from making mistakes Google : cloud.google.com/products/machine

learning Amazon : aws.amazon.com/machine

learning Microsoft : azure.microsoft.com/en

us/ser vices/cognitive

ser vices @codepo8

Intelligent, responsive systems ▪ AI ser vices offer us lots of data to compare our users’ input with ▪ Thus our users don’t need to speak computer but be human instead ▪ We can prevent them from making mistakes ▪ We can help getting around physical barriers Google : cloud.google.com/products/machine

learning Amazon : aws.amazon.com/machine

learning Microsoft : azure.microsoft.com/en

us/ser vices/cognitive

ser vices @codepo8

Our toolkit for more human interfaces Natural language processing Computer Vision Sentiment analysis Speech conversion and analysis Moderation @codepo8

Language and Writing @codepo8

Language and Writing ▪ Probably the oldest task on the web was translation @codepo8

Language and Writing ▪ Probably the oldest task on the web was translation ▪ This moved deeper into Natural Language Processing and Language Detection @codepo8

Language and Writing ▪ Probably the oldest task on the web was translation ▪ This moved deeper into Natural Language Processing and Language Detection ▪ Using these, we can allow for human commands and finding out tasks by analyzing texts. @codepo8

Language and Writing ▪ Probably the oldest task on the web was translation ▪ This moved deeper into Natural Language Processing and Language Detection ▪ Using these, we can allow for human commands and finding out tasks by analyzing texts. “How far am I from the capital of Denmark?” “Where do I find a good restaurant around here?” “Show me documents I wrote five days ago with more than 600 words” @codepo8

Computer Vision @codepo8

Computer Vision ▪ When text wasn’t cool enough, we added images to our web media @codepo8

Computer Vision ▪ When text wasn’t cool enough, we added images to our web media ▪ Often we forget that not ever yone can see them , and we leave them without alternative text @codepo8

Computer Vision ▪ When text wasn’t cool enough, we added images to our web media ▪ Often we forget that not ever yone can see them , and we leave them without alternative text ▪ This is where machine learning steps in to help turning an image into a dataset we can work with. @codepo8

Vision and image analysis… instagram : @ larr yandanke @codepo8

Vision and image analysis… @codepo8

Vision and image analysis… twitter.com/mixedhunty/status/980551155297157126 @codepo8

Vision and image analysis… #vision_api @codepo8

Vision and image analysis… aka.ms/vision

api @codepo8

Vision and image analysis… aka.ms/vision

api @codepo8

Vision and image analysis… aka.ms/vision

api @codepo8

Vision and image analysis… aka.ms/vision

api @codepo8

Sentiment analysis @codepo8

Sentiment analysis ▪ Finding out the sentiment of a text, image or video can help with a lot of things @codepo8

Sentiment analysis ▪ Finding out the sentiment of a text, image or video can help with a lot of things ▪ You can navigate videos by only showing the happy parts @codepo8

Sentiment analysis ▪ Finding out the sentiment of a text, image or video can help with a lot of things ▪ You can navigate videos by only showing the happy parts ▪ You can detect which comment should be answered first by a help desk @codepo8

Sentiment analysis ▪ Finding out the sentiment of a text, image or video can help with a lot of things ▪ You can navigate videos by only showing the happy parts ▪ You can detect which comment should be answered first by a help desk ▪ You can predict when drivers of cars get tired @codepo8

Sentiment analysis github.com/noopkat/face

api

emoji

face @codepo8

Speech @codepo8

Speech ▪ Audio interfaces are all the rage . @codepo8

Speech ▪ Audio interfaces are all the rage . ▪ You can allow hands

free control of devices @codepo8

Speech ▪ Audio interfaces are all the rage . ▪ You can allow hands

free control of devices ▪ You can have an “always on” system to help you out without having to interface with it @codepo8

Speech ▪ Audio interfaces are all the rage . ▪ You can allow hands

free control of devices ▪ You can have an “always on” system to help you out without having to interface with it ▪ It feels natural and has a massive Sci

Fi feeling – when it works. @codepo8

Text to speech aka.ms/text

to

speech @codepo8

Speech recognition aka.ms/text

to

speech @codepo8

Turning sentences into commands aka.ms/luis

api luis.ai @codepo8

Conversation as an interface aka.ms/conversation

ui @codepo8

Speaker recognition aka.ms/speaker

recognition @codepo8

Speaker recognition aka.ms/speaker

recognition @codepo8

Moderation @codepo8

Moderation ▪ Some things are not meant to be consumed by people @codepo8

Moderation ▪ Some things are not meant to be consumed by people ▪ Computers don’t need counselling once they saw them – people should @codepo8

Moderation ▪ Some things are not meant to be consumed by people ▪ Computers don’t need counselling once they saw them – people should ▪ Known illegal and terrible content can be automatically removed @codepo8

With great power comes great responsibility… @codepo8

Our responsibilities.. ▪ AI can be an amazing help for humans @codepo8

Our responsibilities.. ▪ AI can be an amazing help for humans ▪ It does need transparency – if you use people as data sources, they need to know what and where it goes @codepo8

Our responsibilities.. ▪ AI can be an amazing help for humans ▪ It does need transparency – if you use people as data sources, they need to know what and where it goes ▪ When people get information filtered by an algorithm , it should be an opt

in @codepo8

Our responsibilities.. ▪ AI can be an amazing help for humans ▪ It does need transparency – if you use people as data sources, they need to know what and where it goes ▪ When people get information filtered by an algorithm , it should be an opt

in ▪ People need to have a chance to dispute when an algorithm tagged or disallowed them access. @codepo8

Want to go deep? ▪ The Math behind ML ▪ The ethics of AI ▪ Working with Data using Python ▪ Machine Learning Models ▪ Deep Learning Models ▪ Reinforcement Learning Models ▪ Microsoft Professional Program Certificate in Artificial Intelligence aka.ms/learn

ai 10 courses, (8

16 hours each), 10 skills @codepo8

Thanks! Chris Heilmann Christianheilmann.com Developer

evangelism.com @codepo8