Welcome to Azure Saturday 2019 Munich #AzureSaturday 18.05.2019 – Microsoft Munich – azuresaturday.de — @azuresaturday
A presentation at Azure Saturday Munich 2019 in May 2019 in Munich, Germany by Chris Heilmann
Welcome to Azure Saturday 2019 Munich #AzureSaturday 18.05.2019 – Microsoft Munich – azuresaturday.de — @azuresaturday
#AzureSaturday 18.05.2019 – Microsoft Munich – azuresaturday.de – @azuresaturday Building human interfaces powered by AI Speaker: Chris Heilmann
Building human interfaces powered by AI Chris Heilmann (@codepo8) November 2018
All resources: aka.ms/human-ai @codepo8
Let’s talk about “Artificial Intelligence” @codepo8
What is the difference between Machine Learning and Artificial Intelligence? @codepo8
Machine Learning is written in Python, JavaScript… Artificial Intelligence is written in PowerPoint. @codepo8
Artificial Intelligence @codepo8 ▪ Is nothing new – the concepts go back to the 50ies ▪ Is quite the hype and very often misattributed ▪ Is an umbrella term for a lot of math and science around repetition, pattern recognition and machine learning ▪ Got a huge boost because of availability of hardware ▪ Became much more feasible because of the availability of lots of data
Reminders of “genie in the bottle” @codepo8 ▪ Fulfills our wishes seemingly with invisible magic ▪ Useful, and feels too good to be true ▪ Once released, may have a dark, sinister edge to it ▪ Hard to put back into the bottle.
Let’s start with some predictions. @codepo8 ▪ AI is the number one growth market in IT – the others are cloud and security ▪ Machine Learning is already replacing thousands of jobs – boring, terrible jobs humans should not do ▪ This is also happening in IT – we are not invincible because we know hot to exit Vim
Let’s start with some predictions. @codepo8 ▪ There is no stopping this – it is just too convenient ▪ The amount of data we create (actively or by triggering sensors) demands machines to whittle it down for us to make it consumable by humans ▪ If we as developers and decision makers in IT don’t take ownership and lead with good, ethical examples, we’ll throw away decades of work democratising computing
The machines are watching… Florian Ziegler flickr.com/photos/damndirty/41263240134
Social Credit System @codepo8 https://futurism.com/china-social-credit-system-rate-human-value/
Big brother is redundant… @codepo8 ▪ Everything we do online is monitored and recorded ▪ We often don’t realise that our data is how we pay for “free” services ▪ We’re happy to use systems that record all the time in exchange for convenience ▪ Often people don’t realise just how dangerous this can be in the wrong hands.
Everything counts in large amounts @codepo8 ▪ We create a massive amount of information – actively and without our knowledge. ▪ It is tough to make that amount of information consumable again. ▪ That’s why we have computers ▪ With cloud computing, on demand processing and advances in hardware we’re faster than ever.
Leaving invisible marks… @codepo8 ▪ By using other people’s machines and infrastructure, we leave traces ▪ This allows companies to recognise us, and accumulates a usage history ▪ This leads to better results, but can leak data ▪ We should have more transparency about what digital legacy we left behind.
Are machines friend or foe? Florian Ziegler flickr.com/photos/damndirty/40153024740/
Artificial Intelligence Myths @codepo8 ▪ AI can’t replace a thinking, creative human ▪ AI can not magically fill gaps with perfect information – it can only compare and assume ▪ AI doesn’t learn in a creative fashion. It makes no assumptions ▪ AI has no morals and ethics, but – used wrongly – it can amplify our biases
Machines can be great tools or weapons… @codepo8 ▪ Machine Learning is all about returning assumptions ▪ We don’t get any definitive truth from algorithms, we get answers to our questions ▪ AI can answer questions, but it is up to you to ask good questions – generic questions yield assumed results.
Unguided or supervised AI… @codepo8 http://inspirobot.me
It can be demanding @codepo8 http://inspirobot.me
It can mix up needs… @codepo8 http://inspirobot.me
It can be overly excited… @codepo8 http://inspirobot.me
It can be a good warning… @codepo8 http://inspirobot.me
It can be painfully humbling… @codepo8 http://inspirobot.me
Prophetic, even? @codepo8 http://inspirobot.me
Passive aggressive towards humans… @codepo8
It can be adoringly cute… @codepo8 https://twitter.com/eron_gj/status/967672260147470336
Whilst being actually kick-ass @codepo8 https://www.youtube.com/watch?v=gn4nRCC9TwQ
Machines can be great tools or weapons… @codepo8 ▪ Untrained and limited data leads to terrible and biased AI results ▪ It is very easy to get either wrong deductions or false positives ▪ AI is as intelligent and good as the people who apply it
Machine learning helps us in a few ways… @codepo8 ▪ Recommendation ▪ Prediction ▪ Classification ▪ Clustering ▪ Generation
Machines ploughing through lots of data for you. Recommendation @codepo8 ▪ “I feel lucky” moments ▪ Slack finding people in your organization ▪ Intelligent inboxes ▪ Automated photo optimization ▪ Automated tagging and alternative text: “Image may contain”
You’re doing this – you probably want this as the next thing Prediction @codepo8 ▪ Text autocompletion ▪ Task offerings ▪ Image tooling – adding photos to a collage ▪ Creating albums ▪ Offering similar music and videos ▪ Offering products that match
Sort things by what humans told you what they are and scale it up Classification @codepo8 ▪ Google surveys offering the right form elements for a question ▪ Detecting faces and asking for more information ▪ Finding anomalies in health scans and doing the same for all the ones in the system
Find own patterns and collate them Clustering @codepo8 ▪ Photo tagging and ordering ▪ Document analysis ▪ Comment filtering and triaging ▪ Video optimisation dependent on content.
Allow the machine to create things Generation @codepo8 ▪ Art style matching ▪ Generated articles from fact collection ▪ Synthesised music ▪ Filling content with tagged information (grass, houses, brick, etc…) ▪ React to human input
We need to find our place on the scale @codepo8
About face… @codepo8 aka.ms/face-api
About face… @codepo8 ▪ Face rectangle / Landmarks ▪ Pose (pitch/roll/yaw) ▪ Smile ▪ Gender/Age ▪ Type of glasses ▪ Makeup (lips/eye) ▪ Emotion (anger, contempt, disgust, fear, happiness, neutral, sadness, surprise) ▪ Occlusion (forehead/eye/mouth) ▪ Facial hair (moustache/beard/sideburns) ▪ Attributes: Hair (invisible, bald, colour) aka.ms/face-api
Is this you? Are those also you? @codepo8 aka.ms/face-api
Is this your driver? @codepo8 youtube.com/watch?v=aEBi4OpXU4Q
Taking it too far? @codepo8 ntechlab.com
Detecting even more… @codepo8 https://apnews.com/bf75dd1c26c947b7826d270a16e2658a
Those trustworthy avatars… @codepo8 https://blog.insightdatascience.com/ generating-custom-photo-realistic-faces-using-ai-d170b1b59255
Those trustworthy avatars… @codepo8 https://blog.insightdatascience.com/ generating-custom-photo-realistic-faces-using-ai-d170b1b59255
Automated face mapping… @codepo8 https://github.com/SpiderLabs/social_mapper
Once you are known… @codepo8 https://github.com/SpiderLabs/social_mapper
Photo by Florian Ziegler flickr.com/photos/damndirty/40153024740/ AI for humans Andreas Dantz flickr.com/photos/szene/40193567250
I want people to appreciate AI, without giving up their data unwillingly… @codepo8
The best way to do this, is to stop selling it as magic, but as a tool… @codepo8
How AI can help humans… @codepo8 aka.ms/ai-for-good
Humans ▪ Messy and prone to mistakes ▪ Forget things and filter them by their biases Bots and computers… ▪ Make no mistakes, other than physical fatigue ▪ Never forget, don’t judge ▪ Bored when doing repetitive tasks ▪ Great at tedious, boring tasks ▪ When bored create more errors ▪ Repeat things with minor changes on iterations till a result is met ▪ Non-optimised communication, lots of nuances and misunderstanding @codepo8 ▪ Highly optimised, non-nuanced communication.
Humans ▪ Messy and prone to mistakes ▪ Forget things and filter them by their biases Bots and computers… ▪ Make no mistakes, other than physical fatigue ▪ Never forget, don’t judge ▪ Bored when doing repetitive tasks ▪ Great at tedious, boring tasks ▪ When bored create more errors ▪ Repeat things with minor changes on iterations till a result is met ▪ Non-optimised communication, lots of nuances and misunderstanding @codepo8 ▪ Highly optimised, non-nuanced communication.
We need data, so let’s make it joyful for humans to give us some @codepo8
Humans and Bots/Computers @codepo8 autodraw.com
Humans and Bots/Computers @codepo8 autodraw.com
Humans and Bots/Computers @codepo8 quickdraw.withgoogle.com
Humans and Bots/Computers @codepo8 google.com/recaptcha/intro
“Learning” from lots of images @codepo8 https://github.com/jantic/DeOldify
Humans and Bots/Computers aka.ms/nvidia-fix-image
Humans and Bots/Computers aka.ms/nvidia-fix-image
Humans and Bots/Computers aka.ms/nvidia-fix-image
Humans and Bots/Computers gandissect.csail.mit.edu/
Our toolkit for more human interfaces Natural language processing @codepo8 Computer Vision Sentiment analysis Speech conversion and analysis Moderation
Language and Writing @codepo8 ▪ Probably the oldest task on the web was translation ▪ This moved deeper into Natural Language Processing and Language Detection ▪ Using these, we can allow for human commands and finding out tasks by analyzing texts. “How far am I from the capital of Denmark?” “Where do I find a good restaurant around here?” “Show me documents I wrote five days ago with more than 600 words”
Computer Vision @codepo8 ▪ When text wasn’t cool enough, we added images to our web media ▪ Often we forget that not everyone can see them, and we leave them without alternative text ▪ This is where machine learning steps in to help turning an image into a dataset we can work with.
Vision and image analysis… instagram: @larryandanke @codepo8
Vision and image analysis… @codepo8
Vision and image analysis… @codepo8
Vision and image analysis… @codepo8 twitter.com/mixedhunty/status/980551155297157126
Vision and image analysis… @codepo8 #vision_api
Vision and image analysis… @codepo8 aka.ms/vision-api
Vision and image analysis… @codepo8 aka.ms/vision-api
Vision and image analysis… @codepo8 aka.ms/vision-api
Vision and image analysis… @codepo8 aka.ms/vision-api
Vision and image analysis… @codepo8 aka.ms/vision-api
Sentiment analysis @codepo8 ▪ Finding out the sentiment of a text, image or video can help with a lot of things ▪ You can navigate videos by only showing the happy parts ▪ You can detect which comment should be answered first by a help desk ▪ You can predict when drivers of cars get tired
▪ Audio interfaces are all the rage. ▪ You can allow hands-free control of devices ▪ You can have an “always on” system to help you out without having to interface with it ▪ It feels natural and has a massive Sci-Fi feeling – when it works. Speech @codepo8
Speech recognition @codepo8 aka.ms/text-to-speech
Turning sentences into commands @codepo8 luis.ai aka.ms/luis-api
Text to speech @codepo8 aka.ms/text-to-speech
Conversation as an interface @codepo8 aka.ms/conversation-ui
Speaker recognition @codepo8 aka.ms/speaker-recognition
Speaker recognition @codepo8 aka.ms/speaker-recognition
Moderation @codepo8 ▪ Some things are not meant to be consumed by people ▪ Computers don’t need counselling once they saw them – people should ▪ Known illegal and terrible content can be automatically removed
With great power comes great responsibility… @codepo8
Our responsibilities.. @codepo8 ▪ AI can be an amazing help for humans ▪ It does need transparency – if you use people as data sources, they need to know what and where it goes ▪ When people get information filtered by an algorithm, it should be an opt-in ▪ People need to have a chance to dispute when an algorithm tagged or disallowed them access.
Want to go deep? ▪ The Math behind ML ▪ The ethics of AI ▪ Working with Data using Python ▪ Machine Learning Models ▪ Deep Learning Models ▪ Reinforcement Learning Models ▪ Microsoft Professional Program Certificate in Artificial Intelligence aka.ms/learn-ai 10 courses, (8-16 hours each), 10 skills @codepo8
Want to go deep? skl.sh/christian Free with trial sign-up @codepo8
Who controls our data? Who benefits? @codepo8 ▪ With all this we need to make clear who has your data and where it goes. ▪ Wouldn’t it be great if we could do more on our devices? ▪ Much lower latency, better security, increased privacy ▪ Right now, this is only possible in native environments ▪ I want to change that – a W3C proposal to bring accelerated Machine Learning to the web in JavaScript
Who controls our data? Who benefits? w3.org/community/webmachinelearning @codepo8
Don’t forget to have fun! @codepo8
Suz Hinton @codepo8 github.com/noopkat/face-api-emoji-face
Categorising images by gesture @codepo8 http://pointerpointer.com
Find your moves @codepo8 https://experiments.withgoogle.com/move-mirror
Stay silly… @codepo8 Cassie Evans https://codepen.io/cassie-codes/pen/jKaVqo/
Help the human @codepo8 https://charliegerard.github.io/teachable-keyboard/
Artificial Intelligence @codepo8 https://charliegerard.github.io/teachable-keyboard/
Collaborate and share… @codepo8 Linda Liukas https://helloruby.com
Preparing the next generation @codepo8 Linda Liukas https://helloruby.com
Thanks! Chris Heilmann Christianheilmann.com Developer-evangelism.com @codepo8 http://inspirobot.me/
