Holly Cummins @holly_cummins with Sock, the Socratic Helper
A presentation at JavaOne in October 2017 in San Francisco, CA, USA by Holly Cummins
Holly Cummins @holly_cummins with Sock, the Socratic Helper
I’m from the IBM Cloud Garage. (It’s not actually a Garage.)
I’m from the IBM Cloud Garage. I’m from IBM’s London Chatbot Garage. (It’s not actually a Garage.)
Lean startup, innovation, testdriven development, pair programming, design thinking … Can you build me a chatbot please? IBM Bluemix Garage London
It’s 2017. Chatbots are everywhere. Even in games.
There are currently more than 33,000 chatbots on Facebook Messenger alone.
It’s 2017. There’s more natural language understanding. There’s more automation of boring tasks
Why does everyone want chatbots? They’re interesting and they solve problems.
Not all chatbots do the same things.
Chit-chat bot Worker bot
Chit-chat bot Neither user nor bot have a goal Worker bot
Chit-chat bot Worker bot Neither user nor bot have a goal Both user and bot have goals
Chit-chat bot Worker bot Neither user nor bot have a goal Both user and bot have goals I invented these terms!
The success metrics are different. Turing Test Containment
Chatbots are not new.
Person: Men are all the same. ELIZA: IN WHAT WAY. Person: They are always bugging us about something or other. ELIZA: CAN YOU THINK OF A SPECIFIC EXAMPLE. Person: Well, my boyfriend made me come here. ELIZA: YOUR BOYFRIEND MADE YOU COME HERE. Person: He says I’m depressed much of the time. ELIZA: I AM SORRY TO HEAR YOU ARE DEPRESSED. Person: It’s true, I am unhappy. ELIZA: DO YOU THINK COMING HERE WILL HELP YOU NOT TO BE UNHAPPY. Person: I need some help, that much seems certain. ELIZA 1964
2011: 5 minute Turing Test Cleverbot Human
2011: 5 minute Turing Test 63.3% Cleverbot Human
2011: 5 minute Turing Test 59.3% Cleverbot 63.3% Human
2011: 5 minute Turing Test 59.3% Cleverbot 63.3% Human
Estragon: I am a human. You are a computer. Vladimir: No I am not. Estragon: Yes you are, and I’m a human being Vladimir: But you said you are a robot. Estragon: No I didn’t! YOU said I was a robot. Vladimir: But you are. Estragon: I know you are but what am I? Vladimir: You are human. Estragon: And you are a robot. Vladimir: But you just said you were a robot. Estragon: Na I asked if you were a robot and you said yes. Vladimir: Nope. You are a manipulative bunch of metal.
Is this interesting? Yes, very.
Is this useful? Err…
Worker bots are designed to be useful.
They are limited, but (usually) helpful.
Do you need a chatbot?
Do you need a chatbot? Often, no.
Do you need a chatbot? Often, no.
Unless … conversations are kind of boring for the people doing them now.
Unless … users want to use pervasive channels.
Unless … users need aggregated information.
The chatbot fallacy:
The chatbot fallacy: “If a bot can understand a user speaking, it can have a conversation.”
The chatbot fallacy: “If a bot can understand a user speaking, it can have a conversation.”
The chatbot fallacy: “If a bot can understand a user speaking, it can have a conversation.” Voice is a well-solved problem.
The chatbot fallacy: “If a bot can understand a user speaking, it can have a conversation.” Voice is a well-solved problem. That doesn’t mean the rest of conversation is.
Demo!
IBM Bluemix Watson Speech to Text Watson Conversation React.js client Public Internet WebSphere Liberty PDFs Bluemix weather service
IBM Bluemix Watson Speech to Text Watson Conversation React.js client Public Internet WebSphere Liberty PDFs Bluemix weather service Did I mention Open Liberty is now open source?
This is what we’ve learned building a lot of chatbots.
Explain limitations. Give examples of what can be asked. I’m sorry, I’m not smart enough to fix that yet. You can ask me about applying for a credit card or changing your address.
Complex complex my grandmother’s octopus needs a new sofa on rollerskates? Don’t be afraid to say “I don’t know.” ???
Complex complex my grandmother’s octopus needs a new sofa on rollerskates? Don’t be afraid to say “I don’t know.” ???
Complex complex my grandmother’s octopus needs a new sofa on rollerskates? Don’t be afraid to say “I don’t know.” ???
Hand off to a human
Users are lazy.
Don’t make your users type more than they need to. Do give buttons for responses. Sorry, I don’t understand ‘mauve’. You must type ‘Yes please I want a mauve sofa’ or ‘Yes please I want a blue sofa.’
Don’t repeat yourself
A conversation flow
• • Provide multiple responses for each node. Detect repeat visits to a single node, and do something about it.
“I can see this isn’t going so well. Would you like to talk to a real person?” Detect frustration, and handle it.
IBM Bluemix Example architecture for a socially sensitive bot Watson Speech to Text Watson Conversation React.js client WebSphere Liberty PDFs Watson Sentiment Analysis Public Internet Bluemix weather service
Context is important
How complex the conversation gets is up to you.
Should your chatbot admit it’s not a person? Cunning humanlike disguise
Annoying: Hoping to talk to a human, and talking to a bot. More annoying: Believing you’re talking to a human, then realising it’s a bot.
You need a lot of automated tests and you should write them first
Devops still matters. Generation Source controlled inputs Tests. (Lots of tests.) Application Conversation representation.
Source control User (not seeing regressions ) A good devops flow for workspaces. Workspace json Developers, UX designers, and business Development Conversation instance Test Conversation instance Production Conversation instance
Then, observe your users
If a chatbot isn’t usable, it’s nothing.
Your test users may not be representative. Tell me a joke. edkjdflk fehfs fhfiuhf My credit card has been stolen! I need to cancel it. How smart are you? Real user Test user
Test, Monitor, Tune
Record your logs. Then look at them.
Stuff stuff reply … Ooh, I don’t like that language very much. Argh! You piece of %@*&@! Oh, sorry! Even if they know a bot is a bot, users may treat it like a person.
Continuous feedback Machine learning
Chatbots are awesome - but remember, the user has to come first. @holly_cummins https://www.ibm.com/devops/method/
