A presentation at Birmingham on Rails in in Birmingham, AL, USA by Ben Greenberg
What’s Love Got To Do With It? Ruby and Sentiment Analysis @RABBIGREENBERG
Hello! • I 💙 Ruby • Former rabbi & community organizer • 🌎 San Diego ↝ NYC ↝ Boston ↝ Denver ↝ Tel Aviv 🌍 • Developer Advocate @ Nexmo @RABBIGREENBERG
Our Roadmap @RABBIGREENBERG A. B. C. D. E. Why does this matter? What is sentiment analysis? What can I build with it? Let’s build it! Test Run and Data Analysis
Why does this matter? @RABBIGREENBERG
Language is not only a passive vehicle for information @RABBIGREENBERG
“Last night I had dinner with… someone, it was delicious“ @RABBIGREENBERG
’’אתמול בערב אכלתי ארוחת ערב עם… מישהו ,היא טעיימה’’ @RABBIGREENBERG
Egocentric or Geocentric? @RABBIGREENBERG
… if you saw a Guugu Yimithirr speaker pointing at himself, you would naturally assume he meant to draw attention to himself. In fact, he is pointing at a cardinal direction that happens to be behind his back. While we are always at the center of the world, a Guugu Yimithirr speaker points through himself… - Guy Deutscher, The New York Times, 2010 @RABBIGREENBERG
“When a language dies, a way of understanding the world dies with it, a way of looking at the world.” - George Steiner @RABBIGREENBERG
Language is a conduit constructor @RABBIGREENBERG
Understanding language helps us in understanding human society @RABBIGREENBERG
Algorithms can assist in getting us there @RABBIGREENBERG
What is Natural Language Understanding & Sentiment Analysis? @RABBIGREENBERG
“ I believe that at the end of the century the use of words and general educated opinion will have altered so much that one will be able to speak of machines thinking without expecting to be contradicted. “
“Last night I had dinner with someone, it was delicious“ @RABBIGREENBERG
Natural Language Processing @RABBIGREENBERG
Formal Grammar Inflectional Endings Part-of-speech (POS) tagging Parsing sentences Sentence boundaries Root form of words @RABBIGREENBERG
Segment continuous text into separate words (prevalent in many languages) Relationships of named entities to each other (who to whom, what to what, etc.) Topic segmentation Word meaning probability @RABBIGREENBERG
What’s my polarity? “I’d really truly “I’d really “I love my mobiletruly but love going lovenot going outout inin would recommend it thiscolleagues.” weather!” to any of my this weather!” “I dislike broken cars.” *examples from Wikipedia @RABBIGREENBERG
What can I make with it? @RABBIGREENBERG
Sometimes you just need to get to the point @RABBIGREENBERG
…and just tell me What’s the mood of the news? @RABBIGREENBERG
WHAT ARE WE BUILDING (1) “topic” (2) (3) @RABBIGREENBERG
Let’s build it @RABBIGREENBERG
What APIs will we be using? NEWS API “Search worldwide news with code” Generous free access tier Ruby SDK Documentation www.newsapi.org IBM WATSON Natural Language Analyzer API Generous free access tier Ruby SDK Documentation www.ibm.com/watson/developer NEXMO Messages API Generous free access tier Ruby SDK Documentation developer.nexmo.com @RABBIGREENBERG
Gemfile @RABBIGREENBERG
@RABBIGREENBERG
Routes @RABBIGREENBERG
@RABBIGREENBERG
Environment Variables @RABBIGREENBERG
@RABBIGREENBERG
Controller @RABBIGREENBERG
Defining Methods @rabbigreenberg @RABBIGREENBERG
#inbound @RABBIGREENBERG
#analyze_headlines @RABBIGREENBERG
#get_news_headlines @RABBIGREENBERG
_ p p a s t a h w _ d n #se msg @RABBIGREENBERG
#generate_jwt_token @RABBIGREENBERG
Giving It A Go @RABBIGREENBERG
@RABBIGREENBERG
Examining the Data @RABBIGREENBERG
What were the headlines? @RABBIGREENBERG IBM, Microsoft, a medley of others sing support for Google against Oracle in Supremes Java API copyright case Future-proof your tech skills with the Ultimate Deep Learning Bundle An Illuminated Dress Made With FLORA and Neopixels from The Dressed Aesthetic #WearableWednesday @dressed_app George Cheeks Exits As Vice Chairman of NBCUniversal Content Studios To win in a shifting market, the streaming industry is bundling up How CollegeHumor lost its early lead The Rundown: Video metrics will never be totally reliable 台積電 5 奈奈⽶米即將量量產，可望獨拿蘋果、華為訂單 Sequential’s Pro 3 is a new Prophet, while the others clone – so how does it stack up? The Continental: John Wick TV Show Is ‘Moving Along,’ Says Starz Boss iOS 13.3.1 Public Beta 2 Is Back After A Long Hiatus Perfect Sense unveils Gyro to simplify cloud infrastructure management “Stargirl” Is Ready to Embrace Her Destiny – But Will She Survive The Passing of The Cosmic Staff? [TRAILER] “Crisis” Management: Yup, Grant Gustin’s The Flash Just Met [SPOILER] – Snyder Cut of “Crisis” to Follow? “Crisis” Management: What’s A Little “Universal Domination” Among Enemies – Right, Lex? [PREVIEW] Labbers of the World Unite to Write a Book in 1 Week Through a Book Sprint Instagram Makes Long-Lived Access Tokens Available for the Instagram Basic Display API Aaron Rodgers jokes he’ll ‘definitely’ shake Richard Sherman’s hand 49ers’ turnaround, win improvement has put team in exclusive company NFL rumors: 49ers’ Joe Woods finalist for Browns defensive coordinator
In Numbers 18 Headlines in Sample 7 Programming Related 38.8% of Sample Directly Related to Topic @RABBIGREENBERG
In other words… Always examine your data. @RABBIGREENBERG
Examining the Analysis @RABBIGREENBERG
What were the sentiment results? Score: -0.62093 Label: negative @RABBIGREENBERG
What were the emotion results? @RABBIGREENBERG
Why the discrepancy? @RABBIGREENBERG
To understand we return to the data What were the concepts identified? @RABBIGREENBERG
Concept Scoring Concept Relevance Board of Directors 90% Richard Sherman 87% Chairman 84% Application Programming Interface 83% @RABBIGREENBERG
Where does this leave us? @RABBIGREENBERG
“ I believe that at the end of the century the use of words and general educated opinion will have altered so much that one will be able to speak of machines thinking without expecting to be contradicted. “
Thank you! @RABBIGREENBERG
Resources • • • • • • Deutscher, G. (2010). Through the Language Glass: Why The World Looks Different in Other Languages, Metropolitan Books Giannakidou, A. (2011). Positive polarity items and negative polarity items: variation, licensing, and compositionality, University of Chicago IBM Watson Natural Language Understanding API Reference: https://cloud.ibm.com/apidocs/natural-language-understanding/ natural-language-understanding Nexmo Messages API Reference: https://developer.nexmo.com/api/messages-olympus News API Reference: https://newsapi.org/docs “Mood of the News” GitHub Repository: https://github.com/Nexmo/mood-of-the-news @RABBIGREENBERG
The societies we live in, the companies we work for, the media we consume and much more are all shaped by words. Words can infuriate, enlighten, bring joy or cause great despair. Natural Language Understanding gives us a window into analyzing the words that surround us for sentiment and tone.
How does Natural Language Understanding work? What insights can we glean from the data it provides?
We will take a dive into understanding how this technology works, and apply it in a Ruby app that connects Natural Language Understanding analysis, the daily headlines and social media all in one. Get ready to learn some Ruby, some human language insights and how they all intertwine!
for free. You
can too.