Fun with Browser and Sensor APIS

Slide 1

Fun with sensors and browser apis

Slide 2

Mandy Michael @mandymichael @mandy_kerr

Slide 3

Jello the little fluffy fellow @adognamedjello

Slide 4

Slide 5

Web Speech APIs

Slide 6

“ The Web Speech API enables you to incorporate voice data into web apps. ” @mandy_kerr

Slide 7

Demo

Slide 8

new SpeechRecognition(); //constructor for SpeechRecognition object instance

Slide 9

recognition.start(); // method to start speech recognition service recognition.stop(); !// stops the speech recognition service from listening to audio returns a result

Slide 10

recognition.onresult = function(event) {} // event handler when a word or phrase has been recognised

recognition.onspeechend = function(event) {} // will run when the speech recognition service has stopped being detected

Slide 11

@mandy_kerr

Slide 12

Slide 13

Slide 14

Demo - Marshmallow

Slide 15

Demo - Rello the good boy

Slide 16

Demo - Voice and Text

Slide 17

Andre Natal Speech & AI engineer Mozilla @andrenatalbr wiki.mozilla.org/Web_Speech_API_-_Speech_Recognition @mandy_kerr

Slide 18

How Speech works in Firefox

Slide 19

“ Audio & speech allow for more interactive and more accessible experiences. ” @mandy_kerr

Slide 20

Orientation Sensors

Slide 21

“ The accelerometer detects a change to the orientation of the device. ” @mandy_kerr

Slide 22

Slide 23

Slide 24

Slide 25

Slide 26

Slide 27

Slide 28

Demo: Orientation Woops

Slide 29

Demo: Orientation Moon

Slide 30

Demo

Slide 31

Demo

@mandy_kerr

Slide 32

“ We can move on from static, predetermined interactions and allow the user to shape the experience. ” @mandy_kerr

Slide 33

Ambient Light Sensor

Slide 34

“ A photodetector that is used to sense the amount of ambient light present. ” @mandy_kerr

Slide 35

Rijubrata Bhaumik Tech lead, Intel web platform @rijubrata @mandy_kerr

Slide 36

A second attempt at the ambient light sensor.

Slide 37

Addressing the security concerns of light sensors in the browser

Slide 38

Slide 39

Slide 40

Slide 41

Slide 42

Demo: Sun and Moon

Slide 43

Demo: Light it up

Slide 44

Demo: Bloom

Font: BloomGX by Typearture (Arthur Reinders Folmer)

Slide 45

“ Don’t be limited by what we can already do, the web is still young and there is so much for us to create. ” @mandy_kerr

Slide 46

Demo: Secret Message

Slide 47

Demo: A Grumpy Wizard

Slide 48

Demo: UFO Cat Abduction

Slide 49

Slide 50

@mandy_kerr @mandymichael developer.mozilla.org/docs/Web/API developer.mozilla.org/docs/Web/API/Sensor_APIs developer.mozilla.org/docs/Web/API/SpeechRecognition developer.mozilla.org/docs/Web/API/AmbientLightSensor developer.mozilla.org/docs/Web/API/DeviceOrientationEvent @mandy_kerr