Fun with Browser and Sensor APIS

A presentation at Web Directions Summit in November 2019 in Sydney NSW, Australia by Mandy Michael

Slide 1

Slide 1

Fun with sensors and browser apis

Slide 2

Slide 2

Mandy Michael @mandymichael @mandy_kerr

Slide 3

Slide 3

Jello the little fluffy fellow @adognamedjello

Slide 4

Slide 4

Slide 5

Slide 5

Web Speech APIs

Slide 6

Slide 6

“ The Web Speech API enables you to incorporate voice data into web apps. ” @mandy_kerr

Slide 7

Slide 7

Demo

Slide 8

Slide 8

new SpeechRecognition(); //constructor for SpeechRecognition object instance

Slide 9

Slide 9

recognition.start(); // method to start speech recognition service recognition.stop(); !// stops the speech recognition service from listening to audio returns a result

Slide 10

Slide 10

recognition.onresult = function(event) {} // event handler when a word or phrase has been recognised

recognition.onspeechend = function(event) {} // will run when the speech recognition service has stopped being detected

Slide 11

Slide 11

@mandy_kerr

Slide 12

Slide 12

Slide 13

Slide 13

Slide 14

Slide 14

Demo - Marshmallow

Slide 15

Slide 15

Demo - Rello the good boy

Slide 16

Slide 16

Demo - Voice and Text

Slide 17

Slide 17

Andre Natal Speech & AI engineer Mozilla @andrenatalbr wiki.mozilla.org/Web_Speech_API_-_Speech_Recognition @mandy_kerr

Slide 18

Slide 18

How Speech works in Firefox

Slide 19

Slide 19

“ Audio & speech allow for more interactive and more accessible experiences. ” @mandy_kerr

Slide 20

Slide 20

Orientation Sensors

Slide 21

Slide 21

“ The accelerometer detects a change to the orientation of the device. ” @mandy_kerr

Slide 22

Slide 22

Slide 23

Slide 23

Slide 24

Slide 24

Slide 25

Slide 25

Slide 26

Slide 26

Slide 27

Slide 27

Slide 28

Slide 28

Demo: Orientation Woops

Slide 29

Slide 29

Demo: Orientation Moon

Slide 30

Slide 30

Demo

Slide 31

Slide 31

Demo

@mandy_kerr

Slide 32

Slide 32

“ We can move on from static, predetermined interactions and allow the user to shape the experience. ” @mandy_kerr

Slide 33

Slide 33

Ambient Light Sensor

Slide 34

Slide 34

“ A photodetector that is used to sense the amount of ambient light present. ” @mandy_kerr

Slide 35

Slide 35

Rijubrata Bhaumik Tech lead, Intel web platform @rijubrata @mandy_kerr

Slide 36

Slide 36

A second attempt at the ambient light sensor.

Slide 37

Slide 37

Addressing the security concerns of light sensors in the browser

Slide 38

Slide 38

Slide 39

Slide 39

Slide 40

Slide 40

Slide 41

Slide 41

Slide 42

Slide 42

Demo: Sun and Moon

Slide 43

Slide 43

Demo: Light it up

Slide 44

Slide 44

Demo: Bloom

Font: BloomGX by Typearture (Arthur Reinders Folmer)

Slide 45

Slide 45

“ Don’t be limited by what we can already do, the web is still young and there is so much for us to create. ” @mandy_kerr

Slide 46

Slide 46

Demo: Secret Message

Slide 47

Slide 47

Demo: A Grumpy Wizard

Slide 48

Slide 48

Demo: UFO Cat Abduction

Slide 49

Slide 49

Slide 50

Slide 50

@mandy_kerr @mandymichael developer.mozilla.org/docs/Web/API developer.mozilla.org/docs/Web/API/Sensor_APIs developer.mozilla.org/docs/Web/API/SpeechRecognition developer.mozilla.org/docs/Web/API/AmbientLightSensor developer.mozilla.org/docs/Web/API/DeviceOrientationEvent @mandy_kerr