Fun with Sensors and Browser Apis

Twitter: @mandy_kerr Codepen: @mandymichael

Jello the little fluffy fellow - my dog! :D Instagram: adognamedjello

Experimental Features

What i would like you to remember is that the talk is called “FUN” with sensors and browser apis, because i like to play and tinker, so that is what my examples are going to be. I haven’t built anything particularly serious with it, but I see this as a good opportunity to start thinking about how you might be able to make the most of them in your work, experiments or side projects.

Some of my examples will have really good browser support, and some are still very experimental, but i’d say remember that Everything we have on the web started out as a piece of experimental technology so let’s see what we can do with it and have a little fun along the way.

Support Legend

full support (Logo on its own) behind a flag (logo with a flag) coming soon (logo with a star)

“ The Web Speech API enables you to incorporate voice data into web apps. ” @mandy_kerr

new SpeechRecognition(); //constructor for SpeechRecognition object instance @mandy_kerr

recognition.start(); // method to start speech recognition service recognition.stop(); // stops the speech recognition service from listening to audio returns a result @mandy_kerr

recognition.onresult = function(event) {} // event handler when a word or phrase has been recognised recognition.onspeechend = function(event) {} // will run when the speech recognition service has stopped being detected @mandy_kerr

// get SpeechRecognitionResultList object const resultList = event.results; // get first SpeechRecognitionResult object const firstResult = resultList[0]; // get this result’s first SpeechRecognitionAlternative object const firstAlternative = firstResult[0].transcript; @mandy_kerr

Result Alternative List Object const speechResult = event.results[0][0].transcript; Result Text content @mandy_kerr

var SpeechRecognition = SpeechRecognition || webkitSpeechRecognition; function testSpeech() { const recognition = new SpeechRecognition(); const diagnostic = document.querySelector(‘.output’); recognition.start(); recognition.onresult = function(event) { const speechResult = event.results[0][0].transcript; diagnostic.textContent = speechResult; } recognition.onspeechend = function() { recognition.stop(); } } document.body.onclick = function() { testSpeech(); } @mandy_kerr

@mandy_kerr

“ Audio & speech allow for more interactive and more accessible experiences. ” @mandy_kerr

n o i t a t n e ri O @mandy_kerr

“ The accelerometer detects a change to the orientation of the device. ” @mandy_kerr

DeviceOrientationEvent @mandy_kerr

event.alpha; // The direction the device is facing // Z Axis // Ranging from 0 to 360 @mandy_kerr

event.beta; // The front to back motion of the device // X Axis // Ranging from -180 to 180 @mandy_kerr

event.gamma; // The left to right motion of the device // Y Axis // Ranging from -90 to 90 @mandy_kerr

if (window.DeviceOrientationEvent) { window.addEventListener(‘deviceorientation’, deviceOrientationFunction); } @mandy_kerr

function deviceOrientationHandler(event) { const currentGamma = event.gamma; if (currentGamma < -50) { // do something } } @mandy_kerr

@mandy_kerr

@mandy_kerr

@mandy_kerr

@mandy_kerr

“ We can move on from static, predetermined interactions and allow the user to shape the experience. ” @mandy_kerr

“ A photodetector that is used to sense the amount of ambient light present. ” @mandy_kerr

A second attempt at the ambient light sensor. @mandy_kerr

Addressing the security concerns of light sensors in the browser @mandy_kerr

const sensor = new AmbientLightSensor(); // Creates a new AmbientLightSensor object. @mandy_kerr

sensor.onreading // returns the current light level @mandy_kerr

sensor.illuminance // returns the current light level (lux) @mandy_kerr

if ( ‘AmbientLightSensor’ in window ) { const sensor = new AmbientLightSensor(); sensor.onreading = () => { if ( sensor.illuminance < 20) { element.style.setProperty(‘—bg’, ‘black’); } else { element.style.setProperty(‘—bg’, ‘blue’); } }; sensor.start(); } @mandy_kerr

Font: Tiny by Jack Halten Fahnestock @mandy_kerr

Font: BloomGX by Typearture (Arthur Reinders Folmer) @mandy_kerr

“ Don’t be limited by what we can already do, the web is still young and there is so much for us to create. ” @mandy_kerr

@mandy_kerr

@mandy_kerr

developer.mozilla.org/docs/Web/API developer.mozilla.org/docs/Web/API/Sensor_APIs developer.mozilla.org/docs/Web/API/SpeechRecognition developer.mozilla.org/docs/Web/API/AmbientLightSensor developer.mozilla.org/docs/Web/API/DeviceOrientationEvent @mandy_kerr