A presentation at JS Kongress in in Munich, Germany by Trent Willis
WEAVING WEBS F WORKERS @TRENTMWILLIS #JSKongress
Hallo JS Kongress! @TRENTMWILLIS #JSKongress
Hallo JS Kongress! Raise your ✋ if you work on a web application. @TRENTMWILLIS #JSKongress
Cool Web App That Pays The Bills @TRENTMWILLIS #JSKongress
Cool Web App That Pays The Bills @TRENTMWILLIS #JSKongress
Cool Web App That Pays The Bills @TRENTMWILLIS #JSKongress
Cool Web App That Pays The Bills @TRENTMWILLIS #JSKongress
@TRENTMWILLIS #JSKongress
@TRENTMWILLIS #JSKongress
How do we prevent numerous, large, and/or slow data requests from impacting our users? @TRENTMWILLIS #JSKongress
Web Workers. They help, but also complicate. @TRENTMWILLIS #JSKongress
WEAVING WEBS F WORKERS @TRENTMWILLIS #JSKongress
WEAVING WEBS F WORKERS @TRENTMWILLIS #JSKongress
@TRENTMWILLIS Senior UI Engineer at Netflix y e d i p S @TRENTMWILLIS #JSKongress
The Web Workers API “allows Web application authors to spawn background workers running scripts in parallel to their main page.” @TRENTMWILLIS #JSKongress
new Worker(‘worker.js’); @TRENTMWILLIS #JSKongress
new Worker(‘worker.js’); new SharedWorker(‘worker.js’); @TRENTMWILLIS #JSKongress
new Worker(‘worker.js’); It’s like a <script> but loads in a different thread! @TRENTMWILLIS #JSKongress
Web Workers allow “for thread-like operation with message-passing as the coordination mechanism.” @TRENTMWILLIS #JSKongress
// main thread worker;; @TRENTMWILLIS #JSKongress
// main thread worker.postMessage(message);; @TRENTMWILLIS #JSKongress
// worker thread self // WorkerGlobalScope @TRENTMWILLIS #JSKongress
// worker thread self.addEventListener( ‘message’, event => { console.log(event.data); }; ); @TRENTMWILLIS #JSKongress
// worker thread self.addEventListener( ‘message’, event => { console.log(event.data); self.postMessage(message); }; ); @TRENTMWILLIS #JSKongress
Messaging is the bulk of the Web Workers API you need! // main thread worker.addEventListener( ‘message’, event => console.log(event.data) ); @TRENTMWILLIS #JSKongress
// main thread worker.terminate(); @TRENTMWILLIS #JSKongress
PROBLEMS @TRENTMWILLIS #JSKongress
PROBLEM Knowing when a task is completed @TRENTMWILLIS #JSKongress
PROBLEM Management and coordination of multiple workers @TRENTMWILLIS #JSKongress
PROBLEM Difficult to test @TRENTMWILLIS #JSKongress
PROBLEM No dynamic definition of workers @TRENTMWILLIS #JSKongress
SOLUTIONS @TRENTMWILLIS #JSKongress
PROBLEM Knowing when a task is completed @TRENTMWILLIS #JSKongress
PROBLEM Knowing when a task is completed Turn messages into Promises @TRENTMWILLIS Replace one platform feature with another! #JSKongress
PROBLEM Knowing when a task is completed SOLUTION Turn messages into Promises const postMessage = (worker, message) => new Promise(resolve => { const resolution = (event) => { worker.removeEventListener(‘message’, resolution); resolve(event.data); }; worker.addEventListener(‘message’, resolution); worker.postMessage(message); }); @TRENTMWILLIS #JSKongress
PROBLEM Knowing when a task is completed SOLUTION Turn messages into Promises postMessage(worker, data) data).then(response response => console.log(response) console.log(response)); @TRENTMWILLIS #JSKongress
PROBLEM Knowing when a task is completed SOLUTION Turn messages into Promises const response = await postMessage(worker, data); console.log(response); @TRENTMWILLIS #JSKongress
PROBLEM Knowing when a task is completed SOLUTION Turn messages into Promises promise-worker github.com/nolanlawson/promise-worker @TRENTMWILLIS #JSKongress
PROBLEM Management and coordination of multiple workers @TRENTMWILLIS #JSKongress
PROBLEM Management and coordination of multiple workers Use Promises (again) @TRENTMWILLIS #JSKongress
PROBLEM Management and coordination of multiple workers Expose Worker methods as main thread functions @TRENTMWILLIS #JSKongress
PROBLEM Management and coordination of multiple workers SOLUTION Expose Worker methods as main thread functions backendOneWorker backendTwoWorker @TRENTMWILLIS #JSKongress
PROBLEM Management and coordination of multiple workers SOLUTION Expose Worker methods as main thread functions const data = await Promise.all([ backendOneWorker.fetch(‘first’), backendTwoWorker.fetch(‘second’) ]); @TRENTMWILLIS #JSKongress
PROBLEM Management and coordination of multiple workers SOLUTION Expose Worker methods as main thread functions const data = await Promise.all([ backendOneWorker.fetch(‘first’), backendTwoWorker.fetch(‘second’) ]); const result = await processingWorker.process(data); console.log(result); @TRENTMWILLIS #JSKongress
PROBLEM Management and coordination of multiple workers SOLUTION Expose Worker methods as main thread functions const data = await Promise.all([ backendOne.fetch(‘first’), backendTwo.fetch(‘second’) ]); const result = await processing.process(data); console.log(result); A good Worker abstraction looks like any other object! @TRENTMWILLIS #JSKongress
PROBLEM Management and coordination of multiple workers SOLUTION Expose Worker methods as main thread functions Workerize github.com/developit/workerize @TRENTMWILLIS #JSKongress
PROBLEM No dynamic definition of workers @TRENTMWILLIS #JSKongress
PROBLEM No dynamic definition of workers Create Workers from Blob URLs of functions @TRENTMWILLIS #JSKongress
PROBLEM No dynamic definition of workers SOLUTION Create Workers from Blob URLs of functions
const workerFromFunction = (fn) => { const src = (${fn})();; const blob = new Blob([src], {type: ‘application/javascript’}); const url = URL.createObjectURL(blob); return new Worker(url); };
@TRENTMWILLIS
#JSKongress
PROBLEM No dynamic definition of workers SOLUTION Create Workers from Blob URLs of functions greenlet github.com/developit/greenlet @TRENTMWILLIS #JSKongress
Lumen bit.ly/netflix-lumen @TRENTMWILLIS #JSKongress
Lumen @TRENTMWILLIS #JSKongress
Lumen “The majority of data operations in Lumen are done in Web Workers. This allows Lumen to keep the main thread free for user interactions, such as scrolling and interacting with individual charts, as the dashboard loads all of its data.” @TRENTMWILLIS #JSKongress
This is how we “weave” a web of Web Workers! Worker-To-Worker Communication @TRENTMWILLIS #JSKongress
// worker thread const workerInWorker = new Worker(‘worker.js’); @TRENTMWILLIS #JSKongress
MessageChannel @TRENTMWILLIS #JSKongress
MessageChannel consists of 2 MessagePorts @TRENTMWILLIS #JSKongress
// main thread const worker1 = new Worker(‘worker-1.js’); const worker2 = new Worker(‘worker-2.js’); @TRENTMWILLIS #JSKongress
// main thread const worker1 = new Worker(‘worker-1.js’); const worker2 = new Worker(‘worker-2.js’); const channel = new MessageChannel(); @TRENTMWILLIS #JSKongress
// main thread const worker1 = new Worker(‘worker-1.js’); const worker2 = new Worker(‘worker-2.js’); const channel = new MessageChannel(); worker1.postMessage(‘MessagePort’, [channel.port1]); worker2.postMessage(‘MessagePort’, [channel.port2]); @TRENTMWILLIS #JSKongress
Transferable “[A Transferable] represents an object that can be transferred between different execution contexts, like the main thread and Web Workers.” @TRENTMWILLIS #JSKongress
// worker thread self.addEventListener(‘message’, (event) => { if (event.ports.length) { }; }); @TRENTMWILLIS #JSKongress
// worker thread self.addEventListener(‘message’, (event) => { if (event.ports.length) { event.ports[0].onmessage = event => console.log(event.data); event.ports[0].postMessage(‘hello from worker 2’); }; }); @TRENTMWILLIS #JSKongress
const data = await Promise.all([ backendOneWorker.fetch(‘first’), backendTwoWorker.fetch(‘second’) ]); const result = await processingWorker.process(data); console.log(result); @TRENTMWILLIS You can do this entirely off the main thread! #JSKongress
Non-Blocking Canvas Graphics @TRENTMWILLIS #JSKongress
OffscreenCanvas @TRENTMWILLIS #JSKongress
Non-Blocking DOM Manipulation @TRENTMWILLIS #JSKongress
worker-dom github.com/ampproject/worker-dom @TRENTMWILLIS #JSKongress
Conway’s Game of Life canvas-of-life.glitch.me @TRENTMWILLIS #JSKongress
@TRENTMWILLIS So janky! #JSKongress
@TRENTMWILLIS Much better! #JSKongress
You Can Do A LOT With Web Workers… @TRENTMWILLIS #JSKongress
PROBLEM Difficult to test How do we test them? @TRENTMWILLIS #JSKongress
PROBLEM Difficult to test A Tale of Two Strategies @TRENTMWILLIS #JSKongress
PROBLEM Difficult to test Run your testing framework and worker in the same thread @TRENTMWILLIS #JSKongress
PROBLEM Difficult to test // Main <script <script <script @TRENTMWILLIS thread src=”test-framework.js”></script> src=”worker.js”></script> src=”tests.js”></script> #JSKongress
PROBLEM Difficult to test // Main <script <script <script thread src=”test-framework.js”></script> src=”worker.js”></script> src=”tests.js”></script> // Or, worker thread importScripts(‘test-framework.js’, ‘worker.js’); // Your tests here… @TRENTMWILLIS #JSKongress
PROBLEM Difficult to test That is NOT how Workers are used. @TRENTMWILLIS #JSKongress
PROBLEM Difficult to test Treat your Worker as a Function @TRENTMWILLIS #JSKongress
PROBLEM Difficult to test SOLUTION Treat your Worker as a Function
test(‘transforms data’, async (assert) => { const worker = new Worker (‘transform.js’); const data = [1, 2, 3]; const result = postMessage(worker, data); assert.equal(result, I'm transformed!); });
@TRENTMWILLIS
#JSKongress
PROBLEM Difficult to test SOLUTION Treat your Worker as a Function Sub-Problem: How do we mock/stub calls a Worker? @TRENTMWILLIS #JSKongress
PROBLEM Difficult to test SOLUTION Treat your Worker as a Function worker-box github.com/trentmwillis/worker-box @TRENTMWILLIS #JSKongress
canvas-of-life.glitch.me/tests @TRENTMWILLIS #JSKongress
Web Workers are powerful @TRENTMWILLIS #JSKongress
Web Workers are powerful, but avoid using them directly @TRENTMWILLIS #JSKongress
Web Workers are powerful, but avoid using them directly, instead stand on the shoulders of giants. @TRENTMWILLIS #JSKongress
Thank you! @TRENTMWILLIS Web Workers are powerful, but avoid using them directly, instead stand on the shoulders of giants. There is no better time to start than right now. #JSKongress
• • • • • • • • • Resources Spider icon made by Freepik from www.flaticon.com Web Workers spec: www.w3.org/TR/workers/ Promise Worker: github.com/nolanlawson/promise-worker Workerize: github.com/developit/workerize Greenlet: github.com/developit/greenlet Lumen: bit.ly/netflix-lumen Worker DOM: github.com/ampproject/worker-dom Game of Life Demo: canvas-of-life.glitch.me Worker Box: github.com/trentmwillis/worker-box @TRENTMWILLIS #JSKongress
The single-threaded nature of JavaScript can make handling lots of data or complex processing in your web application difficult to balance with providing a good user experience. Web Workers promise to help us find that balance and let us build fast and responsive applications even when doing heavy work, but the APIs for working with them leave much to be desired. In this talk, we’ll explore useful patterns for making Web Workers easier to work with and highlight effective ways to communicate between workers, handle complex data processing off the main thread, and test and debug them.
The following resources were mentioned during the presentation or are useful additional information.
Here’s what was said about this presentation on social media.
for free. You
can too.