Predictive Prefetching

A presentation at performance.now() in November 2019 in Amsterdam, Netherlands by Divya

Slide 1

Slide 1

PREDICTIVE Prefetching

Slide 2

Slide 2

Divya Sasidharan @shortdiv

Slide 3

Slide 3

Slide 4

Slide 4

Slide 5

Slide 5

Slide 6

Slide 6

www.perfnow.nl https:// perfnow .nl/ 53 194.0.28. Data Transfer SYN SYN+ACK ACK

Slide 7

Slide 7

www.perfnow.nl js html css GET /index.html GET /style.css GET /script.js

Slide 8

Slide 8

PREFETCHING Prefetching TM

Slide 9

Slide 9

www.perfnow.nl

Slide 10

Slide 10

DNS Prefetching Link Prefetching Prerendering

Slide 11

Slide 11

DNS Prefetching

<link rel=”dns-prefetch” href=”http://cdn.example.com”>

Slide 12

Slide 12

DNS Prefetching Link Prefetching Prerendering

Slide 13

Slide 13

Link Prefetching

<link rel=”prefetch” href=“http://cdnz.com/library.js”>

Slide 14

Slide 14

DNS Prefetching Link Prefetching Prerendering

Slide 15

Slide 15

Prerendering

<link rel=”prerender” href=”http://cdnz.com/about.html”>

Slide 16

Slide 16

DNS Prefetching Link Prefetching Prerendering

Slide 17

Slide 17

Hint Cost (if wrong) Benefit (if correct) DNS Prefetch Very low Low Link Prefetch Mid high High Prerender Very high Very high

Slide 18

Slide 18

https://duckduckgo.com/?q=wtf+are+war+fries&t=h_&ia=web wtf are war fries A crash course on The Netherlands’ War Fries Call ‘em what you will - friet, friets, patat, Vlaamse Frieten - but one thing is sure, the Dutch can’t seem to get enough of ‘em! In fact, they love ‘em so much they eat over 41 million kilos of these tasty guys per year! (But Dutch people don’t just stop at fries. Every Dutchie Recipe for War Fries The perfect fries are the Belgium/Dutch (most of the credits to Belgium). They are big friesand if they are prepared good, the outside is a little crunchy, but the inside is soft. When you bite, you first go to a little crispy part and then the soft potato, what is the main part of the frie. Mc donalds fries, are french fries. Really thin. 2

Slide 19

Slide 19

username •••••••••••

Slide 20

Slide 20

https://cant-dutch-this.netlify.com 2 Can’t Dutch This Home About Us Menu Contact Blog Go dutch, you won’t regret it. 30 Sept 2018 WTF is war fries? 24 Aug 2018 Fries, the dutch way. TGI Friet day 4 June 2018 1 May 2018

Slide 21

Slide 21

https://cant-dutch-this.netlify.com 4 2 Can’t Dutch This Home About Us Menu Contact Blog Go dutch, you won’t regret it. 30 Sept 2018 WTF is war fries? 24 Aug 2018 Fries, the dutch way. TGI Friet day 4 June 2018 1 May 2018

Slide 22

Slide 22

https://cant-dutch-this.netlify.com 1 2 Can’t Dutch This Home About Us Menu Contact Blog Go dutch, you won’t regret it. 30 Sept 2018 WTF is war fries? 24 Aug 2018 Fries, the dutch way. TGI Friet day 4 June 2018 1 May 2018

Slide 23

Slide 23

SPECULATIV PreDictive

Slide 24

Slide 24

PREDICTIVE PreDictive

Slide 25

Slide 25

Slide 26

Slide 26

???

Slide 27

Slide 27

Slide 28

Slide 28

Slide 29

Slide 29

Today Tomorrow Overcast Rain Overcast 0.75 0.25 Rain 1 0

Slide 30

Slide 30

0.75 0 0.25 Overcast Rainy 1

Slide 31

Slide 31

/about /menu /about /contact /about /contact /about /menu /menu /contact /menu / /menu /about /menu /

Slide 32

Slide 32

/about /menu /about /menu /about /contact 50% 50% /about /contact /menu /contact 25% /menu /about 25% /menu / /menu / 50%

Slide 33

Slide 33

Slide 34

Slide 34

Slide 35

Slide 35

1 { 2 “columnHeader”: { 3 “dimensions”: [ 4 “ga:previousPagePath”, 5 “ga:pagePath” 6 ], 7 “metricHeader”: { 8 “metricHeaderEntries”: [ 9 { 10 “name”: “ga:pageviews”, 11 “type”: “INTEGER” 12 }, 13 { 14 “name”: “ga:exits”, 15 “type”: “INTEGER” 16 } 17 ] 18 } 19 }, 20 “data”: { 21 “rows”: [ 22 { 23 “dimensions”: [ 24 “(entrance)”, 25 “/” 26 ], 27 “metrics”: [ 28 {

Slide 36

Slide 36

36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 { “dimensions”: “/”, “/” ], “metrics”: [ { “values”: “4”, “1” ] } ] }, { “dimensions”: “/”, “/posts/” ], “metrics”: [ { “values”: “4”, “0” ] } ] }, { [ [ Previous Page Path Current Page Path Pageviews Exits [ [

Slide 37

Slide 37

36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 { “dimensions”: “/”, “/” ], “metrics”: [ { “values”: “4”, “1” ] } ] }, { “dimensions”: “/”, “/posts/” ], “metrics”: [ { “values”: “4”, “0” ] } ] }, { [ Previous Page Path Current Page Path [ [ [ Previous Page Path Current Page Path

Slide 38

Slide 38

1 { 2 “/”: { 3 “pagePath”: “/”, 4 “pageviews”: 6, 5 “exits”: 0, 6 “nextPageviews”: 6, 7 “nextExits”: 0, 8 “nextPages”: [ 9 { 10 “pagePath”: “/posts/”, 11 “pageviews”: 4 12 }, 13 { 14 “pagePath”: “/about/”, 15 “pageviews”: 1 16 }, 17 { 18 “pagePath”: “/contact/”, 19 “pageviews”: 1 20 } 21 ], 22 “percentExits”: 0, 23 “topNextPageProbability”: 0.6666666666666666 24 }, 25 “/posts/”: { 26 “pagePath”: “/posts/”, 27 “pageviews”: 9, 28 “exits”: 0,

Slide 39

Slide 39

1 { 2 “/”: { 3 “pagePath”: “/”, 4 “pageviews”: 6, 5 “exits”: 0, 6 “nextPageviews”: 6, 7 “nextExits”: 0, 8 “nextPages”: [ 9 { 10 “pagePath”: “/posts/”, 11 “pageviews”: 4 12 }, 13 { 14 “pagePath”: “/about/”, 15 “pageviews”: 1 16 }, 17 { 18 “pagePath”: “/contact/”, 19 “pageviews”: 1 20 } 21 ], 22 “percentExits”: 0, 23 “topNextPageProbability”: 0.6666666666666666 24 }, 25 “/posts/”: { 26 “pagePath”: “/posts/”, 27 “pageviews”: 9, 28 “exits”: 0,

Slide 40

Slide 40

1 { 2 “/”: { 3 “pagePath”: “/”, 4 “pageviews”: 6, 5 “exits”: 0, 6 “nextPageviews”: 6, 4 7 “nextExits”: 0, 8 “nextPages”: [ 6 9 { 10 “pagePath”: “/posts/”, 11 “pageviews”: 4 12 }, 13 { 14 “pagePath”: “/about/”, 15 “pageviews”: 1 16 }, 17 { 18 “pagePath”: “/contact/”, 19 “pageviews”: 1 20 } 21 ], 22 “percentExits”: 0, 23 “topNextPageProbability”: 0.6666666666666666 24 }, 25 “/posts/”: { 26 “pagePath”: “/posts/”, 27 “pageviews”: 9, 28 “exits”: 0,

Slide 41

Slide 41

1 { 2 “/”: { 3 “pagePath”: “/”, 4 “pageviews”: 6, 5 “exits”: 0, 6 “nextPageviews”: 6, 7 “nextExits”: 0, 8 “nextPages”: [ 1 9 { 10 “pagePath”: “/posts/”, 6 11 “pageviews”: 4 12 }, 13 { 14 “pagePath”: “/about/”, 15 “pageviews”: 1 16 }, 17 { 18 “pagePath”: “/contact/”, 19 “pageviews”: 1 20 } 21 ], 22 “percentExits”: 0, 23 “topNextPageProbability”: 0.6666666666666666 24 }, 25 “/posts/”: { 26 “pagePath”: “/posts/”, 27 “pageviews”: 9, 28 “exits”: 0,

Slide 42

Slide 42

PredIctIve Build Automation

Slide 43

Slide 43

Slide 44

Slide 44

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 const const const const axios = require(‘axios’); {google} = require(‘googleapis’) fs = require(‘fs’) path = require(‘path’) _data/prefetchLinks.js const authClient = new google.auth.JWT({ email: process.env.SERVICE_ACCOUNT_EMAIL, key: fs.readFileSync(path.join(__dirname, “../../key.pem”), ‘utf8’), scopes: [‘https://www.googleapis.com/auth/analytics.readonly’] }) const queryParams = { resource: { reportRequests: [ { viewId: process.env.VIEW_ID, dateRanges: [{startDate: ‘30daysAgo’, endDate: ‘yesterday’}], metrics: [ {expression: ‘ga:pageviews’}, {expression: ‘ga:exits’} ], dimensions: [ {name: ‘ga:previousPagePath’}, {name: ‘ga:pagePath’} ], orderBys: [ {fieldName: ‘ga:previousPagePath’, sortOrder: ‘ASCENDING’}, {fieldName: ‘ga:pageviews’, sortOrder: ‘DESCENDING’}

Slide 45

Slide 45

11 _data/prefetchLinks.js 12 const queryParams = { resource: { 13 reportRequests: [ 14 { 15 viewId: process.env.VIEW_ID, 16 dateRanges: [{startDate: ‘30daysAgo’, endDate: ‘yesterday’}], 17 metrics: [ 18 {expression: ‘ga:pageviews’}, 19 {expression: ‘ga:exits’} 20 ], 21 dimensions: [ 22 {name: ‘ga:previousPagePath’}, 23 {name: ‘ga:pagePath’} 24 ], 25 orderBys: [ 26 {fieldName: ‘ga:previousPagePath’, sortOrder: ‘ASCENDING’}, 27 {fieldName: ‘ga:pageviews’, sortOrder: ‘DESCENDING’} 28 ], 29 pageSize: 10000 30 } 31 ] 32 } 33 34 } 35 36 module.exports = async function() { try { 37 await authClient.authorize() 38 const analytics = google.analyticsreporting({ 39

Slide 46

Slide 46

} 33 _data/prefetchLinks.js 34 } 35 36 module.exports = async function() { try { 37 await authClient.authorize() 38 const analytics = google.analyticsreporting({ 39 version: ‘v4’, 40 auth: authClient 41 }) 42 const response = await analytics.reports.batchGet(queryParams) 43 let [report] = response.data.reports 44 45 const aggregatePages = async (pages) => { 46 … 47 return predictions 48 } 49 50 const aggPages = await aggregatePages(report); 51 52 return aggPages 53 } catch (err) { 54 console.log(“err”) 55 return err 56 } 57 58 } 59 60 61

Slide 47

Slide 47

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 { }, { }, { }, { }, { }, { “pagePath”: “/posts/”, “nextPagePath”: “/posts/thirdpost/”, “nextPageCertainty”: 0.3333333333333333 “pagePath”: “/”, “nextPagePath: ‘/posts/”, “nextPageCertainty”: 0.6666666666666666 “pagePath”: “/posts/thirdpost/”, “nextPagePath”: “/posts/”, “nextPageCertainty”: 0.42857142857142855 “pagePath”: “/posts/secondpost/”, “nextPagePath”: “/posts/thirdpost/”, “nextPageCertainty”: 0.5 “pagePath”: “/about/”, “nextPagePath”: “/contact/”, “nextPageCertainty”: 0.5 “pagePath”: “/contact/”, “nextPagePath”: “/”,

Slide 48

Slide 48

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 { }, { }, { }, { }, { }, { “pagePath”: “/posts/”, “nextPagePath”: “/posts/thirdpost/”, “nextPageCertainty”: 0.3333333333333333 “pagePath”: “/”, “nextPagePath: ‘/posts/”, “nextPageCertainty”: 0.6666666666666666 “pagePath”: “/posts/thirdpost/”, “nextPagePath”: “/posts/”, “nextPageCertainty”: 0.42857142857142855 “pagePath”: “/posts/secondpost/”, “nextPagePath”: “/posts/thirdpost/”, “nextPageCertainty”: 0.5 “pagePath”: “/about/”, “nextPagePath”: “/contact/”, “nextPageCertainty”: 0.5 “pagePath”: “/contact/”, “nextPagePath”: “/”,

Slide 49

Slide 49

base.njk 1 <!doctype html> 2 <html lang=”en”> 3 <head> 4 <meta charset=”utf-8”> 5 <meta name=”viewport” content=”width=device-width, initial-scale=1. 6 <title>{{ renderData.title or title or metadata.title }}</title> 7 <meta name=”Description” content=”{{ renderData.description }}”> 8 <link rel=”stylesheet” href=”{{ ‘/css/index.css’ | url }}”> 9 10 {% for entry in prefetchLinks %} 11 {% if entry.pagePath == page.url %} 12 <link rel=”prefetch” href=”{{ entry | prefetchNextURL }}”> 13 {% endif %} 14 {%- endfor -%} 15 </head> 16 17 <body> 18 {{ content | safe }} 19 </body> 20 21 </html> 22 23 24 25 26 27 28

Slide 50

Slide 50

.eleventy.js 1 module.exports = function(eleventyConfig) { 2 const THRESHOLD = 0.5 3 4 5 eleventyConfig.addFilter(‘prefetchNextURL’, (entry, url) => { 6 // check threshold // 7 if (entry.nextPageCertainty > THRESHOLD) { 8 return entry.nextPagePath 9 } 10 }) 11 12 … 13 } 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28

Slide 51

Slide 51

DEMO

Slide 52

Slide 52

Guess js

Slide 53

Slide 53

PREDICTIVE NAïve NAÏVE

Slide 54

Slide 54

INTELLIGENT PREDICTIVE Intelligent

Slide 55

Slide 55

Slide 56

Slide 56

Slide 57

Slide 57

Slide 58

Slide 58

Slide 59

Slide 59

/about /menu /about /menu /about /menu /about /contact /menu /contact /menu / /menu / /menu /about

Slide 60

Slide 60

/ /about /menu /contact /about /menu / /about /menu / /about /contact /contact /menu /contact / /menu / / /menu / /about /menu /about

Slide 61

Slide 61

GDPR

Slide 62

Slide 62

BuildTime BUILDTIME PREDICTIVE

Slide 63

Slide 63

REALTIME RealTime Throttling PREDICTIVE

Slide 64

Slide 64

PredIctIve Serverless Functions

Slide 65

Slide 65

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 const const const const axios = require(‘axios’); {google} = require(‘googleapis’) fs = require(‘fs’) path = require(‘path’) const authClient = new google.auth.JWT({ email: process.env.SERVICE_ACCOUNT_EMAIL, key: fs.readFileSync(path.join(__dirname, “../../key.pem”), ‘utf8’), scopes: [‘https://www.googleapis.com/auth/analytics.readonly’] }) const queryParams = { resource: { reportRequests: [ { viewId: process.env.VIEW_ID, dateRanges: [{startDate: ‘30daysAgo’, endDate: ‘yesterday’}], metrics: [ {expression: ‘ga:pageviews’}, {expression: ‘ga:exits’} ], dimensions: [ {name: ‘ga:previousPagePath’}, {name: ‘ga:pagePath’} ], orderBys: [ {fieldName: ‘ga:previousPagePath’, sortOrder: ‘ASCENDING’}, {fieldName: ‘ga:pageviews’, sortOrder: ‘DESCENDING’}

Slide 66

Slide 66

35 exports.handler= =async asyncfunction() (event, context, callback) => { { 36 module.exports try { 37 await authClient.authorize() 38 const analytics = google.analyticsreporting({ 39 version: ‘v4’, 40 auth: authClient 41 }) 42 const response = await analytics.reports.batchGet(queryParams) 43 let [report] = response.data.reports 44 45 const aggregatePages = async (pages) => { 46 … 47 return predictions 48 } 49 50 const aggPages = await aggregatePages(report); 51 52 return { 53 statusCode: return aggPages200, 54 headers: { { } catch (err) 55 “Access-Control-Allow-Origin”: “*”, console.log(“err”) 56 “Access-Control-Allow-Headers”: “Content-Type” return err 57 } }, 58 } body: JSON.stringify({ “results”: aggPages }) 59 } 60 61 62 63

Slide 67

Slide 67

35 36 exports.handler = async (event, context, callback) => { try { 37 await authClient.authorize() 38 const analytics = google.analyticsreporting({ 39 version: ‘v4’, 40 auth: authClient 41 }) 42 const response = await analytics.reports.batchGet(queryParams) 43 let [report] = response.data.reports 44 45 const aggregatePages = async (pages) => { 46 … 47 return predictions 48 } 49 50 const aggPages = await aggregatePages(report); 51 52 return { 53 statusCode: 200, 54 headers: { 55 “Access-Control-Allow-Origin”: “*”, 56 “Access-Control-Allow-Headers”: “Content-Type” 57 }, 58 body: JSON.stringify({ “results”: aggPages }) 59 } catch (err) { 60 console.log(“err”) 61 return err 62 } 63 }

Slide 68

Slide 68

DEMO

Slide 69

Slide 69

Threshold

Slide 70

Slide 70

Bandwidth Threshold Recommended Slow 2G 0.9 DNS/Link Prefetch 2G 0.9 DNS/Link Prefetch 3G 0.5 DNS/Link Prefetch 4G 0.2 Link Prefetch Data Saver 0 Null

Slide 71

Slide 71

Hint Cost (if wrong) Benefit (if correct) DNS Prefetch Very low Low Link Prefetch Mid high High Prerender Very high Very high

Slide 72

Slide 72

! Open Sauce Home About Us Sauce Code Contact

Slide 73

Slide 73

” Open Sauce Home About Us Sauce Code Sorry, Americans. Contact Blog

Slide 74

Slide 74

THANKS @shortdiv noti.st/shortdiv https://github.com/shortdiv/cant-dutch-this/