PREDICTIVE Prefetching
A presentation at Perf Matters in April 2020 in 7136 S Yale Ave, Tulsa, OK 74136, USA by Divya
PREDICTIVE Prefetching
Divya Tagtachian @shortdiv
PREFETCHING Prefetching TM
www.perfnow.nl https:// perfnow .nl/ 53 194.0.28. Data Transfer SYN SYN+ACK ACK
www.perfnow.nl js html css GET /index.html GET /style.css GET /script.js
www.perfnow.nl
DNS Prefetching Link Prefetching Prerendering
DNS Prefetching
<link rel=”dns-prefetch” href=”http://cdn.example.com”>
DNS Prefetching Link Prefetching Prerendering
Link Prefetching
<link rel=”prefetch” href=“http://cdnz.com/library.js”>
DNS Prefetching Link Prefetching Prerendering
Prerendering
<link rel=”prerender” href=”http://cdnz.com/about.html”>
DNS Prefetching Link Prefetching Prerendering
Hint Cost (if wrong) Benefit (if correct) DNS Prefetch Very low Low Link Prefetch Mid high High Prerender Very high Very high
https://duckduckgo.com/?q=wtf+are+war+fries&t=h_&ia=web wtf are war fries A crash course on The Netherlands’ War Fries Call ‘em what you will - friet, friets, patat, Vlaamse Frieten - but one thing is sure, the Dutch can’t seem to get enough of ‘em! In fact, they love ‘em so much they eat over 41 million kilos of these tasty guys per year! (But Dutch people don’t just stop at fries. Every Dutchie Recipe for War Fries The perfect fries are the Belgium/Dutch (most of the credits to Belgium). They are big friesand if they are prepared good, the outside is a little crunchy, but the inside is soft. When you bite, you first go to a little crispy part and then the soft potato, what is the main part of the frie. Mc donalds fries, are french fries. Really thin. 2
username •••••••••••
https://cant-dutch-this.netlify.com 2 Can’t Dutch This Home About Us Menu Contact Blog Go dutch, you won’t regret it. 30 Sept 2018 WTF is war fries? 24 Aug 2018 Fries, the dutch way. TGI Friet day 4 June 2018 1 May 2018
https://cant-dutch-this.netlify.com 4 2 Can’t Dutch This Home About Us Menu Contact Blog Go dutch, you won’t regret it. 30 Sept 2018 WTF is war fries? 24 Aug 2018 Fries, the dutch way. TGI Friet day 4 June 2018 1 May 2018
https://cant-dutch-this.netlify.com 1 2 Can’t Dutch This Home About Us Menu Contact Blog Go dutch, you won’t regret it. 30 Sept 2018 WTF is war fries? 24 Aug 2018 Fries, the dutch way. TGI Friet day 4 June 2018 1 May 2018
SPECULATIV PreDictive
PREDICTIVE PreDictive
???
Today Tomorrow Overcast Rain Overcast 0.75 0.25 Rain 1 0
0.75 0 0.25 Overcast Rainy 1
/about /menu /about /contact /about /contact /about /menu /menu /contact /menu / /menu /about /menu /
/about /menu /about /menu /about /contact 50% 50% /about /contact /menu /contact 25% /menu /about 25% /menu / /menu / 50%
1 { 2 “columnHeader”: { 3 “dimensions”: [ 4 “ga:previousPagePath”, 5 “ga:pagePath” 6 ], 7 “metricHeader”: { 8 “metricHeaderEntries”: [ 9 { 10 “name”: “ga:pageviews”, 11 “type”: “INTEGER” 12 }, 13 { 14 “name”: “ga:exits”, 15 “type”: “INTEGER” 16 } 17 ] 18 } 19 }, 20 “data”: { 21 “rows”: [ 22 { 23 “dimensions”: [ 24 “(entrance)”, 25 “/” 26 ], 27 “metrics”: [ 28 {
36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 { “dimensions”: “/”, “/” ], “metrics”: [ { “values”: “4”, “1” ] } ] }, { “dimensions”: “/”, “/posts/” ], “metrics”: [ { “values”: “4”, “0” ] } ] }, { [ [ Previous Page Path Current Page Path Pageviews Exits [ [
36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 { “dimensions”: “/”, “/” ], “metrics”: [ { “values”: “4”, “1” ] } ] }, { “dimensions”: “/”, “/posts/” ], “metrics”: [ { “values”: “4”, “0” ] } ] }, { [ Previous Page Path Current Page Path [ [ [ Previous Page Path Current Page Path
1 { 2 “/”: { 3 “pagePath”: “/”, 4 “pageviews”: 6, 5 “exits”: 0, 6 “nextPageviews”: 6, 7 “nextExits”: 0, 8 “nextPages”: [ 9 { 10 “pagePath”: “/posts/”, 11 “pageviews”: 4 12 }, 13 { 14 “pagePath”: “/about/”, 15 “pageviews”: 1 16 }, 17 { 18 “pagePath”: “/contact/”, 19 “pageviews”: 1 20 } 21 ], 22 “percentExits”: 0, 23 “topNextPageProbability”: 0.6666666666666666 24 }, 25 “/posts/”: { 26 “pagePath”: “/posts/”, 27 “pageviews”: 9, 28 “exits”: 0,
1 { 2 “/”: { 3 “pagePath”: “/”, 4 “pageviews”: 6, 5 “exits”: 0, 6 “nextPageviews”: 6, 7 “nextExits”: 0, 8 “nextPages”: [ 9 { 10 “pagePath”: “/posts/”, 11 “pageviews”: 4 12 }, 13 { 14 “pagePath”: “/about/”, 15 “pageviews”: 1 16 }, 17 { 18 “pagePath”: “/contact/”, 19 “pageviews”: 1 20 } 21 ], 22 “percentExits”: 0, 23 “topNextPageProbability”: 0.6666666666666666 24 }, 25 “/posts/”: { 26 “pagePath”: “/posts/”, 27 “pageviews”: 9, 28 “exits”: 0,
1 { 2 “/”: { 3 “pagePath”: “/”, 4 “pageviews”: 6, 5 “exits”: 0, 6 “nextPageviews”: 6, 4 7 “nextExits”: 0, 8 “nextPages”: [ 6 9 { 10 “pagePath”: “/posts/”, 11 “pageviews”: 4 12 }, 13 { 14 “pagePath”: “/about/”, 15 “pageviews”: 1 16 }, 17 { 18 “pagePath”: “/contact/”, 19 “pageviews”: 1 20 } 21 ], 22 “percentExits”: 0, 23 “topNextPageProbability”: 0.6666666666666666 24 }, 25 “/posts/”: { 26 “pagePath”: “/posts/”, 27 “pageviews”: 9, 28 “exits”: 0,
1 { 2 “/”: { 3 “pagePath”: “/”, 4 “pageviews”: 6, 5 “exits”: 0, 6 “nextPageviews”: 6, 7 “nextExits”: 0, 8 “nextPages”: [ 1 9 { 10 “pagePath”: “/posts/”, 6 11 “pageviews”: 4 12 }, 13 { 14 “pagePath”: “/about/”, 15 “pageviews”: 1 16 }, 17 { 18 “pagePath”: “/contact/”, 19 “pageviews”: 1 20 } 21 ], 22 “percentExits”: 0, 23 “topNextPageProbability”: 0.6666666666666666 24 }, 25 “/posts/”: { 26 “pagePath”: “/posts/”, 27 “pageviews”: 9, 28 “exits”: 0,
PredIctIve Build Automation
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 const const const const axios = require(‘axios’); {google} = require(‘googleapis’) fs = require(‘fs’) path = require(‘path’) _data/prefetchLinks.js const authClient = new google.auth.JWT({ email: process.env.SERVICE_ACCOUNT_EMAIL, key: fs.readFileSync(path.join(__dirname, “../../key.pem”), ‘utf8’), scopes: [‘https://www.googleapis.com/auth/analytics.readonly’] }) const queryParams = { resource: { reportRequests: [ { viewId: process.env.VIEW_ID, dateRanges: [{startDate: ‘30daysAgo’, endDate: ‘yesterday’}], metrics: [ {expression: ‘ga:pageviews’}, {expression: ‘ga:exits’} ], dimensions: [ {name: ‘ga:previousPagePath’}, {name: ‘ga:pagePath’} ], orderBys: [ {fieldName: ‘ga:previousPagePath’, sortOrder: ‘ASCENDING’}, {fieldName: ‘ga:pageviews’, sortOrder: ‘DESCENDING’}
11 _data/prefetchLinks.js 12 const queryParams = { resource: { 13 reportRequests: [ 14 { 15 viewId: process.env.VIEW_ID, 16 dateRanges: [{startDate: ‘30daysAgo’, endDate: ‘yesterday’}], 17 metrics: [ 18 {expression: ‘ga:pageviews’}, 19 {expression: ‘ga:exits’} 20 ], 21 dimensions: [ 22 {name: ‘ga:previousPagePath’}, 23 {name: ‘ga:pagePath’} 24 ], 25 orderBys: [ 26 {fieldName: ‘ga:previousPagePath’, sortOrder: ‘ASCENDING’}, 27 {fieldName: ‘ga:pageviews’, sortOrder: ‘DESCENDING’} 28 ], 29 pageSize: 10000 30 } 31 ] 32 } 33 34 } 35 36 module.exports = async function() { try { 37 await authClient.authorize() 38 const analytics = google.analyticsreporting({ 39
} 33 _data/prefetchLinks.js 34 } 35 36 module.exports = async function() { try { 37 await authClient.authorize() 38 const analytics = google.analyticsreporting({ 39 version: ‘v4’, 40 auth: authClient 41 }) 42 const response = await analytics.reports.batchGet(queryParams) 43 let [report] = response.data.reports 44 45 const aggregatePages = async (pages) => { 46 … 47 return predictions 48 } 49 50 const aggPages = await aggregatePages(report); 51 52 return aggPages 53 } catch (err) { 54 console.log(“err”) 55 return err 56 } 57 58 } 59 60 https://calendar.perfplanet.com/2019/the-subtle-art-of-predictive-prefetching/ 61
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 { }, { }, { }, { }, { }, { “pagePath”: “/posts/”, “nextPagePath”: “/posts/thirdpost/”, “nextPageCertainty”: 0.3333333333333333 “pagePath”: “/”, “nextPagePath: ‘/posts/”, “nextPageCertainty”: 0.6666666666666666 “pagePath”: “/posts/thirdpost/”, “nextPagePath”: “/posts/”, “nextPageCertainty”: 0.42857142857142855 “pagePath”: “/posts/secondpost/”, “nextPagePath”: “/posts/thirdpost/”, “nextPageCertainty”: 0.5 “pagePath”: “/about/”, “nextPagePath”: “/contact/”, “nextPageCertainty”: 0.5 “pagePath”: “/contact/”, “nextPagePath”: “/”,
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 { }, { }, { }, { }, { }, { “pagePath”: “/posts/”, “nextPagePath”: “/posts/thirdpost/”, “nextPageCertainty”: 0.3333333333333333 “pagePath”: “/”, “nextPagePath: ‘/posts/”, “nextPageCertainty”: 0.6666666666666666 “pagePath”: “/posts/thirdpost/”, “nextPagePath”: “/posts/”, “nextPageCertainty”: 0.42857142857142855 “pagePath”: “/posts/secondpost/”, “nextPagePath”: “/posts/thirdpost/”, “nextPageCertainty”: 0.5 “pagePath”: “/about/”, “nextPagePath”: “/contact/”, “nextPageCertainty”: 0.5 “pagePath”: “/contact/”, “nextPagePath”: “/”,
base.njk 1 <!doctype html> 2 <html lang=”en”> 3 <head> 4 <meta charset=”utf-8”> 5 <meta name=”viewport” content=”width=device-width, initial-scale=1. 6 <title>{{ renderData.title or title or metadata.title }}</title> 7 <meta name=”Description” content=”{{ renderData.description }}”> 8 <link rel=”stylesheet” href=”{{ ‘/css/index.css’ | url }}”> 9 10 {% for entry in prefetchLinks %} 11 {% if entry.pagePath == page.url %} 12 <link rel=”prefetch” href=”{{ entry | prefetchNextURL }}”> 13 {% endif %} 14 {%- endfor -%} 15 </head> 16 17 <body> 18 {{ content | safe }} 19 </body> 20 21 </html> 22 23 24 25 26 27 28
.eleventy.js 1 module.exports = function(eleventyConfig) { 2 const THRESHOLD = 0.5 3 4 5 eleventyConfig.addFilter(‘prefetchNextURL’, (entry, url) => { 6 // check threshold // 7 if (entry.nextPageCertainty > THRESHOLD) { 8 return entry.nextPagePath 9 } 10 }) 11 12 … 13 } 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28
DEMO
Guess js
PREDICTIVE NAïve NAÏVE
INTELLIGENT PREDICTIVE Intelligent
/about /menu /about /menu /about /menu /about /contact /menu /contact /menu / /menu / /menu /about
/ /about /menu /contact /about /menu / /about /menu / /about /contact /contact /menu /contact / /menu / / /menu / /about /menu /about
GDPR
BuildTime BUILDTIME PREDICTIVE
REALTIME RealTime Throttling PREDICTIVE
PredIctIve Serverless Functions
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 const const const const axios = require(‘axios’); {google} = require(‘googleapis’) fs = require(‘fs’) path = require(‘path’) const authClient = new google.auth.JWT({ email: process.env.SERVICE_ACCOUNT_EMAIL, key: fs.readFileSync(path.join(__dirname, “../../key.pem”), ‘utf8’), scopes: [‘https://www.googleapis.com/auth/analytics.readonly’] }) const queryParams = { resource: { reportRequests: [ { viewId: process.env.VIEW_ID, dateRanges: [{startDate: ‘30daysAgo’, endDate: ‘yesterday’}], metrics: [ {expression: ‘ga:pageviews’}, {expression: ‘ga:exits’} ], dimensions: [ {name: ‘ga:previousPagePath’}, {name: ‘ga:pagePath’} ], orderBys: [ {fieldName: ‘ga:previousPagePath’, sortOrder: ‘ASCENDING’}, {fieldName: ‘ga:pageviews’, sortOrder: ‘DESCENDING’}
35 exports.handler= =async asyncfunction() (event, context, callback) => { { 36 module.exports try { 37 await authClient.authorize() 38 const analytics = google.analyticsreporting({ 39 version: ‘v4’, 40 auth: authClient 41 }) 42 const response = await analytics.reports.batchGet(queryParams) 43 let [report] = response.data.reports 44 45 const aggregatePages = async (pages) => { 46 … 47 return predictions 48 } 49 50 const aggPages = await aggregatePages(report); 51 52 return { 53 statusCode: return aggPages200, 54 headers: { { } catch (err) 55 “Access-Control-Allow-Origin”: “*”, console.log(“err”) 56 “Access-Control-Allow-Headers”: “Content-Type” return err 57 } }, 58 } body: JSON.stringify({ “results”: aggPages }) 59 } 60 61 62 63
35 36 exports.handler = async (event, context, callback) => { try { 37 await authClient.authorize() 38 const analytics = google.analyticsreporting({ 39 version: ‘v4’, 40 auth: authClient 41 }) 42 const response = await analytics.reports.batchGet(queryParams) 43 let [report] = response.data.reports 44 45 const aggregatePages = async (pages) => { 46 … 47 return predictions 48 } 49 50 const aggPages = await aggregatePages(report); 51 52 return { 53 statusCode: 200, 54 headers: { 55 “Access-Control-Allow-Origin”: “*”, 56 “Access-Control-Allow-Headers”: “Content-Type” 57 }, 58 body: JSON.stringify({ “results”: aggPages }) 59 } catch (err) { 60 console.log(“err”) 61 return err 62 } 63 }
DEMO
Threshold
Bandwidth Threshold Recommended Slow 2G 0.9 DNS/Link Prefetch 2G 0.9 DNS/Link Prefetch 3G 0.5 DNS/Link Prefetch 4G 0.2 Link Prefetch Data Saver 0 Null
Hint Cost (if wrong) Benefit (if correct) DNS Prefetch Very low Low Link Prefetch Mid high High Prerender Very high Very high
! Open Sauce Home About Us Sauce Code Contact
” Open Sauce Home About Us Sauce Code Sorry, Americans. Contact Blog
THANKS @shortdiv https://noti.st/shortdiv/jsHO3Z/predictive-prefetching https://github.com/shortdiv/cant-dutch-this/
