A presentation at #PerfMatters by Andrew Scheuermann
One Number, Multiple Metrics A Multidimensional Approach to Performance Andrew Scheuermann / @a15n_
Who Am I? • Management degree + bootcamp • Senior Engineer at Airbnb • Cook, read, board games
⚡
“ Load is not a single moment in time — it’s an experience that no one metric can fully capture. There are multiple moments during the load experience that can affect whether a user perceives it as ‘fast’ or ‘slow’. - developers.google.com
Time To Airbnb Interactive Reached here
Time To Airbnb Interactive Reached here
Time To Airbnb Interactive Reached here Ignored
~Lighthouse, with real user data
New System Requirements Real User Monitoring Measure Multiple Moments Comprehensible
Metric Requirements • User-centric – Is it happening? – Is it useful? – Is it usable? – Is it delightful? • Comprehensive browser coverage • Direct request + Client-side request
🖌 🖼 👇 🕐 Time To First Contentful Paint Time To First Meaningful Paint First Input Delay Total Blocking Time
Metrics in Action
Time To First Contentful Paint 🖌 Is it happening?
Time To First Meaningful Paint 🖼 Is it useful?
Time To First Meaningful Paint 🖼 Is it useful?
First Input Delay 👇 Is it usable?
Total Blocking Time 🕐 Is it delightful?
Multiple Metrics Total Blocking Time 🕐 Time To First Meaningful Paint 🖼 Time To First Contentful Paint 🖌 First Input Delay 👇
airbnb.com?pps=log
New System Requirements Real User Monitoring Measure Multiple Moments Comprehensible
One Number
“ That which is measured improves. That which is measured and reported improves exponentially. Karl Pearson
🖌 🖼 👇 🕐 Time To First Contentful Paint Time To First Meaningful Paint First Input Delay Total Blocking Time
5 seconds 3 seconds (67% progress) Fast 10 seconds (28% progress) Slow
New System Requirements Real User Monitoring Measure Multiple Moments Comprehensible
Page Performance Score
PPS Real user data 100% browser coverage Lighthouse 0-100 Score User-centric metrics Correlation Lab data Performance advice
Case Studies
Service Worker
• First time visitors download the service worker • Repeat visitors saw the App Shell • A/B tested
Seconds Default Service Worker 0 3.5 4.5 6.5 9 11
Time To First Contentful Paint 20% faster Time To First Meaningful Paint 67% slower Page Views 1.1% higher Bookings 0.6% higher
New Calendar
Old New
Google Map
Client-Side Route to Wishlists
Direct Request Client Request
Time To First Contentful Paint 20% faster Time To First Meaningful Paint 4% faster First Input Delay 27% faster Total Blocking Time 39% lower Page Views 6% higher
Learnings
Speed Seconds Good <3 Average 3-5 Slow
5
Speed Seconds Score Good <3
70 Average 3-5 60-69 Slow 5 <60
Meta Metrics • First Input Delay – Time To Initial Hydrate – Time To First Input • Total Blocking Time – Number of Blocking Tasks – Longest Blocking Task Duration – Longest Blocking Task Index – Page Duration
KEY TAKEAWAYS Load is not a single moment Measure multiple, user-centric metrics Real User Monitoring Combine into a single number
Resources Blog post: https://web.dev/user-centric-performance-metrics/ @a15n_ Polyfills (open source coming soon) Score Calculator: shorturl.at/lnwBN
Users experience the speed of your site at multiple points. Whether it’s loading the site, interacting with content, or completing a multi-page flow, performance is a journey and each step needs to be recorded. We have the tools to capture all of this information, but then what? How do you turn this treasure trove of data into a comprehensive and digestible story that answers the age-old question “is my site fast?” At Airbnb we’ve answered this question with a new approach to performance measurement, the Page Performance Score.
for free. You
can too.