Living On the Edge

A presentation at CodeBEAM BR in August 2021 in by Brooklyn Zelenka

Slide 1

Slide 1

Living On the Edge ๐ŸŒโšกA Brave New (Post-Cloud) World ๐Ÿ›ฐโœจ

Slide 2

Slide 2

Slide 3

Slide 3

[โ€ฆ] by 2025, 75% of data will be processed outside the traditional data centre or cloud ~ IBM (paraphrasing a Gartner study)

Slide 4

Slide 4

Brooklyn Zelenka @expede

Slide 5

Slide 5

Brooklyn Zelenka @expede โ€ข CTO at Fission โ€ข https://fission.codes โ€ข Infra & browser SDK for edge apps โ€ข PLT, distributed systems โ€ข Specs: DIF, ETH Core โ€ข Meetups: Vancouver FP, Code & Co ee YVR ff โ€ข Libs: Witchcraft, Exceptional, Rescue, &c

Slide 6

Slide 6

WebNative ๐Ÿš€ Meta ๐Ÿ”ฎ โ€ข R&D from Fission & others โ€ข Future looking / an emerging area โ€ข Interesting tech, very exciting โ€ข โ€ฆbut not all problems solved today โ€ข Some advantages to flexible tech even before the network changes โ€ข Universal Hostless Substrate (2019)

Slide 7

Slide 7

WebNative ๐Ÿš€ Meta ๐Ÿ”ฎ โ€ข R&D from Fission & others โ€ข Future looking / an emerging area โ€ข Interesting tech, very exciting โ€ข โ€ฆbut not all problems solved today โ€ข Some advantages to flexible tech even before the network changes โ€ข Universal Hostless Substrate (2019)

Slide 8

Slide 8

WebNative ๐Ÿš€ Fission R&D โ€ข Local first โ€ข Edge only โ€ข No servers โ€ข Fully distributed โ€ข Encrypted at Rest, E2EE โ€ข User owned data @FISSIONCodes

Slide 9

Slide 9

WebNative ๐Ÿš€ Overview Part I: Motivation Part II: On the Edge How we got here Why BEAM What changed? Primer All About Data A Few Techniques

Slide 10

Slide 10

Part I Motivation ๐ŸŽญ

Slide 11

Slide 11

Motivation ๐ŸŽญ 90s Web

Slide 12

Slide 12

Motivation ๐ŸŽญ 90s Web ๐Ÿ’ ๐Ÿ–ฅ

Slide 13

Slide 13

Motivation ๐ŸŽญ 90s Web ๐Ÿ’ ๐Ÿ–ฅ ๐Ÿข

Slide 14

Slide 14

Motivation ๐ŸŽญ 90s Web ๐Ÿ’ ๐Ÿ–ฅ ๐Ÿข ๐Ÿ—ƒ

Slide 15

Slide 15

Motivation ๐ŸŽญ 90s Web ๐Ÿ’ ๐Ÿ–ฅ ๐Ÿข โš™ ๐Ÿ—ƒ

Slide 16

Slide 16

Motivation ๐ŸŽญ 90s Web ๐Ÿ’ ๐Ÿ–ฅ ๐Ÿข โš™ ๐Ÿ’ช ๐Ÿ—ƒ

Slide 17

Slide 17

Motivation ๐ŸŽญ 90s Web ๐Ÿ’ ๐Ÿ–ฅ ๐Ÿข โš™ ๐Ÿ’ช ๐Ÿ—ƒ

Slide 18

Slide 18

Motivation ๐ŸŽญ 90s Web ๐Ÿ’ ๐Ÿ–ฅ ๐Ÿข โš™ ๐Ÿ’ช ๐Ÿ—ƒ

Slide 19

Slide 19

Motivation ๐ŸŽญ 90s Web ๐Ÿ’ ๐Ÿ–ฅ ๐Ÿข โš™ ๐Ÿ’ช ๐Ÿ—ƒ

Slide 20

Slide 20

Motivation ๐ŸŽญ 90s Web ๐Ÿ’ ๐Ÿ’ ๐Ÿ’ ๐Ÿ–ฅ ๐Ÿข ๐Ÿ–ฅ ๐Ÿ–ฅ โš™ ๐Ÿ’ช ๐Ÿ—ƒ

Slide 21

Slide 21

Motivation ๐ŸŽญ 90s Web ๐Ÿ’ ๐Ÿ’ ๐Ÿ’ ๐Ÿ–ฅ ๐Ÿข ๐Ÿ–ฅ ๐Ÿ–ฅ โš™ ๐Ÿ’ช ๐Ÿ” ๐Ÿ—ƒ

Slide 22

Slide 22

Motivation ๐ŸŽญ Scaling Up ๐Ÿ’ ๐Ÿ’ ๐Ÿ’ ๐Ÿ–ฅ ๐Ÿ–ฅ ๐Ÿ–ฅ ๐Ÿ™ โš™ โš™ โš™ ๐Ÿ” ๐Ÿ” ๐Ÿ” ๐Ÿ—ƒ ๐Ÿ—ƒ ๐Ÿ—ƒ

Slide 23

Slide 23

Motivation ๐ŸŽญ Scaling Up ๐Ÿ’ ๐Ÿ’ ๐Ÿ’ ๐Ÿ–ฅ ๐Ÿ–ฅ ๐Ÿ–ฅ ๐Ÿ™ โš™ โš™ โš™ ๐Ÿ” ๐Ÿ” ๐Ÿ” ๐Ÿ—ƒ ๐Ÿ—ƒ ๐Ÿ—ƒ

Slide 24

Slide 24

Motivation ๐ŸŽญ Scaling Up ๐Ÿ’ ๐Ÿ’ ๐Ÿ’ ๐Ÿ–ฅ ๐Ÿ–ฅ ๐Ÿ–ฅ โš™ ๐Ÿ—ƒ โ€œThe Cloudโ€ ๐Ÿ” ๐Ÿ™โ˜โ˜โ˜โ˜โ˜โ˜ โš™ ๐Ÿ—ƒ ๐Ÿ” โš™ ๐Ÿ—ƒ ๐Ÿ”

Slide 25

Slide 25

Motivation ๐ŸŽญ Abstracting ๐Ÿ’ ๐Ÿ’ ๐Ÿ’ ๐Ÿ–ฅ ๐Ÿ–ฅ ๐Ÿ–ฅ โš™ ๐Ÿ—ƒ โ€œServerlessโ€ ๐Ÿ” ๐Ÿ™ โš™ ๐Ÿ—ƒ ฮปฮปฮปฮปฮปฮปฮปฮปฮปฮปฮปฮปฮป ๐Ÿ” โš™ ๐Ÿ—ƒ ๐Ÿ”

Slide 26

Slide 26

โ€ฆand so it was for many yearsโ€ฆ

Slide 27

Slide 27

โ€ฆand so it was for many yearsโ€ฆ ๐Ÿฆ–โ˜„๐ŸŒ‹๐ŸŒพ๐Ÿฐ๐Ÿข๐Ÿš€

Slide 28

Slide 28

Motivation ๐ŸŽญ Natural Consequences ๐Ÿƒ

Slide 29

Slide 29

Motivation ๐ŸŽญ Natural Consequences ๐Ÿƒ โ€ข Server-focus โ€ข More stack to learn โ€ข DevOps, Docker, k8s

Slide 30

Slide 30

Motivation ๐ŸŽญ Natural Consequences ๐Ÿƒ โ€ข Server-focus โ€ข More stack to learn โ€ข DevOps, Docker, k8s โ€ข Single source of truth โ€ข i.e. โ€œthe databaseโ€

Slide 31

Slide 31

Motivation ๐ŸŽญ Natural Consequences ๐Ÿƒ โ€ข Server-focus โ€ข More stack to learn โ€ข DevOps, Docker, k8s โ€ข Single source of truth โ€ข i.e. โ€œthe databaseโ€ โ€ข Client concerned with data sync

Slide 32

Slide 32

Motivation ๐ŸŽญ Natural Consequences ๐Ÿƒ โ€ข Server-focus โ€ข More stack to learn โ€ข DevOps, Docker, k8s โ€ข Single source of truth โ€ข i.e. โ€œthe databaseโ€ โ€ข Client concerned with data sync โ€ข AWS, Azure, GCP

Slide 33

Slide 33

Motivation ๐ŸŽญ Natural Consequences ๐Ÿƒ โ€ข Server-focus โ€ข More stack to learn โ€ข DevOps, Docker, k8s โ€ข Single source of truth โ€ข i.e. โ€œthe databaseโ€ โ€ข Client concerned with data sync f โ€ข AWS, Azure, GCP Source: 2021 Stack Over low Developer Survey

Slide 34

Slide 34

Motivation ๐ŸŽญ Sending a โ€œDirectโ€ Message

Slide 35

Slide 35

Motivation ๐ŸŽญ Sending a โ€œDirectโ€ Message

Slide 36

Slide 36

Motivation ๐ŸŽญ Sending a โ€œDirectโ€ Message

Slide 37

Slide 37

Motivation ๐ŸŽญ Sending a โ€œDirectโ€ Message

Slide 38

Slide 38

Motivation ๐ŸŽญ What Even is a โ€œServerโ€? ๐Ÿง

  1. Auth gatekeeper (because multi-tenant data) 2. Resource availability 3. Out-of-band compute (e.g. batch tasks, cron, OLAP)

Slide 39

Slide 39

Motivation ๐ŸŽญ Network Topology ๐Ÿง 

Slide 40

Slide 40

Motivation ๐ŸŽญ Network Topology ๐Ÿง  โš™

Slide 41

Slide 41

Motivation ๐ŸŽญ Network Topology ๐Ÿง  โš™ ๐Ÿ‘จ๐ŸŽค ๐Ÿ‘ฉ๐ŸŒพ Centralized ๐Ÿง‘๐ŸŽจ

Slide 42

Slide 42

Motivation ๐ŸŽญ Network Topology ๐Ÿง  ๐Ÿ’พ โš™ ๐Ÿ‘จ๐ŸŽค ๐Ÿ‘ฉ๐ŸŒพ Centralized ๐Ÿง‘๐ŸŽจ โš™ ๐Ÿค– ๐Ÿ› 

Slide 43

Slide 43

Motivation ๐ŸŽญ Network Topology ๐Ÿง  ๐Ÿ’พ โš™ ๐Ÿ‘จ๐ŸŽค ๐Ÿ‘ฉ๐ŸŒพ Centralized โš™ ๐Ÿค– ๐Ÿ™ ๐Ÿง‘๐ŸŽจ ๐Ÿ› 

Slide 44

Slide 44

Motivation ๐ŸŽญ Network Topology ๐Ÿง  ๐Ÿ’พ โš™ โš™ ๐Ÿ‘จ๐ŸŽค ๐Ÿ‘ฉ๐ŸŒพ Centralized ๐Ÿค– ๐Ÿ›  ๐Ÿ™ ๐Ÿง‘๐ŸŽจ ๐Ÿง‘๐ŸŽจ ๐Ÿ‘ฉ๐ŸŒพ ๐Ÿ‘จ๐ŸŽค Hub (e.g. gateway or load balanced)

Slide 45

Slide 45

Motivation ๐ŸŽญ Network Topology ๐Ÿง  ๐Ÿ’พ โš™ โš™ ๐Ÿ‘จ๐ŸŽค ๐Ÿ‘ฉ๐ŸŒพ Centralized โš™ ๐Ÿค– ๐Ÿ›  ๐Ÿ™ ๐Ÿง‘๐ŸŽจ ๐Ÿง‘๐ŸŽจ ๐Ÿ‘ฉ๐ŸŒพ ๐Ÿ‘จ๐ŸŽค Hub (e.g. gateway or load balanced)

Slide 46

Slide 46

Motivation ๐ŸŽญ Network Topology ๐Ÿง  ๐Ÿ’พ โš™ โš™ ๐Ÿ‘จ๐ŸŽค ๐Ÿ‘ฉ๐ŸŒพ Centralized โš™ ๐Ÿค– ๐Ÿ›  ๐Ÿ™ ๐Ÿง‘๐ŸŽจ ๐Ÿง‘๐ŸŽจ ๐Ÿ‘ฉ๐ŸŒพ ๐Ÿ‘จ๐ŸŽค Hub (e.g. gateway or load balanced) ๐Ÿค– ๐Ÿ’พ

Slide 47

Slide 47

Motivation ๐ŸŽญ Network Topology ๐Ÿง  ๐Ÿ’พ โš™ โš™ ๐Ÿ‘จ๐ŸŽค ๐Ÿ‘ฉ๐ŸŒพ Centralized โš™ ๐Ÿค– ๐Ÿ›  ๐Ÿ™ ๐Ÿง‘๐ŸŽจ ๐Ÿง‘๐ŸŽจ ๐Ÿ‘ฉ๐ŸŒพ ๐Ÿง‘๐ŸŽจ ๐Ÿ‘จ๐ŸŽค Hub (e.g. gateway or load balanced) ๐Ÿค– ๐Ÿ’พ โš™ ๐Ÿ‘จ๐ŸŽค

Slide 48

Slide 48

Motivation ๐ŸŽญ Network Topology ๐Ÿง  ๐Ÿ’พ โš™ โš™ ๐Ÿ‘จ๐ŸŽค ๐Ÿ‘ฉ๐ŸŒพ Centralized โš™ ๐Ÿค– ๐Ÿ›  ๐Ÿ™ ๐Ÿง‘๐ŸŽจ ๐Ÿง‘๐ŸŽจ ๐Ÿ‘ฉ๐ŸŒพ ๐Ÿง‘๐ŸŽจ ๐Ÿ‘จ๐ŸŽค Hub (e.g. gateway or load balanced) ๐Ÿง‘๐ŸŽจ ๐Ÿค– ๐Ÿ’พ โš™ ๐Ÿ‘จ๐ŸŽค ๐Ÿ‘ฉ๐ŸŒพ Hierarchical or pipelined

Slide 49

Slide 49

Motivation ๐ŸŽญ Network Topology ๐Ÿง  ๐Ÿ’พ โš™ โš™ ๐Ÿ‘จ๐ŸŽค ๐Ÿ‘ฉ๐ŸŒพ Centralized โš™ ๐Ÿค– ๐Ÿ›  ๐Ÿ™ ๐Ÿง‘๐ŸŽจ ๐Ÿง‘๐ŸŽจ ๐Ÿ‘ฉ๐ŸŒพ ๐Ÿง‘๐ŸŽจ ๐Ÿ‘จ๐ŸŽค Hub (e.g. gateway or load balanced) ๐Ÿง‘๐ŸŽจ ๐Ÿค– ๐Ÿ’พ โš™ ๐Ÿ‘จ๐ŸŽค ๐Ÿ‘ฉ๐ŸŒพ Hierarchical or pipelined

Slide 50

Slide 50

A Challenger Emerges A New Environment ๐Ÿ›ฐ

Slide 51

Slide 51

New Environment ๐Ÿ›ฐ New Assumptions โ€ข Powerful client devices (e.g. M1 chips, smartphones, IoT) โ€ข Latency is the bottleneck โ€ข Mobile (i.e. smartphone) use only growing โ€ข Lose connection, drop when switching towers โ€ข Do more with the existing physical network โ€ข Not unlike how Mooreโ€™s Law lead to more parallelism

Slide 52

Slide 52

New Environment ๐Ÿ›ฐ New Biz Who Dis?

Slide 53

Slide 53

New Environment ๐Ÿ›ฐ New Biz Who Dis? โ€ข Paradigm shift means new opportunities

Slide 54

Slide 54

New Environment ๐Ÿ›ฐ New Biz Who Dis? โ€ข Paradigm shift means new opportunities โ€ข 5G networks & Starlink โ€ข Put an edge PoP right on the base station โ€ข Low-latency compute across the street

Slide 55

Slide 55

New Environment ๐Ÿ›ฐ New Biz Who Dis? โ€ข Paradigm shift means new opportunities โ€ข 5G networks & Starlink โ€ข Put an edge PoP right on the base station โ€ข Low-latency compute across the street โ€ข Edge PoPs in retail stores (yes really) โ€ข 90% of Americans live <16km from a Walmart โ€ข Walmart has lots of floor space โ€ข Add servers to Walmart = Walmart Edge

Slide 56

Slide 56

A New Environment Low Latency ๐Ÿ‡

Slide 57

Slide 57

Low Latency ๐Ÿ‡ Latency is a Physical Barrier ๐Ÿšง โ€ข Speed of light / speed of causality โ€ข <40ms = edge dominates โ€ข 8ms is ideal โ€ข Ultra Reliable Low Latency (URLLC)

Slide 58

Slide 58

Low Latency ๐Ÿ‡ Latency is a Physical Barrier ๐Ÿšง โ€ข Speed of light / speed of causality โ€ข <40ms = edge dominates โ€ข 8ms is ideal โ€ข Ultra Reliable Low Latency (URLLC) f Source: Ericsson http://cscn2017.ieee-cscn.org/ iles/2017/08/Janne_Peisa_Ericsson_CSCN2017.pdf

Slide 59

Slide 59

Low Latency ๐Ÿ‡ Spherical Cow Assumption ๐Ÿฎ โ€ข No compute, straight line, in a vacuum, guaranteed delivery, etc โ€ข 40ms โ€ข Sรฃo Paulo โžก NYC, Vancouver, Stockholm โ€ข Sรฃo Paulo โŒ Sidney, Tokyo, Seoul Credit: Keenan Crane http://www.cs.cmu.edu/~kmcrane/Projects/ModelRepository/

Slide 60

Slide 60

Low Latency ๐Ÿ‡ Spherical Cow Assumption ๐Ÿฎ โ€ข No compute, straight line, in a vacuum, guaranteed delivery, etc โ€ข 40ms โ€ข Sรฃo Paulo โžก NYC, Vancouver, Stockholm โ€ข Sรฃo Paulo โŒ Sidney, Tokyo, Seoul Credit: Keenan Crane http://www.cs.cmu.edu/~kmcrane/Projects/ModelRepository/

Slide 61

Slide 61

Low Latency ๐Ÿ‡ What 8ms Looks Like

Slide 62

Slide 62

Low Latency ๐Ÿ‡ What 8ms Looks Like Montevideo โžก Rio de Janeiro Ideal Vacuum

Slide 63

Slide 63

Low Latency ๐Ÿ‡ What 8ms Looks Like Montevideo โžก Rio de Janeiro Ideal Vacuum Brasilia ๐Ÿ” Salvador Ideal Vacuum

Slide 64

Slide 64

Low Latency ๐Ÿ‡ What 8ms Looks Like Montevideo โžก Rio de Janeiro Ideal Vacuum Brasilia ๐Ÿ” Salvador Ideal Vacuum Brasilia ๐Ÿ” Barreiras Ideal Fiber

Slide 65

Slide 65

Low Latency ๐Ÿ‡ Causal Islands ๐Ÿ–๐Ÿ

Slide 66

Slide 66

Low Latency ๐Ÿ‡ Causal Islands ๐Ÿ–๐Ÿ

Slide 67

Slide 67

Low Latency ๐Ÿ‡ Causal Islands ๐Ÿ–๐Ÿ

Slide 68

Slide 68

Low Latency ๐Ÿ‡ Causal Islands ๐Ÿ–๐Ÿ

Slide 69

Slide 69

Low Latency ๐Ÿ‡ Light Cone & Relativistic Ordering

Slide 70

Slide 70

Low Latency ๐Ÿ‡ Light Cone & Relativistic Ordering Source: Duesentrieb via Wikimedia Commons

Slide 71

Slide 71

Turning Up High Volume ๐ŸŒŠ

Slide 72

Slide 72

High Volume ๐ŸŒŠ Unprecedented Volume ๐Ÿฆ– โ€ข We have high scale NOW? Only more devices & usage in the future! โ€ข Sensors everywhere: IoT devices, continuous health data โ€ข Geospatial data (e.g. autonomous vehicles, XR)

Slide 73

Slide 73

High Volume ๐ŸŒŠ Feedback Cycle Source: Microsoft โ€ข Remote surgery โ€ข Extended reality โ€ข Location transparency โ€ข Competitive cloud gaming Source: YouTube, South China Morning Post โ€ข Realtime manufacturing โ€ข Continuous ML training Source: Google & Bungie

Slide 74

Slide 74

Slide 75

Slide 75

Sensor data explosion will kill the cloud. Sensors will produce massive amounts of data, but the existing infrastructure will not be able to handle the volumes or the rates [โ€ฆ] We are absolutely going to return to a peer-to-peer computing model [โ€ฆ] not unlike the distributed computing model We are going to move to a world of data-centric programming. ~ a16z, โ€œThe End of Cloud Computingโ€

Slide 76

Slide 76

High Volume ๐ŸŒŠ Edge Absorbs Cloud (and MEC)

Slide 77

Slide 77

High Volume ๐ŸŒŠ Edge Absorbs Cloud (and MEC) ๐Ÿคณ

Slide 78

Slide 78

High Volume ๐ŸŒŠ Edge Absorbs Cloud (and MEC) ๐Ÿคณ ๐Ÿ—ผ ๐Ÿ’พโš™

Slide 79

Slide 79

High Volume ๐ŸŒŠ Edge Absorbs Cloud (and MEC) ๐Ÿคณ ๐Ÿ—ผ ๐Ÿ’พโš™ ๐Ÿข ๐Ÿ’พโš™

Slide 80

Slide 80

High Volume ๐ŸŒŠ Edge Absorbs Cloud (and MEC) ๐Ÿคณ ๐Ÿ›ฐ ๐Ÿ›ฐ ๐Ÿ—ผ ๐Ÿข ๐Ÿ’พโš™ ๐Ÿ’พโš™

Slide 81

Slide 81

High Volume ๐ŸŒŠ Edge Absorbs Cloud (and MEC) ๐Ÿ›ฐ ๐Ÿคณ ๐Ÿ—ผ ๐Ÿ’พโš™ ๐Ÿ›ฐ ๐Ÿ›ฐ ๐Ÿข โ˜ ๐Ÿ’พโš™ ๐Ÿ’พโš™ โš™ ๐Ÿ’พ โš™ ๐Ÿ’พ โš™ ๐Ÿ’พ ๐Ÿ’พโš™ โš™

Slide 82

Slide 82

High Volume ๐ŸŒŠ Edge Absorbs Cloud (and MEC) ๐Ÿ›ฐ Local ๐Ÿคณ First ๐Ÿ—ผ ๐Ÿ’พโš™ ๐Ÿ›ฐ ๐Ÿ›ฐ ๐Ÿข โ˜ ๐Ÿ’พโš™ ๐Ÿ’พโš™ โš™ ๐Ÿ’พ โš™ ๐Ÿ’พ โš™ ๐Ÿ’พ ๐Ÿ’พโš™ โš™

Slide 83

Slide 83

High Volume ๐ŸŒŠ Edge Absorbs Cloud (and MEC) ๐Ÿ›ฐ Local ๐Ÿคณ First Realtime, Storage, Caching, OLTP ๐Ÿ’พโš™ ๐Ÿ—ผ ๐Ÿ›ฐ ๐Ÿ›ฐ ๐Ÿข โ˜ ๐Ÿ’พโš™ ๐Ÿ’พโš™ โš™ ๐Ÿ’พ โš™ ๐Ÿ’พ โš™ ๐Ÿ’พ ๐Ÿ’พโš™ โš™

Slide 84

Slide 84

High Volume ๐ŸŒŠ Edge Absorbs Cloud (and MEC) ๐Ÿ›ฐ Local ๐Ÿคณ First Realtime, Storage, Caching, OLTP ๐Ÿ’พโš™ ๐Ÿ—ผ ๐Ÿ›ฐ Relay, Replication, Consistency, Tasks ๐Ÿข ๐Ÿ’พโš™ ๐Ÿ›ฐ โ˜ ๐Ÿ’พโš™ โš™ ๐Ÿ’พ โš™ ๐Ÿ’พ โš™ ๐Ÿ’พ ๐Ÿ’พโš™ โš™

Slide 85

Slide 85

High Volume ๐ŸŒŠ Edge Absorbs Cloud (and MEC) ๐Ÿ›ฐ Local ๐Ÿคณ First Realtime, Storage, Caching, OLTP ๐Ÿ’พโš™ ๐Ÿ—ผ ๐Ÿ›ฐ Relay, Replication, Consistency, Tasks ๐Ÿข ๐Ÿ’พโš™ ๐Ÿ›ฐ โ˜ Aggregation, Batching, Training, โš™ ๐Ÿ’พ โš™ ๐Ÿ’พ OLAP ๐Ÿ’พโš™ โš™ ๐Ÿ’พ ๐Ÿ’พโš™ โš™

Slide 86

Slide 86

What does this all mean? Consequence ๐Ÿ›ธ

Slide 87

Slide 87

Consequence ๐Ÿ›ธ New Assumptions, New Approach

Slide 88

Slide 88

Consequence ๐Ÿ›ธ New Assumptions, New Approach โ€ข New features naturally fall out of the architecture โ€ข Recognize that weโ€™re increasingly connected/networked โ€ข Local-first means network e cient (in the normal case) ffi โ€ข Data can run anywhere = commons networks

Slide 89

Slide 89

Consequence ๐Ÿ›ธ Tackling the Fallacies

Slide 90

Slide 90

Consequence ๐Ÿ›ธ Tackling the Fallacies Latency is zero Bandwidth is infinite Transport cost is zero The network is secure There is one administrator The network is reliable The network is homogeneous Topology doesnโ€™t change

Slide 91

Slide 91

Consequence ๐Ÿ›ธ Tackling the Fallacies Latency is zero Bandwidth is infinite Transport cost is zero The network is secure There is one administrator The network is reliable The network is homogeneous Topology doesnโ€™t change We need to handle 100% of these up front

Slide 92

Slide 92

Consequence ๐Ÿ›ธ Tackling the Fallacies Latency is zero Treat latency directly (speed of causality) Treat (order of causality / relativistic) Bandwidth is infinite Apps continue to work with zero bandwidth Only push when & what needed Transport cost is zero Minimize network use The network is secure Assume that the pipes are broken Direct access control There is one administrator Fine grained, delegate capabilities (OCAP) The network is reliable Time, delivery, & order independence The network is homogeneous Device agnostic Topology doesnโ€™t change atomic unit is the edge device (same like the atomic unit is the actor)

Slide 93

Slide 93

Consequence ๐Ÿ›ธ Giving Up Topological Control

Slide 94

Slide 94

Consequence ๐Ÿ›ธ Giving Up Topological Control โš™ ๐Ÿ–ฅ ๐Ÿ“ฑ ๐Ÿš™ ๐Ÿ“ฑ ๐Ÿ—ผ ๐Ÿ“ฑ ๐Ÿ’ป ๐Ÿ“ฑ ๐Ÿ“€ ๐Ÿ›ฐ

Slide 95

Slide 95

Consequence ๐Ÿ›ธ Data, Data, Data ๐Ÿ’พ

Slide 96

Slide 96

Consequence ๐Ÿ›ธ Data, Data, Data ๐Ÿ’พ โ€ข Only UI & data are essential

Slide 97

Slide 97

Consequence ๐Ÿ›ธ Data, Data, Data ๐Ÿ’พ โ€ข Only UI & data are essential โ€ข New primitives โ€ข Consistency (CRDTs, STM, Distributed Datalog) โ€ข State transfer โžก state synchronization โžก state views

Slide 98

Slide 98

Consequence ๐Ÿ›ธ Data, Data, Data ๐Ÿ’พ โ€ข Only UI & data are essential โ€ข New primitives โ€ข Consistency (CRDTs, STM, Distributed Datalog) โ€ข State transfer โžก state synchronization โžก state views โ€ข Access control needs to be inherent โ€ข OCAP & CBC methods (AKA cryptography)

Slide 99

Slide 99

Part II On the Edge ๐Ÿง—

Slide 100

Slide 100

On the Edge ๐Ÿง— Why Functional Programming โ€ข Data-oriented โ€ข Pure functions on data is just data โ€ข Shared nothing architectures โ€ข Immutability, easy concurrency โ€ข Manage complexity by being declarative โ€ข What > how โ€ข Data > process

Slide 101

Slide 101

On the Edge ๐Ÿง— Why the BEAM Specifically โ€ข Low conceptual distance from actor model to OCAP โ€ข Community experience with distributed systems โ€ข Used to building up complexity from simple parts โ€ข Weโ€™re already using a bunch of this! โ€ข e.g. Phoenix Presence ๐Ÿ‘‰ ๐Ÿ‘‰ ๐Ÿ‘‰

Slide 102

Slide 102

Slide 103

Slide 103

Whatโ€™s special about Phoenixโ€™s implementation is we have a system that applies cutting edge CS research to tackle day-to-day problems in the applications we all write. Phoenix Presence - has no single point of failure - has no single source of truth - relies entirely on the standard library with no operational dependencies - self heals ~ Chris McCord, โ€œWhat Makes Phoenix Presence Specialโ€

Slide 104

Slide 104

What if we turn Phoenix Live View Upside Down? ๐Ÿ”

Slide 105

Slide 105

On the Edge ๐Ÿง— Phoenix LiveView

Slide 106

Slide 106

On the Edge ๐Ÿง— Phoenix LiveView Users ๐Ÿ‘จ๐Ÿซ๐Ÿ‘ฉ๐Ÿญ๐Ÿง‘โš•๐Ÿ‘ท Client ๐Ÿ–ฅ WSS / REST / GraphQL โ†• Controller Logic โš™ Data Store ๐Ÿ—ƒ DevOps ๐Ÿ“ค Developer ๐Ÿ‘ฉ๐Ÿ’ป

Slide 107

Slide 107

On the Edge ๐Ÿง— Phoenix LiveView Users ๐Ÿ‘จ๐Ÿซ๐Ÿ‘ฉ๐Ÿญ๐Ÿง‘โš•๐Ÿ‘ท Client ๐Ÿ–ฅ WSS / REST / GraphQL โ†• Controller Logic โš™ Data Store ๐Ÿ—ƒ DevOps ๐Ÿ“ค Developer ๐Ÿ‘ฉ๐Ÿ’ป ๐Ÿ–ฅ ๐Ÿ’พ

Slide 108

Slide 108

On the Edge ๐Ÿง— Phoenix LiveView Users ๐Ÿ‘จ๐Ÿซ๐Ÿ‘ฉ๐Ÿญ๐Ÿง‘โš•๐Ÿ‘ท Client ๐Ÿ–ฅ โš™ WSS / REST / GraphQL โ†• Controller Logic โš™ Data Store ๐Ÿ—ƒ DevOps ๐Ÿ“ค Developer ๐Ÿ‘ฉ๐Ÿ’ป ๐Ÿ–ฅ ๐Ÿ’พ ๐Ÿ—ƒ

Slide 109

Slide 109

On the Edge ๐Ÿง— Phoenix LiveView Users ๐Ÿ‘จ๐Ÿซ๐Ÿ‘ฉ๐Ÿญ๐Ÿง‘โš•๐Ÿ‘ท Client ๐Ÿ–ฅ ๐Ÿ’พ๐Ÿ’พ๐Ÿ’พ๐Ÿ’พ ๐Ÿ—ƒ โš™ WSS / REST / GraphQL โ†• Controller Logic โš™ Data Store ๐Ÿ—ƒ DevOps ๐Ÿ“ค Developer ๐Ÿ‘ฉ๐Ÿ’ป ๐Ÿ–ฅ ๐Ÿ’พ

Slide 110

Slide 110

On the Edge ๐Ÿง— Phoenix LiveView Users ๐Ÿ‘จ๐Ÿซ๐Ÿ‘ฉ๐Ÿญ๐Ÿง‘โš•๐Ÿ‘ท Client ๐Ÿ–ฅ ๐Ÿ’พ๐Ÿ’พ๐Ÿ’พ๐Ÿ’พ ๐Ÿ—ƒ โš™ WSS / REST / GraphQL โ†• Controller Logic โš™ Data Store ๐Ÿ—ƒ DevOps ๐Ÿ“ค Developer ๐Ÿ‘ฉ๐Ÿ’ป ๐Ÿ–ฅ ๐Ÿ’พ

Slide 111

Slide 111

On the Edge ๐Ÿง— Phoenix LiveView Users ๐Ÿ‘จ๐Ÿซ๐Ÿ‘ฉ๐Ÿญ๐Ÿง‘โš•๐Ÿ‘ท Client ๐Ÿ–ฅ ๐Ÿ’พ๐Ÿ’พ๐Ÿ’พ๐Ÿ’พ ๐Ÿ—ƒ โš™ WSS / REST / GraphQL โ†• Controller Logic โš™ Data Store ๐Ÿ—ƒ DevOps ๐Ÿ“ค Developer ๐Ÿ‘ฉ๐Ÿ’ป ๐Ÿ–ฅ ๐Ÿ’พ

Slide 112

Slide 112

On the Edge ๐Ÿง— Phoenix LiveView Users ๐Ÿ‘จ๐Ÿซ๐Ÿ‘ฉ๐Ÿญ๐Ÿง‘โš•๐Ÿ‘ท Client ๐Ÿ–ฅ ๐Ÿ’พ๐Ÿ’พ๐Ÿ’พ๐Ÿ’พ ๐Ÿ—ƒ โš™ WSS / REST / GraphQL โ†• Controller Logic โš™ Data Store ๐Ÿ—ƒ DevOps ๐Ÿ“ค Developer ๐Ÿ‘ฉ๐Ÿ’ป ๐Ÿ–ฅ ๐Ÿ’พ

Slide 113

Slide 113

On the Edge ๐Ÿง— Phoenix LiveView Users ๐Ÿ‘จ๐Ÿซ๐Ÿ‘ฉ๐Ÿญ๐Ÿง‘โš•๐Ÿ‘ท Client ๐Ÿ–ฅ ๐Ÿ’พ๐Ÿ’พ๐Ÿ’พ๐Ÿ’พ ๐Ÿ—ƒ โš™ WSS / REST / GraphQL โ†• Controller Logic โš™ Data Store ๐Ÿ—ƒ DevOps ๐Ÿ“ค Developer ๐Ÿ‘ฉ๐Ÿ’ป ๐Ÿ–ฅ ๐Ÿ’พ

Slide 114

Slide 114

On the Edge ๐Ÿง— Phoenix LiveView Users ๐Ÿ‘จ๐Ÿซ๐Ÿ‘ฉ๐Ÿญ๐Ÿง‘โš•๐Ÿ‘ท Client ๐Ÿ–ฅ ๐Ÿ’พ๐Ÿ’พ๐Ÿ’พ๐Ÿ’พ ๐Ÿ—ƒ โš™ โš™ WSS / REST / GraphQL โ†• Controller Logic โš™ Data Store ๐Ÿ—ƒ DevOps ๐Ÿ“ค Developer ๐Ÿ‘ฉ๐Ÿ’ป ๐Ÿ–ฅ ๐Ÿ’พ

Slide 115

Slide 115

On the Edge ๐Ÿง— Phoenix LiveView Users ๐Ÿ‘จ๐Ÿซ๐Ÿ‘ฉ๐Ÿญ๐Ÿง‘โš•๐Ÿ‘ท Client ๐Ÿ–ฅ ๐Ÿ’พ๐Ÿ’พ๐Ÿ’พ๐Ÿ’พ ๐Ÿ—ƒ โš™ โš™ WSS / REST / GraphQL โ†• Controller Logic โš™ Data Store ๐Ÿ—ƒ DevOps ๐Ÿ“ค Developer ๐Ÿ‘ฉ๐Ÿ’ป ๐Ÿ–ฅ ๐Ÿ’พ ๐Ÿ–ฅ ๐Ÿ’พ

Slide 116

Slide 116

On the Edge ๐Ÿง— Phoenix LiveView Users ๐Ÿ‘จ๐Ÿซ๐Ÿ‘ฉ๐Ÿญ๐Ÿง‘โš•๐Ÿ‘ท Client ๐Ÿ–ฅ ๐Ÿ’พ๐Ÿ’พ๐Ÿ’พ๐Ÿ’พ ๐Ÿ—ƒ โš™ โš™ WSS / REST / GraphQL โ†• Controller Logic โš™ Data Store ๐Ÿ—ƒ DevOps ๐Ÿ“ค Developer ๐Ÿ‘ฉ๐Ÿ’ป ๐Ÿ–ฅ ๐Ÿ’พ ๐Ÿ–ฅ ๐Ÿ’พ

Slide 117

Slide 117

On the Edge ๐Ÿง— Upside Down โš™ ๐Ÿ–ฅ ๐Ÿ—ƒ๐Ÿ’พ ๐Ÿ’พ๐Ÿ’พ๐Ÿ’พ ๐Ÿ–ฅ ๐Ÿ’พ๐Ÿ’พ

Slide 118

Slide 118

Itโ€™s all about the Data, Data, Data ๐Ÿ“Š

Slide 119

Slide 119

Data dominates. If youโ€™ve chosen the right data structures and organized things well, the algorithms will almost always be self-evident. Data structures, not algorithms, are central to programming. Rob Pike, 5 Rules of Programming

Slide 120

Slide 120

Itโ€™s All About the Data ๐Ÿ“Š f Problems! Property Consequence Run anywhere No process in charge of access control Casual islands Inconsistent views of data (or downtime) Unstable topology No consistent connections Local irst In accessible, no replicas

Slide 121

Slide 121

Itโ€™s All About the Data ๐Ÿ“Š CAP โžก PACELC ๐Ÿ“ฆ๐ŸฆŒ

Slide 122

Slide 122

Itโ€™s All About the Data ๐Ÿ“Š CAP โžก PACELC ๐Ÿ“ฆ๐ŸฆŒ โ€ข If network partition (P) โ€ข Choose between: โ€ข Availability (A) โœ… Local-first & uptime โ€ข Consistency (C)

Slide 123

Slide 123

Itโ€™s All About the Data ๐Ÿ“Š CAP โžก PACELC ๐Ÿ“ฆ๐ŸฆŒ C โ€ข If network partition (P) โ€ข Choose between: โ€ข Availability (A) โœ… Local-first & uptime โ€ข Consistency (C) A

Slide 124

Slide 124

Itโ€™s All About the Data ๐Ÿ“Š CAP โžก PACELC ๐Ÿ“ฆ๐ŸฆŒ C โ€ข If network partition (P) โ€ข Choose between: P โ€ข Availability (A) โœ… Local-first & uptime โ€ข Consistency (C) A

Slide 125

Slide 125

Itโ€™s All About the Data ๐Ÿ“Š CAP โžก PACELC ๐Ÿ“ฆ๐ŸฆŒ C โ€ข If network partition (P) โ€ข Choose between: P โ€ข Availability (A) โœ… Local-first & uptime โ€ข Consistency (C) โ€ข Else (E) when running normally: โ€ข Choose between: โ€ข Latency (L) โœ… โ€ข Consistency (C) A

Slide 126

Slide 126

Itโ€™s All About the Data ๐Ÿ“Š CAP โžก PACELC ๐Ÿ“ฆ๐ŸฆŒ C โ€ข If network partition (P) โ€ข Choose between: P โ€ข Availability (A) โœ… Local-first & uptime E โ€ข Consistency (C) โ€ข Else (E) when running normally: โ€ข Choose between: โ€ข Latency (L) โœ… โ€ข Consistency (C) A L

Slide 127

Slide 127

Itโ€™s All About the Data ๐Ÿ“Š CAP โžก PACELC ๐Ÿ“ฆ๐ŸฆŒ C โ€ข If network partition (P) โ€ข Choose between: P โ€ข Availability (A) โœ… Local-first & uptime E โ€ข Consistency (C) โ€ข Else (E) when running normally: โ€ข Choose between: โ€ข Latency (L) โœ… โ€ข Consistency (C) A L

Slide 128

Slide 128

Itโ€™s All About the Data ๐Ÿ“Š CAP โžก PACELC ๐Ÿ“ฆ๐ŸฆŒ C โ€ข If network partition (P) โ€ข Choose between: P โ€ข Availability (A) โœ… Local-first & uptime E โ€ข Consistency (C) โ€ข Else (E) when running normally: โ€ข Choose between: โ€ข Latency (L) โœ… โ€ข Consistency (C) A L PA/EL

Slide 129

Slide 129

Itโ€™s All About the Data ๐Ÿ“Š Mutable Content โ€ข Predominantly single-source (per file) server/client โ€ข %{node_id => %{path => content}} โ€ข DNS maps names to IP addresses โ€ข PIDs associate processes with numbers โ€ข e.g. send(:example@42.123.45.6, :ping) โ€ข Focused on the physical network โ€ข Referential opacity ff โ€ข Calling same PID often will return di erent data

Slide 130

Slide 130

Itโ€™s All About the Data ๐Ÿ“Š Mutable Content โ€ข Predominantly single-source (per file) server/client โ€ข %{node_id => %{path => content}} โ€ข DNS maps names to IP addresses โ€ข PIDs associate processes with numbers โ€ข e.g. send(:example@42.123.45.6, :ping) โ€ข Focused on the physical network โ€ข Referential opacity ff โ€ข Calling same PID often will return di erent data V I R T UA L A D D R E S S P H Y S I C A L L O C AT I O N

Slide 131

Slide 131

Itโ€™s All About the Data ๐Ÿ“Š Consistent Keys โ€ข A layer of abstraction above location โ€ข %{hash(content) => content} โ€ข Hash AKA โ€œcontent identifierโ€ or CID โ€ข Special โ€œuniversalโ€ relationship to content โ€ข Focused on the data โ€ข Stored anywhere, same ID โ€ข E cient caching โ€ข Immutable data++ ffi โ€ข Not just consistent pointers; consistent data V I R T UA L A D D R E S S P H Y S I C A L L O C AT I O N

Slide 132

Slide 132

Itโ€™s All About the Data ๐Ÿ“Š Consistent Keys โ€ข A layer of abstraction above location โ€ข %{hash(content) => content} CONTENT ID โ€ข Hash AKA โ€œcontent identifierโ€ or CID โ€ข Special โ€œuniversalโ€ relationship to content โ€ข Focused on the data โ€ข Stored anywhere, same ID โ€ข E cient caching โ€ข Immutable data++ ffi โ€ข Not just consistent pointers; consistent data V I R T UA L A D D R E S S P H Y S I C A L L O C AT I O N

Slide 133

Slide 133

Itโ€™s All About the Data ๐Ÿ“Š Hash-Based Relationships

Slide 134

Slide 134

Itโ€™s All About the Data ๐Ÿ“Š Hash-Based Relationships (CID ~ Data PID) { Qm123456โ€ฆ: { data: nil, links: [ {name: โ€œcompanyโ€, hash: Qmabcdefโ€ฆ} {name: โ€œlicenseโ€, hash: Qmzyxwvuโ€ฆ} ] } }

Slide 135

Slide 135

Itโ€™s All About the Data ๐Ÿ“Š Hash-Based Relationships (CID ~ Data PID) { { Qm123456โ€ฆ: { data: nil, links: [ {name: โ€œcompanyโ€, hash: Qmabcdefโ€ฆ} {name: โ€œlicenseโ€, hash: Qmzyxwvuโ€ฆ} ] } } Qmabcdefโ€ฆ: { data: โ€œFissionโ€, links: [ {name: โ€œcityโ€, hash: Qm1gb5snโ€ฆ}, {name: โ€œaboutโ€, hash: Qmzyxwvuโ€ฆ} ] } }

Slide 136

Slide 136

Itโ€™s All About the Data ๐Ÿ“Š Hash-Based Relationships (CID ~ Data PID) { { Qm123456โ€ฆ: { data: nil, links: [ {name: โ€œcompanyโ€, hash: Qmabcdefโ€ฆ} {name: โ€œlicenseโ€, hash: Qmzyxwvuโ€ฆ} ] } } Qmabcdefโ€ฆ: { data: โ€œFissionโ€, links: [ {name: โ€œcityโ€, hash: Qm1gb5snโ€ฆ}, {name: โ€œaboutโ€, hash: Qmzyxwvuโ€ฆ} ] } } Qm123456โ€ฆ/company/about/ceo => โ€œBoris Mannโ€

Slide 137

Slide 137

Itโ€™s All About the Data ๐Ÿ“Š Content IDs Are Easy [no network version]

Slide 138

Slide 138

Itโ€™s All About the Data ๐Ÿ“Š Partial Dependencies

Slide 139

Slide 139

Itโ€™s All About the Data ๐Ÿ“Š Partial Dependencies t

Slide 140

Slide 140

Itโ€™s All About the Data ๐Ÿ“Š Partial Dependencies t

Slide 141

Slide 141

Itโ€™s All About the Data ๐Ÿ“Š Partial Dependencies t

Slide 142

Slide 142

Itโ€™s All About the Data ๐Ÿ“Š Partial Dependencies t

Slide 143

Slide 143

Itโ€™s All About the Data ๐Ÿ“Š Partial Dependencies t

Slide 144

Slide 144

Itโ€™s All About the Data ๐Ÿ“Š This all worksโ€ฆ

Slide 145

Slide 145

Itโ€™s All About the Data ๐Ÿ“Š Associative

Slide 146

Slide 146

Itโ€™s All About the Data ๐Ÿ“Š Out of Order Delivery ๐Ÿ’Œ โœ‰ ๐Ÿ“ง โš™ ๐Ÿ“ง ๐Ÿ’Œ ๐Ÿ’Œ

Slide 147

Slide 147

Itโ€™s All About the Data ๐Ÿ“Š Out of Order Delivery ๐Ÿ’Œ โœ‰ ๐Ÿ“ง โš™ ๐Ÿ“ง ๐Ÿ’Œ ๐Ÿ’Œ

Slide 148

Slide 148

Itโ€™s All About the Data ๐Ÿ“Š Commutative Monoid (AKA Minimal CRDT)

Slide 149

Slide 149

Itโ€™s All About the Data ๐Ÿ“Š Commutative Monoid (AKA Minimal CRDT) Sibling / Concurrent

Slide 150

Slide 150

Itโ€™s All About the Data ๐Ÿ“Š PNCounter

Slide 151

Slide 151

Itโ€™s All About the Data ๐Ÿ“Š PNCounter

Slide 152

Slide 152

Itโ€™s All About the Data ๐Ÿ“Š PNCounter

Slide 153

Slide 153

Itโ€™s All About the Data ๐Ÿ“Š PNCounter

Slide 154

Slide 154

Itโ€™s All About the Data ๐Ÿ“Š PNCounter

Slide 155

Slide 155

The Age of Decentralized Systems ๐ŸŒˆ

Slide 156

Slide 156

Decentralized Systems ๐ŸŒˆ Scale Curve Adapted from http://www.perfdynamics.com/Manifesto/USLscalability.html

Slide 157

Slide 157

Decentralized Systems ๐ŸŒˆ Scale Curve Linear Ideal Adapted from http://www.perfdynamics.com/Manifesto/USLscalability.html

Slide 158

Slide 158

Decentralized Systems ๐ŸŒˆ Scale Curve Linear Ideal Amdahlโ€™s Law Adapted from http://www.perfdynamics.com/Manifesto/USLscalability.html

Slide 159

Slide 159

Decentralized Systems ๐ŸŒˆ Scale Curve Linear Ideal Amdahlโ€™s Law Data Contention Adapted from http://www.perfdynamics.com/Manifesto/USLscalability.html Universal Scaling Law

Slide 160

Slide 160

Decentralized Systems ๐ŸŒˆ Scale Curve ๐Ÿคฏ Linear Ideal Shared Adaptive Memoization (โ€œTheoretical) Amdahlโ€™s Law Data Contention Adapted from http://www.perfdynamics.com/Manifesto/USLscalability.html Universal Scaling Law

Slide 161

Slide 161

Decentralized Systems ๐ŸŒˆ Conflict Free Effects ๐Ÿ•Š๐Ÿงฑ Side Effect Stream Pure Effect Stream Pure Function Stream Base Event Stream

Slide 162

Slide 162

Decentralized Systems ๐ŸŒˆ Conflict Free Effects ๐Ÿ•Š๐Ÿงฑ Side Effect Stream Pure Effect Stream Pure Function Stream Base Event Stream t

Slide 163

Slide 163

Decentralized Systems ๐ŸŒˆ GenEffect ๐Ÿš€

Slide 164

Slide 164

Decentralized Systems ๐ŸŒˆ Different Clients ~ Schema Drift Source: Project Cambria, Ink & Switch https://www.inkandswitch.com/cambria.html

Slide 165

Slide 165

Secure Decentralized Data Access Fixing the Leaky Pipes ๐Ÿšฟ

Slide 166

Slide 166

Fixing the Leaky Pipes ๐Ÿšฟ Object Capability Model (OCAP)

Slide 167

Slide 167

Fixing the Leaky Pipes ๐Ÿšฟ Object Capability Model (OCAP) โ€ข ACL is โ€œreactive authโ€ / OCAP is โ€œproactive authโ€

Slide 168

Slide 168

Fixing the Leaky Pipes ๐Ÿšฟ Object Capability Model (OCAP) โ€ข ACL is โ€œreactive authโ€ / OCAP is โ€œproactive authโ€ โ€ข OCAP contains all the info about access

Slide 169

Slide 169

Fixing the Leaky Pipes ๐Ÿšฟ Object Capability Model (OCAP) โ€ข ACL is โ€œreactive authโ€ / OCAP is โ€œproactive authโ€ โ€ข OCAP contains all the info about access โ€ข Generally some reference, proof, or key โ€ข โ€ฆnot unlike having a PID โ€ข Rights to anything directly created (parenthood) โ€ข The right to delegate subset of access to another (introduction)

Slide 170

Slide 170

Fixing the Leaky Pipes ๐Ÿšฟ Object Capability Model (OCAP) โ€ข ACL is โ€œreactive authโ€ / OCAP is โ€œproactive authโ€ โ€ข OCAP contains all the info about access โ€ข Generally some reference, proof, or key โ€ข โ€ฆnot unlike having a PID โ€ข Rights to anything directly created (parenthood) โ€ข The right to delegate subset of access to another (introduction) โ€ข Long history (e.g. X.509, SDSI, SPKI, Macaroons)

Slide 171

Slide 171

Fixing the Leaky Pipes ๐Ÿšฟ 3rd-Party Subdelegation & Attenuation

Slide 172

Slide 172

Fixing the Leaky Pipes ๐Ÿšฟ 3rd-Party Subdelegation & Attenuation ๐Ÿ–ฅ

Slide 173

Slide 173

Fixing the Leaky Pipes ๐Ÿšฟ 3rd-Party Subdelegation & Attenuation ๐Ÿ–ฅ โš™

Slide 174

Slide 174

Fixing the Leaky Pipes ๐Ÿšฟ 3rd-Party Subdelegation & Attenuation ๐Ÿ–ฅ ๐Ÿ”Ÿ โš™

Slide 175

Slide 175

Fixing the Leaky Pipes ๐Ÿšฟ 3rd-Party Subdelegation & Attenuation ๐Ÿ–ฅ ๐Ÿ”Ÿ โš™ ๐Ÿ› 

Slide 176

Slide 176

Fixing the Leaky Pipes ๐Ÿšฟ 3rd-Party Subdelegation & Attenuation ๐Ÿ–ฅ ๐Ÿ”Ÿ โš™ 2โƒฃ ๐Ÿ› 

Slide 177

Slide 177

Fixing the Leaky Pipes ๐Ÿšฟ Direct Access Control โ€ขAdvantages โ€ขProactive โ€ขProactive โ€ขWorks o ine โ€ขRevocation โ€ขAttenuation โ€ขGive up (more) access stats โ€ขEasy to understand rules โ€ขUser control (GDPR, CCPA) โ€ขInteroperable ffl โ€ขChallenges

Slide 178

Slide 178

Fixing the Leaky Pipes ๐Ÿšฟ Hierarchal Read Access

Slide 179

Slide 179

Fixing the Leaky Pipes ๐Ÿšฟ Cryptree ๐ŸŽ„ JSON Binary Encrypted Node ๐Ÿ”’ AES256 + ๐Ÿ”‘ Virtual Node = Index ๐Ÿ”‘ ๐Ÿ”‘ Metadata ๐Ÿ”‘

Slide 180

Slide 180

Fixing the Leaky Pipes ๐Ÿšฟ Cryptree Sketch โœ

Slide 181

Slide 181

Fixing the Leaky Pipes ๐Ÿšฟ Cryptree Sketch โœ Local stateful, remote stateless

Slide 182

Slide 182

How to Do O ine & Distributed Auth ffl Universal Auth & ID ๐Ÿ—

Slide 183

Slide 183

Universal Auth & ID ๐Ÿ— Universal IDs

Slide 184

Slide 184

Universal Auth & ID ๐Ÿ— Universal IDs โ€ข W3C, DIF, Microsoft

Slide 185

Slide 185

Universal Auth & ID ๐Ÿ— Universal IDs โ€ข W3C, DIF, Microsoft โ€ข Based on public-key cryptography

Slide 186

Slide 186

Universal Auth & ID ๐Ÿ— Universal IDs โ€ข W3C, DIF, Microsoft โ€ข Based on public-key cryptography โ€ข Truly โ€œuniversalโ€ user IDs

Slide 187

Slide 187

Universal Auth & ID ๐Ÿ— Universal IDs โ€ข W3C, DIF, Microsoft โ€ข Based on public-key cryptography โ€ข Truly โ€œuniversalโ€ user IDs โ€ข Agnostic about backing

Slide 188

Slide 188

Universal Auth & ID ๐Ÿ— Universal IDs โ€ข W3C, DIF, Microsoft โ€ข Based on public-key cryptography โ€ข Truly โ€œuniversalโ€ user IDs โ€ข Agnostic about backing โ€ข For users, devices, and more

Slide 189

Slide 189

Universal Auth & ID ๐Ÿ— JWT Encoded

Slide 190

Slide 190

Universal Auth & ID ๐Ÿ— JWT Encoded

Slide 191

Slide 191

Universal Auth & ID ๐Ÿ— JWT Encoded

Slide 192

Slide 192

Universal Auth & ID ๐Ÿ— Auth Chaining

Slide 193

Slide 193

Universal Auth & ID ๐Ÿ— OAuth vs UCAN Sequence

Slide 194

Slide 194

Universal Auth & ID ๐Ÿ— OAuth vs UCAN Sequence

Slide 195

Slide 195

Universal Auth & ID ๐Ÿ— OAuth vs UCAN Sequence (Verifiable & user originated)

Slide 196

Slide 196

Universal Auth & ID ๐Ÿ—

Slide 197

Slide 197

Universal Auth & ID ๐Ÿ— ๐Ÿ›‚ External OIDC Server ๐Ÿค– Service A ๐Ÿ‘ฝ Service B ๐Ÿ’ User

Slide 198

Slide 198

Universal Auth & ID ๐Ÿ— ๐Ÿ›‚ External OIDC Server ๐Ÿค– Service A ๐Ÿ‘ฝ ๐Ÿ’ Service B User ff UCAN with ๐Ÿ’ ID / email Describes o er for ๐Ÿค–

Slide 199

Slide 199

Universal Auth & ID ๐Ÿ— ๐Ÿ›‚ External OIDC Server ๐Ÿค– Service A ๐Ÿ‘ฝ ๐Ÿ’ Service B User UCAN with ๐Ÿ’ ID / email Describes o er for ๐Ÿค– ff OIDC Login

Slide 200

Slide 200

Universal Auth & ID ๐Ÿ— ๐Ÿ›‚ External OIDC Server ๐Ÿค– Service A ๐Ÿ‘ฝ ๐Ÿ’ Service B User UCAN with ๐Ÿ’ ID / email Describes o er for ๐Ÿค– OIDC Login ff OIDC Token

Slide 201

Slide 201

Universal Auth & ID ๐Ÿ— ๐Ÿ›‚ External OIDC Server ๐Ÿค– Service A ๐Ÿ‘ฝ ๐Ÿ’ Service B User UCAN with ๐Ÿ’ ID / email Describes o er for ๐Ÿค– OIDC Login OIDC Token ff ff O er for ๐Ÿค–+๐Ÿ’ Secured with signature ๐Ÿ‘ฝ and HMAC ๐Ÿ’๐Ÿ›‚

Slide 202

Slide 202

Universal Auth & ID ๐Ÿ— ๐Ÿ›‚ External OIDC Server ๐Ÿค– Service A ๐Ÿ‘ฝ ๐Ÿ’ Service B User UCAN with ๐Ÿ’ ID / email Describes o er for ๐Ÿค– OIDC Login OIDC Token O er for ๐Ÿค–+๐Ÿ’ Secured with signature ๐Ÿ‘ฝ and HMAC ๐Ÿ’๐Ÿ›‚ ff ff ๐Ÿ’โ€™s OIDC token?

Slide 203

Slide 203

Universal Auth & ID ๐Ÿ— ๐Ÿ›‚ External OIDC Server ๐Ÿค– Service A ๐Ÿ‘ฝ ๐Ÿ’ Service B User UCAN with ๐Ÿ’ ID / email Describes o er for ๐Ÿค– OIDC Login OIDC Token O er for ๐Ÿค–+๐Ÿ’ Secured with signature ๐Ÿ‘ฝ and HMAC ๐Ÿ’๐Ÿ›‚ ๐Ÿ’โ€™s OIDC token? ff ff ๐Ÿ’โ€™s OIDC token!

Slide 204

Slide 204

Universal Auth & ID ๐Ÿ— ๐Ÿ›‚ External OIDC Server ๐Ÿค– Service A ๐Ÿ‘ฝ ๐Ÿ’ Service B User UCAN with ๐Ÿ’ ID / email Describes o er for ๐Ÿค– OIDC Login OIDC Token O er for ๐Ÿค–+๐Ÿ’ Secured with signature ๐Ÿ‘ฝ and HMAC ๐Ÿ’๐Ÿ›‚ ๐Ÿ’โ€™s OIDC token? ๐Ÿ’โ€™s OIDC token! ff ff Check ๐Ÿ’ HMAC and ๐Ÿ‘ฝ signature

Slide 205

Slide 205

Universal Auth & ID ๐Ÿ— ๐Ÿ›‚ External OIDC Server ๐Ÿค– Service A ๐Ÿ‘ฝ ๐Ÿ’ Service B User UCAN with ๐Ÿ’ ID / email Describes o er for ๐Ÿค– OIDC Login OIDC Token O er for ๐Ÿค–+๐Ÿ’ Secured with signature ๐Ÿ‘ฝ and HMAC ๐Ÿ’๐Ÿ›‚ ๐Ÿ’โ€™s OIDC token? ๐Ÿ’โ€™s OIDC token! Check ๐Ÿ’ HMAC and ๐Ÿ‘ฝ signature ff ff Update ๐Ÿค– subscription for ๐Ÿ’

Slide 206

Slide 206

Universal Auth & ID ๐Ÿ— ๐Ÿ›‚ External OIDC Server ๐Ÿค– ๐Ÿ‘ฝ Service A ๐Ÿ’ Service B User UCAN with ๐Ÿ’ ID / email Describes o er for ๐Ÿค– OIDC Login OIDC Token O er for ๐Ÿค–+๐Ÿ’ Secured with signature ๐Ÿ‘ฝ and HMAC ๐Ÿ’๐Ÿ›‚ ๐Ÿ’โ€™s OIDC token? ๐Ÿ’โ€™s OIDC token! Check ๐Ÿ’ HMAC and ๐Ÿ‘ฝ signature Update ๐Ÿค– subscription for ๐Ÿ’ ff ff 204 Accepted

Slide 207

Slide 207

Summary ๐Ÿฑ

Slide 208

Slide 208

Slide 209

Slide 209

Instead of immediately asking โ€œwhich database would be best to hold presences?โ€, we could ask โ€œhow can we best replicate data in a distributed system without the user having to worry about it?โ€. The platforms you build on top of drive the design decisions you make in your products. With Elixir, you are empowered to tackle problems that in other platforms would feel impossible to solve without tradeoffs with heavy dependencies. ~ Chris McCord, What Makes Phoenix Presence Special

Slide 210

Slide 210

Getting Ready ๐Ÿฑ Data > Compute โ€ข Focus on data & structure โ€ข Clarify โ€œrealโ€ dependencies on data โ€ข Start thinking about the properties in your code โ€ข Adopt OCAP โ€ข Use abstraction for declarative interfaces

Slide 211

Slide 211

๐Ÿ‡ง๐Ÿ‡ท Thank You, CodeBEAM BR ๐ŸŽ‰ brooklyn@fission.codes https://fission.codes github.com/expede @expede