Seamless Services for an Open World
We cannot sow seeds with clenched fists. To sow we must open our hands. β Adolfo Perez Esquivel
Slide 4
Seamless Services for an Open World
Slide 5
Seamless Services for an Open World
Jesper, I have this idea in which weβll connect all of the worlds Erlang systems to each other, imagine if every process could talk to every other process, world-wide! β Joe Armstrong, email to Jesper L. Andersen
Slide 6
Seamless Services for an Open World
Brooklyn Zelenka @expede
github.com/expede
Slide 7
Seamless Services for an Open World
Brooklyn Zelenka @expede Cofounder & CTO at Fission fission.codes discord.gg/fissioncodes @fission@plnetwork.xyz PLT and DSys are my jam π€ UCAN, WNFS, Rhizome, IPVM, Multiformats, EIPs, others github.com/expede
Slide 8
Seamless Services for an Open World
Rolling Weight
Slide 9
Seamless Services for an Open World
Rolling Weight
Other 35%
IBM 4% Alibaba 4% Google 9%
AWS 32%
Azure 17%
Slide 10
Seamless Services for an Open World
Rolling Weight
Other 35%
IBM 4% Alibaba 4% Google 9%
AWS 32%
Azure 17%
Slide 11
Seamless Services for an Open World
Rolling Weight
Other 35%
IBM 4% Alibaba 4% Google 9%
AWS 32%
Azure 17%
Slide 12
Seamless Services for an Open World
Rolling Weight
Other 35%
IBM 4% Alibaba 4% Google 9%
AWS 32%
Azure 17%
Slide 13
Seamless Services for an Open World
Rolling Weight
Other 35%
IBM 4% Alibaba 4% Google 9%
AWS 32%
Azure 17%
Slide 14
Seamless Services for an Open World
Everything, Everywhere, All At Once
Slide 15
Seamless Services for an Open World
Everything, Everywhere, All At Once Nothing less than connecting all of the worldβs users & services. The βHTTPβ storage and compute equivalent: open, interoperable, & everywhere. Must be substantially better than the status quo.
Slide 16
Space to Play
Dependency Stack
Slide 17
Space to Play
Dependency Stack Compute β
Slide 18
Space to Play
Dependency Stack Compute β Data πΎ
Slide 19
Space to Play
Dependency Stack Compute β Data πΎ Auth π
Slide 20
Slide 21
β A Series of Very Reasonable Steps β
How We Got Here
Slide 22
How We Got Here
One-to-One
Slide 23
How We Got Here
One-to-One
π
π₯
Slide 24
How We Got Here
One-to-One
π
π₯ π’
Slide 25
How We Got Here
One-to-One
π
π₯ π’
π
Slide 26
How We Got Here
One-to-One
π
π₯ π’
β
π
Slide 27
How We Got Here
One-to-One
π
π₯ π’
β
πͺ
π
Slide 28
How We Got Here
One-to-One
π
π₯ π’
β
πͺ
π
Slide 29
How We Got Here
One-to-One
π π π
π₯ π’ π₯ π’ π₯ π’
β
πͺ
π
Slide 30
How We Got Here
One-to-One
π π π
π₯ π’ π₯ π’ π₯ π’
β
πͺ
π
π
Slide 31
How We Got Here
Hidden Many-to-Many
π π π
π₯ π₯ π₯
π
β β β
π π π
π π π
Slide 32
How We Got Here
Invisible Many-to-Many
π π π
π₯ π₯ π₯
π
β β β
π π π
π π π
Slide 33
How We Got Here
Invisible Many-to-Many
π π π
π₯ π₯ π₯
β π π π β π βCloudβ π β π
β
π
How We Got Here
β¦and so it was for many yearsβ¦
Slide 36
How We Got Here
β¦and so it was for many yearsβ¦ π¦βππΎπ°π’π
Slide 37
How We Got Here
Consequences π
Slide 38
How We Got Here
Consequences π
β’ Single source of truth (βtheβ database)
Slide 39
How We Got Here
Consequences π
β’ Single source of truth (βtheβ database) β’ Server-centric β’ βFull stack developmentβ β’ DevOps, Docker, k8s β’ How to train enough engineers?
Slide 40
How We Got Here
Consequences π
β’ Single source of truth (βtheβ database) β’ Server-centric β’ βFull stack developmentβ β’ DevOps, Docker, k8s β’ How to train enough engineers?
Slide 41
How We Got Here
Random Walk
Image by Pradyumna Yadav
Slide 42
How We Got Here
Random Walk
Image by Pradyumna Yadav
Slide 43
How We Got Here
Random Walk
π
Image by Pradyumna Yadav
Slide 44
How We Got Here
Random Walk
π π
Image by Pradyumna Yadav
Slide 45
How We Got Here
Random Walk π§ π π
Image by Pradyumna Yadav
Slide 46
How We Got Here
Random Walk
π
π§ π π
Image by Pradyumna Yadav
Slide 47
How We Got Here
Slide 48
How We Got Here
Not to be bound by certain βobviousβ methodological rules [β¦] is both reasonable and absolutely necessary for the growth of knowledge. [β¦] There are always circumstances when it is advisable not only to ignore the rule, but to adopt its opposite. β Paul Feyerabend, Against Method
Slide 49
How We Got Here
Not to be bound by certain βobviousβ methodological rules [β¦] is both reasonable and absolutely necessary for the growth of knowledge. [β¦] There are always circumstances when it is advisable not only to ignore the rule, but to adopt its opposite. β Paul Feyerabend, Against Method
Slide 50
Slide 51
β Weβre Not On Dial-Up Anymore β
A New Environment
Slide 52
A New Environment
Sending a βDirectβ Message
Slide 53
A New Environment
Sending a βDirectβ Message
Slide 54
A New Environment
Sending a βDirectβ Message
Slide 55
A New Environment
Sending a βDirectβ Message
β³ππ’πͺ
Slide 56
A New Environment
Data Gravity Aggregated Data
Slide 57
A New Environment
Users vs Cloud Infra
Source: AWS
Slide 58
A New Environment
Users vs Cloud Infra 6
7
6
2 1 1
1
1 Source: AWS
Slide 59
A New Environment
Users vs Cloud Infra 6
7
6
2 1 1
1
1 Source: AWS
Slide 60
A New Environment
Users vs Cloud Infra 6
7
6
2 1 1
1
1 Source: AWS
Slide 61
A New Environment
Users vs Cloud Infra 6
7
6
2 1 1
1
1 Source: AWS
Slide 62
A New Environment
Users vs Cloud Infra 6
7
2
6
50M
1 1
1
1 Source: AWS
Slide 63
A New Environment
Users vs Cloud Infra
371 million 56M/centre
6
7
2
6
50M
1 1
1
1 Source: AWS
Slide 64
A New Environment
Users vs Cloud Infra
371 million 56M/centre
6
7
2
6
50M
1 1 ~435 million 435M/centre
1
1 Source: AWS
Slide 65
A New Environment
Users vs Cloud Infra
371 million 56M/centre
6
7
2
6
50M
1 1 ~435 million 435M/centre
1
~1.4 billion 1400M/centre
1 Source: AWS
Slide 66
A New Environment
Users vs Cloud Infra
371 million 56M/centre
6
7
2
6
50M
1 1 ~435 million 435M/centre
1
~1.4 billion 1400M/centre
1 Source: AWS
Slide 67
A New Environment
What 8ms Looks Like
Slide 68
A New Environment
What 8ms Looks Like Toronto β‘ (almost) Havana Ideal Vacuum π«
Slide 69
A New Environment
What 8ms Looks Like Toronto β‘ (almost) Havana Ideal Vacuum π«
Toronto π Halifax Ideal Vacuum π«
Slide 70
A New Environment
What 8ms Looks Like Toronto β‘ (almost) Havana Ideal Vacuum π«
Toronto π Halifax Ideal Vacuum π«
Toronto π Montreal Ideal Fibre π§Ά
Slide 71
A New Environment
Causal Islands ππ
Slide 72
A New Environment
Causal Islands ππ
Slide 73
A New Environment
Causal Islands ππ
Slide 74
A New Environment
Causal Islands ππ
Slide 75
A New Environment
Causal Islands ππ
Slide 76
A New Environment
Causal Islands ππ
Slide 77
A New Environment
Causal Islands ππ
Slide 78
A New Environment
Causal Islands ππ
βCausal Subjectivityβ
Slide 79
A New Environment
Slide 80
A New Environment
As we continue to increase the number of globally connected devices, we must embrace a design that considers every single member in the system as the primary site for the data that it is generates. It is completely impractical that we can look at a single, or a small number, of globally distributed data centers as the primary site for all global information that we desire to perform computations with. β Meiklejohn, A Certain Tendency Of The Database Community
Slide 81
A New Environment
Everything, Everywhere
Slide 82
A New Environment
Everything, Everywhere Commons
Cloud & Edge
Far Edge
Slide 83
A New Environment
Everything, Everywhere Commons
Cloud & Edge
Far Edge
Slide 84
A New Environment
Everything, Everywhere Commons
Cloud & Edge
Far Edge
Slide 85
A New Environment
Slide 86
Slide 87
β Getting Back to Our Roots β
Networks for Good
Slide 88
Networks for Good
Flat Circle
Slide 89
Networks for Good
Flat Circle
THIS HAS ALL HAPPENED BEFORE
Slide 90
Networks for Good
Flat Circle
THIS HAS ALL HAPPENED BEFORE β¦and will happen again
Slide 91
Networks for Good
Back to Our Roots
Slide 92
Networks for Good
Back to Our Roots 1. Decentralisation 2. Non-discrimination 3. Bottom-up Design 4. Universality 5. Consensus β The Web Foundation, History of the Web
Slide 93
Networks for Good
Back to Our Roots 1. Decentralisation 2. Non-discrimination 3. Bottom-up Design 4. Universality 5. Consensus
en.wikipedia.org/wiki/OSI_model
β The Web Foundation, History of the Web
Slide 94
Networks for Good
Back to Our Roots 1. Decentralisation 2. Non-discrimination 3. Bottom-up Design 4. Universality 5. Consensus
en.wikipedia.org/wiki/OSI_model
β The Web Foundation, History of the Web
Slide 95
Networks for Good
Back to Our Roots 1. Decentralisation 2. Non-discrimination 3. Bottom-up Design 4. Universality 5. Consensus
en.wikipedia.org/wiki/OSI_model
β The Web Foundation, History of the Web
Networks for Good
User Agency
β’ Empower users to participate (entry) β’ Option to leave (exit) β’ Control access to your data (safety) β’ Provide capacity to others (serve)
Slide 108
Slide 109
β The Times They Are A-Changinβ β
Signs of a Way Out
Slide 110
Signs of a Way Out
Emerging Definitions
Slide 111
Signs of a Way Out
ACL Redux
Slide 112
Signs of a Way Out
ACL Redux
π§πΎ
Slide 113
Signs of a Way Out
ACL Redux
π§πΎ
β
Slide 114
Signs of a Way Out
ACL Redux
π§πΎ
π
β
β
Slide 115
Signs of a Way Out
ACL Redux
π§πΎ
π π
β
β
Slide 116
Signs of a Way Out
ACL Redux
π§πΎ
π π
β
β
Slide 117
Signs of a Way Out
ACL Redux
π§πΎ
π π
β
β
Slide 118
Signs of a Way Out
ACL Redux
π§πΎ
π π
β
Not in control
β
Slide 119
Signs of a Way Out
ACL Redux
π In control
π§πΎ
π
β
Not in control
β
Slide 120
Signs of a Way Out
ACL Redux
π
π
β
In control
π§πΎ
π
β
Not in control
β
Slide 121
Signs of a Way Out
Capabilities
Slide 122
Signs of a Way Out
Capabilities
π΅
Slide 123
Signs of a Way Out
Capabilities
π΅
β
Slide 124
Signs of a Way Out
Capabilities
π΅ πΊ Addr
β
Slide 125
Signs of a Way Out
Capabilities
π΅ πΊ Addr
π
β
Slide 126
Signs of a Way Out
Capabilities
π΅ πΊ Addr
In control
π
β
Slide 127
Signs of a Way Out
Capabilities
π΅ πΊ Addr
In control
π
β All req info
Slide 128
Signs of a Way Out
Capabilities
π΅ πΊ Addr
π
β
Slide 129
Signs of a Way Out
Capabilities
π΅ π πΊ π π Addr
β
Slide 130
Signs of a Way Out
Capabilities
π΅ πΊ Addr
π
β
Slide 131
Signs of a Way Out
Capabilities
π΅ πΊ Addr
π¨π¨
π
β
Slide 132
Signs of a Way Out
Capabilities
π΅ πΊ Addr
π
π
π¨π¨ β
Slide 133
Signs of a Way Out
Capabilities
π΅ πΊ Addr
π
π
π¨π¨
β
π
Slide 134
Signs of a Way Out
OAuth Sequence
Slide 135
Signs of a Way Out
OAuth Sequence
Slide 136
Signs of a Way Out
OAuth Sequence
Slide 137
Signs of a Way Out
OAuth Sequence
Slide 138
Signs of a Way Out
UCAN Sequence
π
Slide 139
Signs of a Way Out
UCAN Sequence
π
Slide 140
Signs of a Way Out
UCAN Sequence
π
Slide 141
Signs of a Way Out
UCAN Sequence
π
Slide 142
Signs of a Way Out
Addressing Stack
Slide 143
Signs of a Way Out
Addressing Stack
Physical Location πΊ
send(42.123.45.6, path) = content1 send(42.123.45.6, path) = content2
Slide 144
Signs of a Way Out
Addressing Stack
Naming π¬ Physical Location πΊ
{DNS: IP} send(42.123.45.6, path) = content1 send(42.123.45.6, path) = content2
Slide 145
Signs of a Way Out
Addressing Stack
Content ID β Naming π¬ Physical Location πΊ
{hash(content): content} {DNS: IP} send(42.123.45.6, path) = content1 send(42.123.45.6, path) = content2
Slide 146
Signs of a Way Out
Universal Relationships
Slide 147
Signs of a Way Out
Universal Relationships
{
}
bafy123456β¦: { data: nil, links: [ {name: βcompanyβ, hash: bafyabcβ¦} {name: βindustryβ, hash: bafyzyxβ¦} ] }
Compute Substrate
Run Once, And Never Again Input Hash β Cached Output
Slide 214
Compute Substrate
Run Once, And Never Again Input Hash β Cached Output AI moderation & tagging
Slide 215
Compute Substrate
Run Once, And Never Again Input Hash β Cached Output AI moderation & tagging Distributed token validation
Slide 216
Slide 217
Slide 218
Compute Substrate
With a Little Help From My Friends
Slide 219
Compute Substrate
Throughput
With a Little Help From My Friends
Parallelisation
Slide 220
Compute Substrate
With a Little Help From My Friends
Throughput
Ideal (Linear)
Parallelisation
Slide 221
Compute Substrate
With a Little Help From My Friends Ideal (Linear)
Throughput
Amdahlβs Law
Parallelisation
Slide 222
Compute Substrate
With a Little Help From My Friends Ideal (Linear)
Throughput
Amdahlβs Law
Universal Scaling Law
Parallelisation
Slide 223
Compute Substrate
With a Little Help From My Friends Ideal (Linear)
Throughput
Amdahlβs Law
Incoherence, Data Contention
Parallelisation
Universal Scaling Law
Slide 224
Compute Substrate
With a Little Help From My Friends
Throughput
Global Adaptive Optimisation
π
Ideal (Linear)
Amdahlβs Law
Incoherence, Data Contention
Parallelisation
Universal Scaling Law
Slide 225
Compute Substrate
With a Little Help From My Friends
Throughput
Global Adaptive Optimisation
π
Ideal (Linear)
Amdahlβs Law
Incoherence, Data Contention
Parallelisation
Universal Scaling Law
The Safety Dance β Out of the Locality Tar Pit β
Slide 234
The Safety Dance πΊ
Description vs Invocation
Slide 235
The Safety Dance πΊ
Description vs Invocation Impure functions produce side effects Pure functions manipulate data Side effects β managed effects
Slide 236
The Safety Dance πΊ
Description vs Invocation Impure functions produce side effects Pure functions manipulate data Side effects β managed effects
function
Slide 237
The Safety Dance πΊ
Description vs Invocation Impure functions produce side effects Pure functions manipulate data Side effects β managed effects input
function
Slide 238
The Safety Dance πΊ
Description vs Invocation Impure functions produce side effects Pure functions manipulate data Side effects β managed effects input
function
output
function
Slide 239
The Safety Dance πΊ
Description vs Invocation Impure functions produce side effects Pure functions manipulate data Side effects β managed effects effect
input
effect
function
output
function
Slide 240
The Safety Dance πΊ
Description vs Invocation Impure functions produce side effects Pure functions manipulate data Side effects β managed effects effect
input
function
output
effect
function
Slide 241
The Safety Dance πΊ
Managed Effects Mutation E ect Stream
Query E ect Stream
Pure Function Stream
ff
ff
Base Event Stream
tβ
Slide 242
The Safety Dance πΊ
Managed Effects Mutation E ect Stream
Query E ect Stream
Pure Function Stream
ff
ff
Base Event Stream
tβ
Slide 243
The Safety Dance πΊ
Managed Effects Mutation E ect Stream
Query E ect Stream
Pure Function Stream
ff
ff
Base Event Stream
tβ
Slide 244
The Safety Dance πΊ
Managed Effects Mutation E ect Stream
Query E ect Stream
Pure Function Stream
ff
ff
Base Event Stream
tβ
Slide 245
The Safety Dance πΊ
Managed Effects Mutation E ect Stream
Query E ect Stream
Pure Function Stream
ff
ff
Base Event Stream
tβ
π
Slide 246
The Safety Dance πΊ
Managed Effects
π
Mutation E ect Stream
Query E ect Stream
Pure Function Stream
ff
ff
Base Event Stream
tβ
π
Slide 247
The Safety Dance πΊ
Managed Effects
π
Mutation E ect Stream
Query E ect Stream
Pure Function Stream
ff
ff
Base Event Stream
tβ
π
π
The Safety Dance πΊ
Virtual Resiliency
Mutable π¦ Idempotent π Deterministic π
Query A
Query B
Slide 251
The Safety Dance πΊ
Virtual Resiliency
Query A
Query B
Compute A
Mutable π¦ Idempotent π Deterministic π
Slide 252
The Safety Dance πΊ
Virtual Resiliency
Query A
Query B
Compute A
Mutable π¦ Query C
Idempotent π Deterministic π
Slide 253
The Safety Dance πΊ
Virtual Resiliency
Query A
Query B
Compute A
Mutable π¦ Query C
Idempotent π Deterministic π
Compute B
Query D
Slide 254
The Safety Dance πΊ
Virtual Resiliency
Query A
Query B
Compute A
Mutable π¦ Query C
Idempotent π Deterministic π
Compute B
Query D
Mutation
Slide 255
The Safety Dance πΊ
Virtual Resiliency Mutation
Query A
Query B
Compute A
Mutable π¦ Query C
Mutation
Idempotent π Deterministic π
Compute B
Query D
Mutation
Slide 256
The Safety Dance πΊ
Slide 257
The Safety Dance πΊ
If their application can be cast as pure data processing, they benefit from the past 40-50 years of work form the database community, [and] completely isolate the developer from the possibility of failure β Goldstein et al, AMBROSIA: Providing Performant Virtual Resiliency for Distributed Applications
Slide 258
The Safety Dance πΊ
Simplified Safe Layout
Slide 259
The Safety Dance πΊ
Simplified Safe Layout
Queries Queries Queries
Slide 260
The Safety Dance πΊ
Simplified Safe Layout
Queries Queries Queries
Pure Computation Pure Computation Pure Computation
Slide 261
The Safety Dance πΊ
Simplified Safe Layout
Queries Queries Queries
Pure Computation Pure Computation Pure Computation
Slide 262
The Safety Dance πΊ
Simplified Safe Layout
Queries Queries Queries
Pure Computation Pure Computation Pure Computation
Mutation
Slide 263
The Safety Dance πΊ
Simple Example Compute A
Query
Compute B
Mutation
Slide 264
The Safety Dance πΊ
Simple Example Compute A
Query
Compute B
Mutation
Slide 265
The Safety Dance πΊ
Simple Example Compute A
Query
π¦Ίβ Compute B
Mutation
Slide 266
Slide 267
Wrap Up
Slide 268
Wrap Up
Open Source
Slide 269
Wrap Up
Open Source Lots of people work on it, everybody benefits from it, and then people can build upon it (even in a revenue generating fashion) β Ted Leung, Commons Based Peer Production (2005)
Slide 270
Wrap Up
Commons Infrastructure Lots of people work on it, everybody benefits from it, and then people can build upon it (even in a revenue generating fashion), and is βownedβ by everyone
Slide 271
Wrap Up
Commons Infrastructure Lots of people work on it, everybody benefits from it, and then people can build upon it (even in a revenue generating fashion), and is βownedβ by everyone
By participating (even with competitive goals) you are strengthening the network for everyone
Slide 272
Wrap Up
Commons Infrastructure Lots of people work on it, everybody benefits from it, and then people can build upon it (even in a revenue generating fashion), and is βownedβ by everyone
By participating (even with competitive goals) you are strengthening the network for everyone
Slide 273
Wrap Up
Commons Infrastructure Lots of people work on it, everybody benefits from it, and then people can build upon it (even in a revenue generating fashion), and is βownedβ by everyone
By participating (even with competitive goals) you are strengthening the network for everyone
Slide 274
Wrap Up
Reusable/Remixable Specs
Slide 275
Wrap Up
Reusable/Remixable Specs
UCAN Core π Distributed Authority
IPLD-WIT β ABI
Varsig β Signature Multiformat