IPVM (Interplanetary VM)

A presentation at IPFS þing in July 2022 in Reykjavík, Iceland by Brooklyn Zelenka

Slide 1

Slide 1

IPVM The Long-Fabled Execution Layer

Slide 2

Slide 2

IPVM The Long-Fabled Execution Layer “The Easiest Way to Run Wasm Everywhere”

Slide 3

Slide 3

IPVM The Long-Fabled Execution Layer “The Easiest Way to Run Wasm Everywhere” The Fastest Way to Ship IPFS Features to Users

Slide 4

Slide 4

IPVM The Long-Fabled Execution Layer “The Easiest Way to Run Wasm Everywhere” The Fastest Way to Ship IPFS Features to Users A Step Towards IPOS

Slide 5

Slide 5

Agenda 1. 2. 3. 4. 5. 6. 🦖 What 🌋 s y a d y Why l r a e r e p Security u S Scheduling Compute Commons Open Discussion 🌶

Slide 6

Slide 6

IPVM IP What?

Slide 7

Slide 7

IPVM IP What? A blessed VM (Wasm) in every IPFS node Transparent IPFS node upgrades (like the web) 🎭 Support features like Autocodec Compute without (required) consensus Global adaptive optimization Mobile (ambient) computing

Slide 8

Slide 8

IPVM What This Gets You

Slide 9

Slide 9

IPVM What This Gets You Like data, compute should be ubiquitous End users and IPFS teams can depend on having compute around Deep integration with tooling Fully consistent functionality between clients Replace AWS’s proprietary Lambda with an open protocol + nodes “The HTTP of Compute” 🤩

Slide 10

Slide 10

Requirements Working Backwards

Slide 11

Slide 11

Requirements The Basics

Slide 12

Slide 12

Requirements The Basics Portable Deterministic, verifiable Purity? Managed e ects? Enforced termination (enabled-by-default?) Move compute to data / move data to compute Push & pull both important ff (e.g.) UCAN for remote invocation

Slide 13

Slide 13

Requirements The Basics Portable Deterministic, verifiable Purity? Managed e ects? Enforced termination (enabled-by-default?) Move compute to data / move data to compute Push & pull both important ff (e.g.) UCAN for remote invocation https://twitter.com/impurepics

Slide 14

Slide 14

Requirements Familiarity as Adoption Tactic

Slide 15

Slide 15

Requirements Familiarity as Adoption Tactic Bring your own language Common patterns (e.g. manifests, cron, systemd, build packs)

Slide 16

Slide 16

Requirements Deep Integration

Slide 17

Slide 17

Requirements Deep Integration Lean into content addressing More than the sum of its parts Local and remote execution Reuse Wasm e ort, infra, tooling, community experience, etc ff FVM, Aquamarine, CloudFlare Workers, Bacalhau, web3.storage invocations, IPFS-FAN

Slide 18

Slide 18

Requirements Deep Integration Lean into content addressing More than the sum of its parts Local and remote execution Reuse Wasm e ort, infra, tooling, community experience, etc ff FVM, Aquamarine, CloudFlare Workers, Bacalhau, web3.storage invocations, IPFS-FAN

Slide 19

Slide 19

Requirements Juan’s Triangle

Slide 20

Slide 20

Requirements Antigoals?

Slide 21

Slide 21

Requirements Antigoals? Suggestions? 🙏

Slide 22

Slide 22

Execution-as-IPLD Interplanetary Linked Invocation (IPLI)

Slide 23

Slide 23

Execution-as-IPLD Invocation via IPLD

Slide 24

Slide 24

Execution-as-IPLD Invocation via IPLD Description of jobs & results Index and/or names for later lookup Streams of results per machine

Slide 25

Slide 25

Execution-as-IPLD Invocation via IPLD IPLI Input Description of jobs & results Index and/or names for later lookup Streams of results per machine f Arguments Scheduling Con ig, etc

Slide 26

Slide 26

Execution-as-IPLD Invocation via IPLD IPLI Output IPLI Input Description of jobs & results Index and/or names for later lookup Streams of results per machine f Arguments Scheduling Con ig, etc

Slide 27

Slide 27

Execution-as-IPLD Invocation via IPLD IPLI Output IPLI Input Description of jobs & results Results Index and/or names for later lookup Streams of results per machine f Arguments Scheduling Con ig, etc More (Stats etc)

Slide 28

Slide 28

Execution-as-IPLD Job Streams (Scheduler + Events)

Slide 29

Slide 29

Execution-as-IPLD Job Streams (Scheduler + Events) Pure E ect Stream Pure Function Stream ff Base Event Stream

Slide 30

Slide 30

Execution-as-IPLD Job Streams (Scheduler + Events) Pure E ect Stream Pure Function Stream ff Base Event Stream t→

Slide 31

Slide 31

Execution-as-IPLD Job Streams (Scheduler + Events) Pure E ect Stream Pure Function Stream ff Base Event Stream t→

Slide 32

Slide 32

Execution-as-IPLD Job Streams (Scheduler + Events) Pure E ect Stream Pure Function Stream ff Base Event Stream t→

Slide 33

Slide 33

Execution-as-IPLD Job Streams (Scheduler + Events) Pure E ect Stream Pure Function Stream ff Base Event Stream t→

Slide 34

Slide 34

Execution-as-IPLD Job Streams (Scheduler + Events) Pure E ect Stream Pure Function Stream ff Base Event Stream t→

Slide 35

Slide 35

Execution-as-IPLD With a Little Scale From My Friends

Slide 36

Slide 36

Execution-as-IPLD Throughput With a Little Scale From My Friends Parallelization

Slide 37

Slide 37

Execution-as-IPLD With a Little Scale From My Friends Throughput Ideal (Linear) Parallelization

Slide 38

Slide 38

Execution-as-IPLD With a Little Scale From My Friends Ideal (Linear) Throughput Amdahl’s Law Parallelization

Slide 39

Slide 39

Execution-as-IPLD With a Little Scale From My Friends Ideal (Linear) Throughput Amdahl’s Law Universal Scaling Law Parallelization

Slide 40

Slide 40

Execution-as-IPLD With a Little Scale From My Friends Ideal (Linear) Throughput Amdahl’s Law Incoherence, Data Contention Parallelization Universal Scaling Law

Slide 41

Slide 41

Execution-as-IPLD With a Little Scale From My Friends Throughput Global Adaptive Optimization 🤯 Ideal (Linear) Amdahl’s Law Incoherence, Data Contention Parallelization Universal Scaling Law

Slide 42

Slide 42

Execution-as-IPLD Cache Intermediate Results

Slide 43

Slide 43

Execution-as-IPLD Cache Intermediate Results 🚰 🚰

Slide 44

Slide 44

Execution-as-IPLD Cache Intermediate Results 🚰 🚰

Slide 45

Slide 45

Execution-as-IPLD Cache Intermediate Results 🚰 🚰

Slide 46

Slide 46

Execution-as-IPLD Cache Intermediate Results 🚰 🚰 🚰

Slide 47

Slide 47

Execution-as-IPLD Cache Intermediate Results 🚰 Suspend / resume 🚰 🚰

Slide 48

Slide 48

Execution-as-IPLD Cache Intermediate Results 🚰 Suspend / resume 🚰 🚰

Slide 49

Slide 49

Execution-as-IPLD Feedback Loop

Slide 50

Slide 50

Execution-as-IPLD Feedback Loop

  1. Fetch data

Slide 51

Slide 51

Execution-as-IPLD Feedback Loop

  1. Fetch data 2. Compute on data

Slide 52

Slide 52

Execution-as-IPLD Feedback Loop

  1. Fetch data 2. Compute on data 3. Output more data

Slide 53

Slide 53

Execution-as-IPLD Feedback Loop

  1. Fetch data 2. Compute on data 3. Output more data 4. GOTO 2

Slide 54

Slide 54

Execution-as-IPLD Feedback Loop

  1. Fetch data 2. Compute on data 3. Output more data 4. GOTO 2

Slide 55

Slide 55

Execution-as-IPLD Managed Effects Pure E ect Stream Pure Function Stream ff Base Event Stream t→

Slide 56

Slide 56

Execution-as-IPLD Managed Effects Managed E ect Stream Pure E ect Stream Pure Function Stream ff ff Base Event Stream t→

Slide 57

Slide 57

Execution-as-IPLD Managed Effects Managed E ect Stream Pure E ect Stream Pure Function Stream ff ff Base Event Stream t→

Slide 58

Slide 58

Execution-as-IPLD Managed Effects Managed E ect Stream Pure E ect Stream Pure Function Stream ff ff Base Event Stream t→

Slide 59

Slide 59

Execution-as-IPLD Managed Effects Managed E ect Stream Pure E ect Stream Pure Function Stream ff ff Base Event Stream t→

Slide 60

Slide 60

Execution-as-IPLD Managed Effects 🚀 Managed E ect Stream Pure E ect Stream Pure Function Stream ff ff Base Event Stream t→

Slide 61

Slide 61

Execution-as-IPLD Managed Effects 🚀 Managed E ect Stream Pure E ect Stream Pure Function Stream ff ff Base Event Stream t→

Slide 62

Slide 62

IPVM for IPFS Internals Shipping IPFS in IPFS

Slide 63

Slide 63

IPVM for IPFS Internals Content Addressing IPFS Itself

Slide 64

Slide 64

IPVM for IPFS Internals Content Addressing IPFS Itself New codecs (autocodec) Cryptography Smarter chunkers, incremental verifiers Critical bugfixes ff Share e ort between projects (Kubo, Iroh, UCAN, WNFS, Skip Ratchet, etc)

Slide 65

Slide 65

Security How Trusted is Your Execution?

Slide 66

Slide 66

Security Models Mobile computing OCAP, eRights, encryption, UCAN http://erights.org/elib/concurrency/msg-passing.html

Slide 67

Slide 67

Security More Capabilities, More Problems

Slide 68

Slide 68

Security More Capabilities, More Problems Minimum Viable Capability Option to switch on more powerful, trusted features Remote capabilities on other people’s systems (service providers, peers, etc)

Slide 69

Slide 69

State Channels Remote Deals & Execution

Slide 70

Slide 70

State Channels Eager Job Discovery 💁 💁 Discover providers (“matchmaking DHT”) Register providers, just like a bootstrap list ⚙ 💁 (Optional) reputation 💁 ⚙

Slide 71

Slide 71

State Channels Payments

Slide 72

Slide 72

State Channels Payments Reuse generalized state channels for (optional) payment, reputation, etc (Future: hierarchical consensus)

Slide 73

Slide 73

Where to Start 🗺

Slide 74

Slide 74

Where to Start Bootstrapping First Steps: Brass Tacks

Slide 75

Slide 75

Where to Start Bootstrapping First Steps: Brass Tacks Ship Wasm into an IPFS implementation Manual invocation from CLI IPLI format experimentation Concurrent job scheduler, trust & resource limits Figure out sensible default configs from experience Experiment with deeper integration: wasm-ipld or similar Cron, event triggers, etc Push jobs, associated authZ

Slide 76

Slide 76

Open Discussion 🌶 u o y y h w ! e s a m l e l d e i T e s e h t 💔 r o ❤

Slide 77

Slide 77

Open Discussion 🌶 u o y y h w ! e s a m l e l d e i T e s e h t 💔 r o ❤

Slide 78

Slide 78

🎉 Thank You, IPFS þing 🇮🇸 https://fission.codes brooklyn@fission.codes @expede

Slide 79

Slide 79

Slide 80

Slide 80

Therefore, the worse-is-better software first will […] be improved to a point that is almost the right thing. In concrete terms, even though Lisp compilers in 1987 were about as good as C compilers, there are many more compiler experts who want to make C compilers better than want to make Lisp compilers better. – Richard P. Gabriel, Lisp: Good News, Bad News, How to Win Big (1991)