WebNative File System

Fission Web Native File System (WNFS)

Web Native File System Brooklyn Zelenka, @expede

Web Native File System Brooklyn Zelenka, @expede • Cofounder/CTO at Fission • PLT, VMs, FP, Distributed Systems • Previously an Ethereum Core Dev • FOSS — Witchcraft, Exceptional, &c

Web Native 🚀

Web Native Web Apps are Too Hard — Shrink the Development Cycle

Web Native Web Apps are Too Hard — Shrink the Development Cycle Users 👨🏫👩🏭🧑⚕👷 Developer 👩💻

Web Native Web Apps are Too Hard — Shrink the Development Cycle Users 👨🏫👩🏭🧑⚕👷 Browser 🖥 API / GQL ↕ Server ⚙ Data Store 🗃 DevOps 📤 Developer 👩💻

Web Native Web Apps are Too Hard — Shrink the Development Cycle Users 👨🏫👩🏭🧑⚕👷 Browser 🖥 API / GQL ↕ Server ⚙ Data Store 🗃 DevOps 📤 Developer 👩💻 Users 👨🏫👩🏭🧑⚕👷 Browser 🖥 Developer 👩💻 Fission ⚛

Web Native Web Apps are Too Hard — Shrink the Development Cycle Users 👨🏫👩🏭🧑⚕👷 Browser 🖥 API / GQL ↕ Current Server ⚙ Data Store 🗃 DevOps 📤 Developer 👩💻 Users 👨🏫👩🏭🧑⚕👷 Browser 🖥 Developer 👩💻 Fission ⚛

Web Native Web Apps are Too Hard — Shrink the Development Cycle Users 👨🏫👩🏭🧑⚕👷 Browser 🖥 API / GQL ↕ Current Server ⚙ Data Store 🗃 DevOps 📤 Developer 👩💻 Users 👨🏫👩🏭🧑⚕👷 Browser 🖥 Developer 👩💻 Fission ⚛ New

Web Native Web Apps are Too Hard — Shrink the Development Cycle Users 👨🏫👩🏭🧑⚕👷 Browser 🖥 Current Users 👨🏫👩🏭🧑⚕👷 Browser 🖥 Fission ⚛ API / GQL ↕ Developer 👩💻 Server ⚙ Quick iteration 🔄 Data Store 🗃 DevOps 📤 Developer 👩💻 Focus on end users 💁 Lower barrier to entry 👨🎨 New

Web Native New assumptions ~ new approach • Different world from web’s beginnings • Don’t need to rely on client/server • New features naturally fall out of the architecture • Location independent data • Client side encryption • Networked machines → networked data

Web Native Abstraction Layers

Web Native Abstraction Layers 🎮 Application Layer Business Logic 🛠 Platform Layer WebNative File System 🌐 Protocol Layer IPFS

Web Native Components

Web Native Components Write Access UCAN AuthN & AuthZ Read Access OCAP Identity did:key

Web Native Components Write Access UCAN Networking Stack DNS, IPFS, PubSub AuthN & AuthZ Read Access OCAP Identity did:key

Web Native Components Durable Store WebNative File System Data Layer Write Access UCAN Networking Stack DNS, IPFS, PubSub AuthN & AuthZ Read Access OCAP Identity did:key

Web Native Components Multi Device / User Offline & Async Sharing Exchange Store Soft Realtime Multiuser Automerge CRDT Durable Store WebNative File System Data Layer Write Access UCAN Networking Stack DNS, IPFS, PubSub AuthN & AuthZ Read Access OCAP Identity did:key

Web Native Components App Abstractions Fission SDK Multi Device / User Offline & Async Sharing Exchange Store Soft Realtime Multiuser Automerge CRDT Durable Store WebNative File System Data Layer Write Access UCAN Networking Stack DNS, IPFS, PubSub AuthN & AuthZ Read Access OCAP Identity did:key

Web Native Dev’s App Business Logic & View Components App Abstractions Fission SDK Multi Device / User Offline & Async Sharing Exchange Store Soft Realtime Multiuser Automerge CRDT Durable Store WebNative File System Data Layer Write Access UCAN Networking Stack DNS, IPFS, PubSub AuthN & AuthZ Read Access OCAP Identity did:key

Web Native File System 🗂🚀

File System 🛠 High Level Layout

File System 🛠 High Level Layout boris.fission.name

File System 🛠 High Level Layout Public Photos Avatars Apps boris.fission.name

File System boris.fission.name 🛠 High Level Layout Public Photos Avatars Private Apps Photos Apps Family Photos My Gallery

File System boris.fission.name 🛠 High Level Layout Public Photos Avatars Private Apps Shared By Me Photos Apps Family Photos My Gallery Keys and Keys Keysand and Pointers Pointers Pointers

File System boris.fission.name 🛠 High Level Layout Public Photos Avatars Private Apps Shared By Me Photos Apps Family Photos My Gallery Keys and Keys Keysand and Pointers Pointers Pointers Shared w/ Me Keys and Pointers

File System Quick Demo 📺

Common Concepts 🤝

Common Concepts 🛠🌐 Virtual Nodes

Common Concepts 🛠🌐 Virtual Nodes Raw Node

Common Concepts 🛠🌐 Virtual Nodes Raw Node File Node Raw Data Metadata

Common Concepts 🛠🌐 Virtual Nodes Raw Node File Node Raw Data Metadata Directory Node Index Metadata

Common Concepts 🛠🌐 Virtual Nodes Raw Node • Virtual Node • Consistent interface • Arbitrary metadata • More than POSIX • Tags, creators, MIME, sources, &c File Node Raw Data Metadata Directory Node Index Metadata

Common Concepts 🛠 Hard vs Soft Links

Common Concepts 🛠 Hard vs Soft Links • Hard links • Direct reference • 2 pointers ~ duplicate

Common Concepts 🛠 Hard vs Soft Links • Hard links • Direct reference • 2 pointers ~ duplicate • Soft links • Like a symlink or web link • 2 pointers ~ latest • May break • Always some version available

Common Concepts 🛠 Hard vs Soft Links • Hard links • Direct reference • 2 pointers ~ duplicate • Soft links • Like a symlink or web link • 2 pointers ~ latest • May break • Always some version available

Common Concepts 🛠 Hard vs Soft Links • Hard links • Direct reference • 2 pointers ~ duplicate • Soft links • Like a symlink or web link • 2 pointers ~ latest • May break • Always some version available

Common Concepts 🛠 Hard vs Soft Links • Hard links • Direct reference • 2 pointers ~ duplicate • Soft links • Like a symlink or web link • 2 pointers ~ latest • May break • Always some version available

Common Concepts 🛠 Hard vs Soft Links • Hard links • Direct reference • 2 pointers ~ duplicate • Soft links • Like a symlink or web link • 2 pointers ~ latest • May break • Always some version available

File System 🛠 The z-dimension: versioning & events

File System 🛠 The z-dimension: versioning & events Photos@r0 Vacation Avatars@r0 beach.png caricature.jpg Revision 0

File System 🛠 The z-dimension: versioning & events Photos@r0 Vacation Avatars@r0 beach.png caricature.jpg Revision 0 🕙

File System 🛠 The z-dimension: versioning & events Photos@r0 Photos@r1 Vacation Avatars@r0 Avatars@r1 beach.png caricature.jpg headshot.png Revision 0 Revision 1 🕙

File System 🛠 The z-dimension: versioning & events Photos@r0 ChildEvent Vacation Avatars@r0 beach.png caricature.jpg Photos@r1 InsertNew Revision 0 headshot.png Revision 1 🕙 Avatars@r1

File System 🛠 The z-dimension: versioning & events Photos@r0 ChildEvent Vacation Avatars@r0 beach.png caricature.jpg Photos@r1 InsertNew Revision 0 headshot.png Revision 1 🕙 Avatars@r1

File System Generation 0 🛠 The z-dimension: versioning & events Photos@r0 ChildEvent Vacation Avatars@r0 beach.png caricature.jpg Photos@r1 InsertNew Revision 0 headshot.png Revision 1 🕙 Avatars@r1

File System Generation 0 🛠 The z-dimension: versioning & events Photos@r0 ChildEvent Vacation Avatars@r0 beach.png caricature.jpg Generation 1 Photos@r1 InsertNew Revision 0 headshot.png Revision 1 🕙 Avatars@r1

File System 🛠 Same — Rooted DAG Photos@r1 Photos@r0 Merkle Root Avatars@r1 Vacation beach.png Avatars@r0 caricature.jpg headshot.png

Private Nodes 🤫

Private Nodes 🛠 Components CBOR Binary Encrypted Node 🔒 AES256 + 🔑 Virtual Node = Index 🔑 🔑 Metadata 🔑

Private Nodes 🛠 Cryptree Virtual Node Virtual Node 🔑 Index 🔑 🔑 Metadata 🔑 🔑 🔑 Index Metadata

Private Nodes 🛠 Cryptree Virtual Node Virtual Node 🔑 Index 🔑 🔑 Metadata 🔑 { 🔑 name: “beach.jpg”, revision: 42, key: “B374A26A71490437A…” 🔑 } Index Metadata

🔑 Private Nodes 🛠 Cryptree Virtual Node Virtual Node 🔑 Index 🔑 🔑 Metadata 🔑 { 🔑 name: “beach.jpg”, revision: 42, key: “B374A26A71490437A…” 🔑 } Index Metadata

🔑 Private Nodes 🛠 Cryptree Virtual Node Virtual Node 🔑 Index 🔑 🔑 Metadata 🔑 { 🔑 name: “beach.jpg”, revision: 42, key: “B374A26A71490437A…” 🔑 } Index Metadata

Private Nodes 🌐 Private Data Store

Private Nodes 🌐 Private Data Store MMPT (weight 16)

Private Nodes 🌐 Private Data Store MMPT 16^3 = 4,096 items (weight 16) 16^4 = 65,536 items

Private Nodes 🌐 Private Data Store MMPT 16^3 = 4,096 items (weight 16) 16^4 = 65,536 items Append-only Quick Read/Write Merkleized Concurrency Friendly

Private Nodes 🌐 Private Data Store MMPT 16^3 = 4,096 items (weight 16) 16^4 = 65,536 items Append-only Quick Read/Write Merkleized Concurrency Friendly 🔑

Private Nodes Namefilters

Private Nodes Namefilters • Constraints: • Deterministic • Versioned • Addressable • Prove subpath for UCAN • Minimal knowledge • AES keys ~ path segments but secret

Private Nodes Namefilters • Constraints: • Deterministic • Versioned • Addressable • Prove subpath for UCAN • Minimal knowledge • AES keys ~ path segments but secret • bareFilter • parentFilter • AND bloom(SHA(aesKey)) • AND bloom(SHA(aesKey ++ revision)) • Saturation • nameFilter AND bloom(SHA(nameFilter)) • Repeat until threshold bits flipped

Private Nodes 🛠 Writing as Much as UCAN

Private Nodes 🛠 Writing as Much as UCAN Rev 0

Private Nodes 🛠 Writing as Much as UCAN Rev 0

Private Nodes 🛠 Writing as Much as UCAN Rev 0 Rev 1 (Partial)

Private Nodes 🛠 Writing as Much as UCAN Rooting progress Rev 0 Rev 1 (Partial)

Private Nodes 🛠 Writing as Much as UCAN Rooting progress Rev 0 Rev 1 (Partial)

Private Nodes 🛠 Writing as Much as UCAN Rooting progress No common root at this layer! Attached via MMPT Rev 0 Rev 1 (Partial)

Private Nodes 🛠 Progressive Fast Forward Rev 0 Rev 1 (Partial)

Private Nodes 🛠 Progressive Fast Forward Rev 0 Rev 1 (Partial) Rev 2 (Partial)

Private Nodes 🛠 Progressive Fast Forward Rev 3 (Complete) Rev 0 Rev 1 (Partial) Rev 2 (Partial)

Private Nodes 🛠 Progressive Fast Forward Rev 3 (Complete) Rev 0 Rev 1 (Partial) Rev 2 (Partial)

Private Nodes 🛠 Progressive Fast Forward Rev 3 (Complete) Rev 0 Rev 1 (Partial) Rev 2 (Partial)

Private Nodes 🛠 Progressive Fast Forward Rev 3 (Complete) Rev 0 Rev 1 (Partial) Rev 2 (Partial)

Private Nodes current rev + 2n 🛠 Progressive Fast Forward Rev 3 (Complete) Rev 0 Rev 1 (Partial) Rev 2 (Partial)

Private Nodes current rev + 2n 🛠 Progressive Fast Forward Rev 3 (Complete) Rev 0 Rev 1 (Partial) Rev 2 (Partial)

Private Nodes current rev + 2n —2m current rev + 2n 🛠 Progressive Fast Forward Rev 3 (Complete) Rev 0 Rev 1 (Partial) Rev 2 (Partial)

Private Nodes current rev + 2n —2m current rev + 2n 🛠 Progressive Fast Forward Rev 3 (Complete) Rev 0 Rev 1 (Partial) Rev 2 (Partial)

Private Nodes current rev + 2n —2m current rev + 2n 🛠 Progressive Fast Forward Rev 3 (Complete) Rev 0 Rev 1 (Partial) Rev 2 (Partial)

Private Nodes current rev + 2n —2m current rev + 2n 🛠 Progressive Fast Forward O(2 * log n) Rev 3 (Complete) Rev 0 Rev 1 (Partial) Rev 2 (Partial)

File Sharing 🎁

File Sharing Shared by/with Me

File Sharing Shared by/with Me Shared by Me

File Sharing Shared by Me Shared by/with Me did:key:zStEksDrxkwYmpzqB did:key:zStEksDrxkwYmpzqB did:key:zStEksDrxkwYmpzqB dAQjjx1PRbHG3fq4ChGeJcYU dAQjjx1PRbHG3fq4ChGeJcYU dAQjjx1PRbHG3fq4ChGeJcYU YU44a4CBUExTTjeCbop6Uur YU44a4CBUExTTjeCbop6Uur YU44a4CBUExTTjeCbop6Uur

File Sharing Shared by Me Shared by/with Me did:key:zStEksDrxkwYmpzqB did:key:zStEksDrxkwYmpzqB did:key:zStEksDrxkwYmpzqB dAQjjx1PRbHG3fq4ChGeJcYU dAQjjx1PRbHG3fq4ChGeJcYU dAQjjx1PRbHG3fq4ChGeJcYU YU44a4CBUExTTjeCbop6Uur YU44a4CBUExTTjeCbop6Uur YU44a4CBUExTTjeCbop6Uur Human Readable Name 🔑 Symlink

File Sharing Shared by Me Shared by/with Me did:key:zStEksDrxkwYmpzqB did:key:zStEksDrxkwYmpzqB did:key:zStEksDrxkwYmpzqB dAQjjx1PRbHG3fq4ChGeJcYU dAQjjx1PRbHG3fq4ChGeJcYU dAQjjx1PRbHG3fq4ChGeJcYU YU44a4CBUExTTjeCbop6Uur YU44a4CBUExTTjeCbop6Uur YU44a4CBUExTTjeCbop6Uur Human Readable Name 🔑 Symlink

File Sharing Shared by Me Shared by/with Me did:key:zStEksDrxkwYmpzqB did:key:zStEksDrxkwYmpzqB did:key:zStEksDrxkwYmpzqB dAQjjx1PRbHG3fq4ChGeJcYU dAQjjx1PRbHG3fq4ChGeJcYU dAQjjx1PRbHG3fq4ChGeJcYU YU44a4CBUExTTjeCbop6Uur YU44a4CBUExTTjeCbop6Uur YU44a4CBUExTTjeCbop6Uur Human Readable Name 🔑 Symlink

File Sharing Shared by Me Shared by/with Me Shared with Me did:key:zStEksDrxkwYmpzqB did:key:zStEksDrxkwYmpzqB did:key:zStEksDrxkwYmpzqB dAQjjx1PRbHG3fq4ChGeJcYU dAQjjx1PRbHG3fq4ChGeJcYU dAQjjx1PRbHG3fq4ChGeJcYU YU44a4CBUExTTjeCbop6Uur YU44a4CBUExTTjeCbop6Uur YU44a4CBUExTTjeCbop6Uur Human Readable Name 🔑 Symlink

Recap 🔁

Recap boris.fission.name Public Photos Avatars Private Apps Shared By Me Photos Apps Family Photos My Gallery Keys and Keys Keysand and Pointers Pointers Pointers Shared w/ Me Keys and Pointers

https://fission.codes https://talk .fission.codes h t t p s : / /d i s c o r d . g g / z A Q B D E q 😁 THANK YOU 🎉 brooklyn@fission.codes g i t h u b . c o m /e x p e d e @expede

https://fission.codes https://talk .fission.codes h t t p s : / /d i s c o r d . g g / z A Q B D E q 😁 THANK YOU 🎉 brooklyn@fission.codes g i t h u b . c o m /e x p e d e @expede