Beyond UnixFS: Extending IPFS for Edge Apps & Web3

🪐 WebNative File System 🚀 Beyond UnixFS: Extending IPFS for Edge Apps & Web3 github.com/WebNativeFileSystem

BROOKLYN ZELENKA PHILIPP KRÜGER @expede CTO @ Fission @matheus23 Protocol Engineer @ Fission Won’t have time to cover everything in depth. Come talk to us after, we’re friendly & happy to share what we’ve learned along the way!

BROOKLYN ZELENKA PHILIPP KRÜGER @expede CTO @ Fission @matheus23 Protocol Engineer @ Fission H C TE Won’t have time to cover everything in depth. Come talk to us after, we’re friendly & happy to share what we’ve learned along the way!

Beyond UnixFS 🚀 Requirements Diagram Web3 Apps Portable Data & Runtime Hostile Hosts e.g. Browsers Nonextractable Signing Keys Web 2.0 Feature Partity Access Control Authenticated Data Content Addressed Authorized Data Encryption at Rest Capability Auth Collaborative Extensible Multiuser Arbitrary Metadata Concurrent Kernel vs Userspace

Beyond UnixFS 🚀 Drive (File Explorer)

Beyond UnixFS 🚀 High-Level Layout

Beyond UnixFS 🚀 High-Level Layout IPNS/DNSLink alice.fission.name

Beyond UnixFS 🚀 High-Level Layout Public Photos Avatars Apps IPNS/DNSLink alice.fission.name

Beyond UnixFS 🚀 High-Level Layout Public Photos Avatars IPNS/DNSLink alice.fission.name Secret Apps Photos Apps Family Photos My Gallery

Beyond UnixFS 🚀 High-Level Layout Public Photos Avatars IPNS/DNSLink alice.fission.name Secret Apps Sharing Inbox Photos Apps Family Photos My Gallery Keys and Keys Keysand and Pointers Pointers Pointers

Beyond UnixFS 🚀 High-Level Layout Public Photos Avatars IPNS/DNSLink alice.fission.name Secret Apps Sharing Inbox Photos Apps Family Photos My Gallery Keys and Keys Keysand and Pointers Pointers Pointers Sharing Outbox Keys and Pointers

Public Files 📂 General WNFS Data Model Public Photos Avatars Apps

Public Files 📂 Virtual Nodes

Public Files 📂 Virtual Nodes Raw IPLD Node

Public Files 📂 Virtual Nodes Raw IPLD Node File Node Raw Data Metadata

Public Files 📂 Virtual Nodes File Node Raw IPLD Node Virtual Node Raw Data Consistent interface Arbitrary metadata Tags, creators, MIME, sources, &c Metadata

Public Files 📂 Virtual Nodes File Node Raw IPLD Node Virtual Node Raw Data Consistent interface Arbitrary metadata Tags, creators, MIME, sources, &c Metadata Directory Node Index Metadata

Public Files 📂 Hard & Soft Links

Public Files 📂 Hard & Soft Links alice.fission.name

Public Files 📂 Hard & Soft Links alice.fission.name bob.fission.name

Public Files 📂 Hard Links Hard & Soft Links New for the web! Direct reference alice.fission.name 2 pointers ~ deduplicate bob.fission.name

Public Files 📂 Hard Links Hard & Soft Links New for the web! Direct reference alice.fission.name 2 pointers ~ deduplicate bob.fission.name Soft Links Symlink or web link (URL) Pointer ~ latest Latest may break

Public Files 📂 Persistent Versioning

Public Files 📂 Persistent Versioning Photos@r0 Vacation Avatars@r0 beach.png caricature.jpg Revision 0

Public Files 📂 Persistent Versioning Photos@r0 Vacation Avatars@r0 beach.png caricature.jpg Revision 0 🕙

Public Files 📂 Persistent Versioning Photos@r0 Photos@r1 Vacation Avatars@r0 Avatars@r1 beach.png caricature.jpg headshot.png Revision 0 Revision 1 🕙

Public Files 📂 Persistent Versioning Photos@r0 ChildHasEvent Vacation Avatars@r0 beach.png caricature.jpg Photos@r1 InsertNew Revision 0 headshot.png Revision 1 🕙 Avatars@r1

Public Files 📂 Rearranged Photos@r1 ChildHasEvent Avatars@r1 Photos@r0 InsertNew Vacation Avatars@r0 beach.png caricature.jpg headshot.png

Secret Files 🥷 Confluence

Secret Files 🥷 Confluence One of the first Merkle CRDT papers was from PL 🙌 Persistent data structure Automatic file-level reconciliation Pluggable sub-file reconciliation (forthcoming) Basis of upcoming BFT Datalog “at scale” work 😉

Secret Files 🥷 Confluence One of the first Merkle CRDT papers was from PL 🙌 Persistent data structure Automatic file-level reconciliation Pluggable sub-file reconciliation (forthcoming) Basis of upcoming BFT Datalog “at scale” work 😉 Single File History / “Causal Shadow”

Secret Files 🥷 Confluence F B E One of the first Merkle CRDT papers was from PL 🙌 Persistent data structure D Automatic file-level reconciliation Pluggable sub-file reconciliation (forthcoming) Basis of upcoming BFT Datalog “at scale” work 😉 A C Single File History / “Causal Shadow”

Secret Files 🥷 A Dark Forest of Data

Secret Files 🥷 Private Nodes Binary CBOR Encrypted Node 🔒 AES256 Encrypted Node 🔒 Virtual Node 🔑 Index Metadata Encrypted Node 🔒 🔑 🔑 🔑

Secret Files 🥷 Read Window Virtual Node Virtual Node 🔑 Index 🔑 🔑 Metadata 🔑 🔑 🔑 Index Metadata

Secret Files 🥷 Read Window Virtual Node Virtual Node 🔑 Index 🔑 🔑 Metadata 🔑 { 🔑 🔑 name: “beach.jpg”, key: “B374A26A71490437A…”, … } Index Metadata

Secret Files 🥷 Subgraph Access

Secret Files 🥷 Subgraph Access 🗝

Secret Files 🥷 Subgraph Access 🗝 🗝

Secret Files 🥷 Subgraph Access 🗝 🗝 🗝

Secret Files 🥷 Subgraph Access 🗝 🗝 🗝 🔩➡🗝

Secret Files 🥷 Progressive Fast Forward Rev 0

Secret Files 🥷 Progressive Fast Forward Rev 0 Rev 1 (Partial)

Secret Files 🥷 Progressive Fast Forward Rooting progress Rev 0 Rev 1 (Partial)

Secret Files 🥷 Progressive Fast Forward Rooting progress Rev 0 Rev 1 (Partial) Rev 2 (Partial)

Secret Files 🥷 Progressive Fast Forward Rooting progress Rev 3 (Complete) Rev 0 Rev 1 (Partial) Rev 2 (Partial)

Secret Files 🥷 Progressive Fast Forward Rev 3 (Complete) Rev 0 Rev 1 (Partial) Rev 2 (Partial)

Secret Files 🥷 Skip Ratchet

Secret Files 🥷 Skip Ratchet Ratchet keys for future (backwards) secrecy Skip ratchet KDF for log-time fast forwards

Secret Files 🥷 Skip Ratchet Ratchet keys for future (backwards) secrecy Skip ratchet KDF for log-time fast forwards Inter e s ted in Ask the fu Brook ll pap lyn f er? or a p repri nt

Secret Files 🥷 Write Access

Secret Files 🥷 Write Access Bare Filter simple = Bloom.insert(element: iNumber, into: parentFilter) versioned = Bloom.insert(element: hash(aesKey), into: simple) Saturation nextStep = Bloom.insert(element: xof_hash(n++), into: prevStep) Repeat until threshold bits flipped “Optimal” threshold depends on Bloom configuration, but often ~49% of bits

Secret Files 🥷 Dark Forest

Secret Files 🥷 Reasonably Efficient Secret KV

Secret Files 🥷 Reasonably Efficient Secret KV HAMT (weight 16)

Secret Files 🥷 Reasonably Efficient Secret KV HAMT 163 = 4,096 buckets (weight 16) 164 = 65,536 buckets

Secret Files 🥷 Reasonably Efficient Secret KV HAMT 163 = 4,096 buckets (weight 16) 164 = 65,536 buckets Append-only Quick Read/Write Merkleized Concurrency Friendly

{brooklyn,philipp}@fission.codes @expede & @matheus23 🎉 Thank You, IPFS þing 🇮🇸 https://whitepaper.fission.codes ✨ github.com/WebNativeFileSystem(/spec) ✨ More in-depth at Strange Loop 2022 (or talk to us today)