A presentation at IPFS þing in in Reykjavík, Iceland by Brooklyn Zelenka
🪐 WebNative File System 🚀 Beyond UnixFS: Extending IPFS for Edge Apps & Web3 github.com/WebNativeFileSystem
BROOKLYN ZELENKA PHILIPP KRÜGER @expede CTO @ Fission @matheus23 Protocol Engineer @ Fission Won’t have time to cover everything in depth. Come talk to us after, we’re friendly & happy to share what we’ve learned along the way!
BROOKLYN ZELENKA PHILIPP KRÜGER @expede CTO @ Fission @matheus23 Protocol Engineer @ Fission H C TE Won’t have time to cover everything in depth. Come talk to us after, we’re friendly & happy to share what we’ve learned along the way!
Beyond UnixFS 🚀 Requirements Diagram Web3 Apps Portable Data & Runtime Hostile Hosts e.g. Browsers Nonextractable Signing Keys Web 2.0 Feature Partity Access Control Authenticated Data Content Addressed Authorized Data Encryption at Rest Capability Auth Collaborative Extensible Multiuser Arbitrary Metadata Concurrent Kernel vs Userspace
Beyond UnixFS 🚀 Drive (File Explorer)
Beyond UnixFS 🚀 High-Level Layout
Beyond UnixFS 🚀 High-Level Layout IPNS/DNSLink alice.fission.name
Beyond UnixFS 🚀 High-Level Layout Public Photos Avatars Apps IPNS/DNSLink alice.fission.name
Beyond UnixFS 🚀 High-Level Layout Public Photos Avatars IPNS/DNSLink alice.fission.name Secret Apps Photos Apps Family Photos My Gallery
Beyond UnixFS 🚀 High-Level Layout Public Photos Avatars IPNS/DNSLink alice.fission.name Secret Apps Sharing Inbox Photos Apps Family Photos My Gallery Keys and Keys Keysand and Pointers Pointers Pointers
Beyond UnixFS 🚀 High-Level Layout Public Photos Avatars IPNS/DNSLink alice.fission.name Secret Apps Sharing Inbox Photos Apps Family Photos My Gallery Keys and Keys Keysand and Pointers Pointers Pointers Sharing Outbox Keys and Pointers
Public Files 📂 General WNFS Data Model Public Photos Avatars Apps
Public Files 📂 Virtual Nodes
Public Files 📂 Virtual Nodes Raw IPLD Node
Public Files 📂 Virtual Nodes Raw IPLD Node File Node Raw Data Metadata
Public Files 📂 Virtual Nodes File Node Raw IPLD Node Virtual Node Raw Data Consistent interface Arbitrary metadata Tags, creators, MIME, sources, &c Metadata
Public Files 📂 Virtual Nodes File Node Raw IPLD Node Virtual Node Raw Data Consistent interface Arbitrary metadata Tags, creators, MIME, sources, &c Metadata Directory Node Index Metadata
Public Files 📂 Hard & Soft Links
Public Files 📂 Hard & Soft Links alice.fission.name
Public Files 📂 Hard & Soft Links alice.fission.name bob.fission.name
Public Files 📂 Hard Links Hard & Soft Links New for the web! Direct reference alice.fission.name 2 pointers ~ deduplicate bob.fission.name
Public Files 📂 Hard Links Hard & Soft Links New for the web! Direct reference alice.fission.name 2 pointers ~ deduplicate bob.fission.name
Public Files 📂 Hard Links Hard & Soft Links New for the web! Direct reference alice.fission.name 2 pointers ~ deduplicate bob.fission.name Soft Links Symlink or web link (URL) Pointer ~ latest Latest may break
Public Files 📂 Persistent Versioning
Public Files 📂 Persistent Versioning Photos@r0 Vacation Avatars@r0 beach.png caricature.jpg Revision 0
Public Files 📂 Persistent Versioning Photos@r0 Vacation Avatars@r0 beach.png caricature.jpg Revision 0 🕙
Public Files 📂 Persistent Versioning Photos@r0 Photos@r1 Vacation Avatars@r0 Avatars@r1 beach.png caricature.jpg headshot.png Revision 0 Revision 1 🕙
Public Files 📂 Persistent Versioning Photos@r0 ChildHasEvent Vacation Avatars@r0 beach.png caricature.jpg Photos@r1 InsertNew Revision 0 headshot.png Revision 1 🕙 Avatars@r1
Public Files 📂 Persistent Versioning Photos@r0 ChildHasEvent Vacation Avatars@r0 beach.png caricature.jpg Photos@r1 InsertNew Revision 0 headshot.png Revision 1 🕙 Avatars@r1
Public Files 📂 Rearranged Photos@r1 ChildHasEvent Avatars@r1 Photos@r0 InsertNew Vacation Avatars@r0 beach.png caricature.jpg headshot.png
Secret Files 🥷 Confluence
Secret Files 🥷 Confluence One of the first Merkle CRDT papers was from PL 🙌 Persistent data structure Automatic file-level reconciliation Pluggable sub-file reconciliation (forthcoming) Basis of upcoming BFT Datalog “at scale” work 😉
Secret Files 🥷 Confluence One of the first Merkle CRDT papers was from PL 🙌 Persistent data structure Automatic file-level reconciliation Pluggable sub-file reconciliation (forthcoming) Basis of upcoming BFT Datalog “at scale” work 😉 Single File History / “Causal Shadow”
Secret Files 🥷 Confluence F B E One of the first Merkle CRDT papers was from PL 🙌 Persistent data structure D Automatic file-level reconciliation Pluggable sub-file reconciliation (forthcoming) Basis of upcoming BFT Datalog “at scale” work 😉 A C Single File History / “Causal Shadow”
Secret Files 🥷 A Dark Forest of Data
Secret Files 🥷 Private Nodes Binary CBOR Encrypted Node 🔒 AES256 Encrypted Node 🔒 Virtual Node 🔑 Index Metadata Encrypted Node 🔒 🔑 🔑 🔑
Secret Files 🥷 Read Window Virtual Node Virtual Node 🔑 Index 🔑 🔑 Metadata 🔑 🔑 🔑 Index Metadata
Secret Files 🥷 Read Window Virtual Node Virtual Node 🔑 Index 🔑 🔑 Metadata 🔑 { 🔑 🔑 name: “beach.jpg”, key: “B374A26A71490437A…”, … } Index Metadata
Secret Files 🥷 Subgraph Access
Secret Files 🥷 Subgraph Access
Secret Files 🥷 Subgraph Access 🗝
Secret Files 🥷 Subgraph Access 🗝 🗝
Secret Files 🥷 Subgraph Access 🗝 🗝 🗝
Secret Files 🥷 Subgraph Access 🗝 🗝 🗝 🔩➡🗝
Secret Files 🥷 Progressive Fast Forward Rev 0
Secret Files 🥷 Progressive Fast Forward Rev 0 Rev 1 (Partial)
Secret Files 🥷 Progressive Fast Forward Rooting progress Rev 0 Rev 1 (Partial)
Secret Files 🥷 Progressive Fast Forward Rooting progress Rev 0 Rev 1 (Partial) Rev 2 (Partial)
Secret Files 🥷 Progressive Fast Forward Rooting progress Rev 3 (Complete) Rev 0 Rev 1 (Partial) Rev 2 (Partial)
Secret Files 🥷 Progressive Fast Forward Rooting progress Rev 3 (Complete) Rev 0 Rev 1 (Partial) Rev 2 (Partial)
Secret Files 🥷 Progressive Fast Forward Rev 3 (Complete) Rev 0 Rev 1 (Partial) Rev 2 (Partial)
Secret Files 🥷 Progressive Fast Forward Rev 3 (Complete) Rev 0 Rev 1 (Partial) Rev 2 (Partial)
Secret Files 🥷 Skip Ratchet
Secret Files 🥷 Skip Ratchet Ratchet keys for future (backwards) secrecy Skip ratchet KDF for log-time fast forwards
Secret Files 🥷 Skip Ratchet Ratchet keys for future (backwards) secrecy Skip ratchet KDF for log-time fast forwards
Secret Files 🥷 Skip Ratchet Ratchet keys for future (backwards) secrecy Skip ratchet KDF for log-time fast forwards Inter e s ted in Ask the fu Brook ll pap lyn f er? or a p repri nt
Secret Files 🥷 Write Access
Secret Files 🥷 Write Access Bare Filter simple = Bloom.insert(element: iNumber, into: parentFilter) versioned = Bloom.insert(element: hash(aesKey), into: simple) Saturation nextStep = Bloom.insert(element: xof_hash(n++), into: prevStep) Repeat until threshold bits flipped “Optimal” threshold depends on Bloom configuration, but often ~49% of bits
Secret Files 🥷 Dark Forest
Secret Files 🥷 Dark Forest
Secret Files 🥷 Dark Forest
Secret Files 🥷 Dark Forest
Secret Files 🥷 Reasonably Efficient Secret KV
Secret Files 🥷 Reasonably Efficient Secret KV HAMT (weight 16)
Secret Files 🥷 Reasonably Efficient Secret KV HAMT 163 = 4,096 buckets (weight 16) 164 = 65,536 buckets
Secret Files 🥷 Reasonably Efficient Secret KV HAMT 163 = 4,096 buckets (weight 16) 164 = 65,536 buckets Append-only Quick Read/Write Merkleized Concurrency Friendly
{brooklyn,philipp}@fission.codes @expede & @matheus23 🎉 Thank You, IPFS þing 🇮🇸 https://whitepaper.fission.codes ✨ github.com/WebNativeFileSystem(/spec) ✨ More in-depth at Strange Loop 2022 (or talk to us today)
for free. You
can too.