A presentation at IPFS รพing in in Reykjavรญk, Iceland by Brooklyn Zelenka
๐ช WebNative File System ๐ Beyond UnixFS: Extending IPFS for Edge Apps & Web3 github.com/WebNativeFileSystem
BROOKLYN ZELENKA PHILIPP KRรGER @expede CTO @ Fission @matheus23 Protocol Engineer @ Fission Wonโt have time to cover everything in depth. Come talk to us after, weโre friendly & happy to share what weโve learned along the way!
BROOKLYN ZELENKA PHILIPP KRรGER @expede CTO @ Fission @matheus23 Protocol Engineer @ Fission H C TE Wonโt have time to cover everything in depth. Come talk to us after, weโre friendly & happy to share what weโve learned along the way!
Beyond UnixFS ๐ Requirements Diagram Web3 Apps Portable Data & Runtime Hostile Hosts e.g. Browsers Nonextractable Signing Keys Web 2.0 Feature Partity Access Control Authenticated Data Content Addressed Authorized Data Encryption at Rest Capability Auth Collaborative Extensible Multiuser Arbitrary Metadata Concurrent Kernel vs Userspace
Beyond UnixFS ๐ Drive (File Explorer)
Beyond UnixFS ๐ High-Level Layout
Beyond UnixFS ๐ High-Level Layout IPNS/DNSLink alice.fission.name
Beyond UnixFS ๐ High-Level Layout Public Photos Avatars Apps IPNS/DNSLink alice.fission.name
Beyond UnixFS ๐ High-Level Layout Public Photos Avatars IPNS/DNSLink alice.fission.name Secret Apps Photos Apps Family Photos My Gallery
Beyond UnixFS ๐ High-Level Layout Public Photos Avatars IPNS/DNSLink alice.fission.name Secret Apps Sharing Inbox Photos Apps Family Photos My Gallery Keys and Keys Keysand and Pointers Pointers Pointers
Beyond UnixFS ๐ High-Level Layout Public Photos Avatars IPNS/DNSLink alice.fission.name Secret Apps Sharing Inbox Photos Apps Family Photos My Gallery Keys and Keys Keysand and Pointers Pointers Pointers Sharing Outbox Keys and Pointers
Public Files ๐ General WNFS Data Model Public Photos Avatars Apps
Public Files ๐ Virtual Nodes
Public Files ๐ Virtual Nodes Raw IPLD Node
Public Files ๐ Virtual Nodes Raw IPLD Node File Node Raw Data Metadata
Public Files ๐ Virtual Nodes File Node Raw IPLD Node Virtual Node Raw Data Consistent interface Arbitrary metadata Tags, creators, MIME, sources, &c Metadata
Public Files ๐ Virtual Nodes File Node Raw IPLD Node Virtual Node Raw Data Consistent interface Arbitrary metadata Tags, creators, MIME, sources, &c Metadata Directory Node Index Metadata
Public Files ๐ Hard & Soft Links
Public Files ๐ Hard & Soft Links alice.fission.name
Public Files ๐ Hard & Soft Links alice.fission.name bob.fission.name
Public Files ๐ Hard Links Hard & Soft Links New for the web! Direct reference alice.fission.name 2 pointers ~ deduplicate bob.fission.name
Public Files ๐ Hard Links Hard & Soft Links New for the web! Direct reference alice.fission.name 2 pointers ~ deduplicate bob.fission.name
Public Files ๐ Hard Links Hard & Soft Links New for the web! Direct reference alice.fission.name 2 pointers ~ deduplicate bob.fission.name Soft Links Symlink or web link (URL) Pointer ~ latest Latest may break
Public Files ๐ Persistent Versioning
Public Files ๐ Persistent Versioning Photos@r0 Vacation Avatars@r0 beach.png caricature.jpg Revision 0
Public Files ๐ Persistent Versioning Photos@r0 Vacation Avatars@r0 beach.png caricature.jpg Revision 0 ๐
Public Files ๐ Persistent Versioning Photos@r0 Photos@r1 Vacation Avatars@r0 Avatars@r1 beach.png caricature.jpg headshot.png Revision 0 Revision 1 ๐
Public Files ๐ Persistent Versioning Photos@r0 ChildHasEvent Vacation Avatars@r0 beach.png caricature.jpg Photos@r1 InsertNew Revision 0 headshot.png Revision 1 ๐ Avatars@r1
Public Files ๐ Persistent Versioning Photos@r0 ChildHasEvent Vacation Avatars@r0 beach.png caricature.jpg Photos@r1 InsertNew Revision 0 headshot.png Revision 1 ๐ Avatars@r1
Public Files ๐ Rearranged Photos@r1 ChildHasEvent Avatars@r1 Photos@r0 InsertNew Vacation Avatars@r0 beach.png caricature.jpg headshot.png
Secret Files ๐ฅท Confluence
Secret Files ๐ฅท Confluence One of the first Merkle CRDT papers was from PL ๐ Persistent data structure Automatic file-level reconciliation Pluggable sub-file reconciliation (forthcoming) Basis of upcoming BFT Datalog โat scaleโ work ๐
Secret Files ๐ฅท Confluence One of the first Merkle CRDT papers was from PL ๐ Persistent data structure Automatic file-level reconciliation Pluggable sub-file reconciliation (forthcoming) Basis of upcoming BFT Datalog โat scaleโ work ๐ Single File History / โCausal Shadowโ
Secret Files ๐ฅท Confluence F B E One of the first Merkle CRDT papers was from PL ๐ Persistent data structure D Automatic file-level reconciliation Pluggable sub-file reconciliation (forthcoming) Basis of upcoming BFT Datalog โat scaleโ work ๐ A C Single File History / โCausal Shadowโ
Secret Files ๐ฅท A Dark Forest of Data
Secret Files ๐ฅท Private Nodes Binary CBOR Encrypted Node ๐ AES256 Encrypted Node ๐ Virtual Node ๐ Index Metadata Encrypted Node ๐ ๐ ๐ ๐
Secret Files ๐ฅท Read Window Virtual Node Virtual Node ๐ Index ๐ ๐ Metadata ๐ ๐ ๐ Index Metadata
Secret Files ๐ฅท Read Window Virtual Node Virtual Node ๐ Index ๐ ๐ Metadata ๐ { ๐ ๐ name: โbeach.jpgโ, key: โB374A26A71490437Aโฆโ, โฆ } Index Metadata
Secret Files ๐ฅท Subgraph Access
Secret Files ๐ฅท Subgraph Access
Secret Files ๐ฅท Subgraph Access ๐
Secret Files ๐ฅท Subgraph Access ๐ ๐
Secret Files ๐ฅท Subgraph Access ๐ ๐ ๐
Secret Files ๐ฅท Subgraph Access ๐ ๐ ๐ ๐ฉโก๐
Secret Files ๐ฅท Progressive Fast Forward Rev 0
Secret Files ๐ฅท Progressive Fast Forward Rev 0 Rev 1 (Partial)
Secret Files ๐ฅท Progressive Fast Forward Rooting progress Rev 0 Rev 1 (Partial)
Secret Files ๐ฅท Progressive Fast Forward Rooting progress Rev 0 Rev 1 (Partial) Rev 2 (Partial)
Secret Files ๐ฅท Progressive Fast Forward Rooting progress Rev 3 (Complete) Rev 0 Rev 1 (Partial) Rev 2 (Partial)
Secret Files ๐ฅท Progressive Fast Forward Rooting progress Rev 3 (Complete) Rev 0 Rev 1 (Partial) Rev 2 (Partial)
Secret Files ๐ฅท Progressive Fast Forward Rev 3 (Complete) Rev 0 Rev 1 (Partial) Rev 2 (Partial)
Secret Files ๐ฅท Progressive Fast Forward Rev 3 (Complete) Rev 0 Rev 1 (Partial) Rev 2 (Partial)
Secret Files ๐ฅท Skip Ratchet
Secret Files ๐ฅท Skip Ratchet Ratchet keys for future (backwards) secrecy Skip ratchet KDF for log-time fast forwards
Secret Files ๐ฅท Skip Ratchet Ratchet keys for future (backwards) secrecy Skip ratchet KDF for log-time fast forwards
Secret Files ๐ฅท Skip Ratchet Ratchet keys for future (backwards) secrecy Skip ratchet KDF for log-time fast forwards Inter e s ted in Ask the fu Brook ll pap lyn f er? or a p repri nt
Secret Files ๐ฅท Write Access
Secret Files ๐ฅท Write Access Bare Filter simple = Bloom.insert(element: iNumber, into: parentFilter) versioned = Bloom.insert(element: hash(aesKey), into: simple) Saturation nextStep = Bloom.insert(element: xof_hash(n++), into: prevStep) Repeat until threshold bits flipped โOptimalโ threshold depends on Bloom configuration, but often ~49% of bits
Secret Files ๐ฅท Dark Forest
Secret Files ๐ฅท Dark Forest
Secret Files ๐ฅท Dark Forest
Secret Files ๐ฅท Dark Forest
Secret Files ๐ฅท Reasonably Efficient Secret KV
Secret Files ๐ฅท Reasonably Efficient Secret KV HAMT (weight 16)
Secret Files ๐ฅท Reasonably Efficient Secret KV HAMT 163 = 4,096 buckets (weight 16) 164 = 65,536 buckets
Secret Files ๐ฅท Reasonably Efficient Secret KV HAMT 163 = 4,096 buckets (weight 16) 164 = 65,536 buckets Append-only Quick Read/Write Merkleized Concurrency Friendly
{brooklyn,philipp}@fission.codes @expede & @matheus23 ๐ Thank You, IPFS รพing ๐ฎ๐ธ https://whitepaper.fission.codes โจ github.com/WebNativeFileSystem(/spec) โจ More in-depth at Strange Loop 2022 (or talk to us today)