TS T RYU CRT UAR E LNE S SNN E SYS THE H H O W M O R E M E A N I N G F U L C O D E C A N M A K E Y O U R P R O J E C T M O R E R E S I L I E N T & M A I N TA I N A B L E H OF
A presentation at Code BEAM Lite Amsterdam 2019 in November 2019 in Amsterdam, Netherlands by Brooklyn Zelenka
TS T RYU CRT UAR E LNE S SNN E SYS THE H H O W M O R E M E A N I N G F U L C O D E C A N M A K E Y O U R P R O J E C T M O R E R E S I L I E N T & M A I N TA I N A B L E H OF
THE TYRANNY OF STRUCTURELESSNESS For a number of years I have been familiar with the observation that the quality of programmers is a decreasing function of the density of GOTO statements in the programs they produce 🎤💧 EDSGER DIJKSTRA
THE TYRANNY OF STRUCTURELESSNESS For a number of years I have been familiar with the observation that the quality of programmers is a decreasing function of the density of GOTO statements in the programs they produce 🎤💧 EDSGER DIJKSTRA
THE TYRANNY OF STRUCTURELESSNESS What’s she on about? Elixir doesn’t have GOTOs… 🤔 THIS AUDIENCE
THE TYRANNY OF STRUCTURELESSNESS B R O O K LY N Z E L E N K A , @ e x p e d e
THE TYRANNY OF STRUCTURELESSNESS B R O O K LY N Z E L E N K A , @ e x p e d e • Cofounder/CTO at Fission • https://fission.codes • Make DevOps & Backend obsolete 😈 • Spending a lot of time with IPFS & DIDs • PLT & VM enthusiast • Ethereum Core Developer • Primary author of Witchcraft Suite & Exceptional
THE TYRANNY OF STRUCTURELESSNESS B R O O K LY N Z E L E N K A , @ e x p e d e • Cofounder/CTO at Fission • https://fission.codes • Make DevOps & Backend obsolete 😈 • Spending a lot of time with IPFS & DIDs • PLT & VM enthusiast • Ethereum Core Developer • Primary author of Witchcraft Suite & Exceptional
THE BIG IDEA
THE BIG IDEA 💭⚙🚀
THE BIG IDEA ONE-LINER
THE BIG IDEA ONE-LINER ✈ Work at a higher level 🛸
THE BIG IDEA LANGUAGE DESIGN REFLECTS INTENDED USE 🌐
THE BIG IDEA W H O ’S O RG LO O KS L I K E T H I S ?
,
,
,
,
THE BIG IDEA QUESTIONS
THE BIG IDEA QUESTIONS We want more type of features over time. As a result, complexity grows at an exponential rate.
THE BIG IDEA QUESTIONS We want more type of features over time. As a result, complexity grows at an exponential rate. How do you make Elixir code more flexible and easier to reason about at scale?
THE BIG IDEA QUESTIONS We want more type of features over time. As a result, complexity grows at an exponential rate. How do you make Elixir code more flexible and easier to reason about at scale? Do you think that the patterns we use today are the best possible patterns for software?
THE BIG IDEA QUESTIONS We want more type of features over time. As a result, complexity grows at an exponential rate. How do you make Elixir code more flexible and easier to reason about at scale? Do you think that the patterns we use today are the best possible patterns for software? How will you write code in 2020, 2025, and 2050?
THE BIG IDEA CORE We need to evolve our approach: focus on domain and structure!
THE BIG IDEA CORE We need to evolve our approach: focus on domain and structure! ✨ 🦄 🚀
IN THE LARGE
IN THE LARGE 🌟
IN THE LARGE C O D E Y O U U S E D TO W R I T E Imperative
IN THE LARGE C O D E Y O U U S E D TO W R I T E Imperative
IN THE LARGE “GOOD” ELIXIR Imperative
IN THE LARGE “GOOD” ELIXIR Imperative λ
IN THE LARGE Imperative 3LA FUTURE 🚀 λ
IN THE LARGE Imperative 3LA FUTURE 🚀 Semantic DSL / OO λ
IN THE LARGE PROP + MODEL TEST
GOTOS CONSIDERED HARMFUL
GOTOS CONSIDERED HARMFUL W H AT ’ S S O B A D A B O U T H AV I N G C O N T R O L? 🦶 🔫
GOTOS CONSIDERED HARMFUL W H AT ’ S S O B A D A B O U T H AV I N G C O N T R O L? 🦶 🔫 • GOTOs • Low level instruction • Literally how the machine is going to see it • Extremely flexible • Highly concrete • Huge number of implicit states
GOTOS CONSIDERED HARMFUL W H AT ’ S S O B A D A B O U T H AV I N G C O N T R O L? 🦶 🔫 • GOTOs • Low level instruction • Literally how the machine is going to see it • Extremely flexible • Highly concrete • Huge number of implicit states Line 1 Line 2 Line 3 Line 4 Line 5 — GOTO Line 6
GOTOS CONSIDERED HARMFUL W H AT ’ S S O B A D A B O U T H AV I N G C O N T R O L? 🦶 🔫 • GOTOs • Low level instruction • Literally how the machine is going to see it • Extremely flexible • Highly concrete • Huge number of implicit states Line 1 Line 2 Line 3 Line 4 Line 5 — GOTO Line 6
GOTOS CONSIDERED HARMFUL W H AT ’ S S O B A D A B O U T H AV I N G C O N T R O L? 🦶 🔫 • GOTOs • Low level instruction • Literally how the machine is going to see it • Extremely flexible • Highly concrete • Huge number of implicit states Line 1 Line 2 Line 3 Line 4 Line 5 — GOTO Line 6
GOTOS CONSIDERED HARMFUL W H AT ’ S S O B A D A B O U T H AV I N G C O N T R O L? 🦶 🔫 • GOTOs • Low level instruction • Literally how the machine is going to see it • Extremely flexible • Highly concrete • Huge number of implicit states Line 1 Line 2 Line 3 Line 4 Line 5 — GOTO Line 6 💥
GOTOS CONSIDERED HARMFUL STRUCTURED PROGRAMMING while
GOTOS CONSIDERED HARMFUL STRUCTURED PROGRAMMING • Subroutines • Loops • Switch/branching • Named routines while
GOTOS CONSIDERED HARMFUL T H E N E X T G E N E R AT I O N 🚀
GOTOS CONSIDERED HARMFUL T H E N E X T G E N E R AT I O N 🚀 • Functions • Map • Reduce • Filter • Constraint solvers
GOTOS CONSIDERED HARMFUL TRADEOFFS
GOTOS CONSIDERED HARMFUL TRADEOFFS • Exchange granular control for structure • Meaning over mechanics • More human than machine • Safer! • Spectrum • Turing Tarpit • Church Chasm • Haskell Fan Fiction
250 GOTOS CONSIDERED HARMFUL PAY O F F Structured Unstructured
250 GOTOS CONSIDERED HARMFUL COMPLEXITY PAY O F F Structured Unstructured TIME
GOTOS CONSIDERED HARMFUL PAY O F F 1000 COMPLEXITY 750 500 Unstructured 250 TIME Structured
GOTOS CONSIDERED HARMFUL PAY O F F 1000 COMPLEXITY 750 500 Unstructured 250 TIME Structured
GOTOS CONSIDERED HARMFUL PAY O F F 1000 COMPLEXITY 750 500 Unstructured 250 TIME Structured
ON COMPLEXITY
ON COMPLEXITY
COMPLEXITY
COMPLEXITY
COMPLEXITY THE BAD KIND ☠
COMPLEXITY THE BAD KIND ☠ • Probably pretty familiar with this • Euphemism for • Complicated • Inconsistent • No plan • “Unstructured mess”
COMPLEXITY GOOD COMPLEXITY = DEEP What do these have in common? (a+b)/a ~ a / b
COMPLEXITY ORTHOGONAL COMPLECTING
COMPLEXITY ORTHOGONAL COMPLECTING
COMPLEXITY ORTHOGONAL COMPLECTING
COMPLEXITY ORTHOGONAL COMPLECTING
COMPLEXITY ORTHOGONAL COMPLECTING
COMPLEXITY ORTHOGONAL COMPLECTING
COMPLEXITY ORTHOGONAL COMPLECTING
COMPLEXITY ORTHOGONAL COMPLECTING
COMPLEXITY ORTHOGONAL COMPLECTING
COMPLEXITY ORTHOGONAL COMPLECTING
COMPLEXITY ORTHOGONAL COMPLECTING 🕐
COMPLEXITY ORTHOGONAL COMPLECTING 🕐 Structures: 4
COMPLEXITY ORTHOGONAL COMPLECTING 🕐 Structures: 4 Results: effectively limitless
COMPLEXITY C O M P L E X ! = C O M P L I C AT E D
COMPLEXITY C O M P L E X ! = C O M P L I C AT E D • Complex: interconnected parts • Complicated: difficult to understand
COMPLEXITY ACTO R A BYS S 🕳
COMPLEXITY ACTO R A BYS S 🕳 1. Each step is very simple 2. Reasoning about dynamic organisms is hard 1. Remember to store your data for crash recovery 2. Called collaborator may not be there 3. Complexity grows faster than linear 4. Find common factors — your abstraction
COMPLEXITY ACTO R A BYS S 🕳 1. Each step is very simple 2. Reasoning about dynamic organisms is hard 1. Remember to store your data for crash recovery 2. Called collaborator may not be there 3. Complexity grows faster than linear 4. Find common factors — your abstraction
FIGHTING GenSoup
FIGHTING GenSoup 🚫🍲⚔
FIGHTING GenSoup G O O D I N T E R FA C E S ! = G O O D A B S T R A C T I O N S
FIGHTING GenSoup G O O D I N T E R FA C E S ! = G O O D A B S T R A C T I O N S • GenServer & co are actually pretty low level • Please add some semantics! • Don’t reinvent the wheel every time 🎡 • Let’s look at a very common example
FIGHTING GenSoup ABSTRACTION
FIGHTING GenSoup ABSTRACTION
FIGHTING GenSoup SIMPLE CASE
FIGHTING GenSoup A S Y N C C A S E — B A S E L AY E R
FIGHTING GenSoup A S Y N C C A S E — I M P L E M E N TAT I O N
FIGHTING GenSoup W H AT D I D W E G E T ?
FIGHTING GenSoup W H AT D I D W E G E T ? • Common interface • Encapsulate the detail • Don’t have to think about mechanics anymore
FIGHTING GenSoup ABSTRACTION = FOCUS/ESSENCE
FIGHTING GenSoup ABSTRACTION = FOCUS/ESSENCE
FIGHTING GenSoup ABSTRACTION = FOCUS/ESSENCE 🙈
ON ABSTRACTION & DSLS
ON ABSTRACTION & DSLS N O T G E T T I N G T R A P P E D I N T H E D E TA I L S
ABSTRACTION & DSLS COMMONALITIES
ABSTRACTION & DSLS COMMONALITIES
ABSTRACTION & DSLS COMMONALITIES • They clearly have a similar structure • NOT equally expressive • Enumerable • Always converted to List • Witchcraft.Functor
ABSTRACTION & DSLS COMMONALITIES
ABSTRACTION & DSLS COMMONALITIES
ABSTRACTION & DSLS COMMONALITIES
ABSTRACTION & DSLS COMMONALITIES • Different, but also have similar structure • Not very pipeable because 2 paths • …lots of duplicate code
ABSTRACTION & DSLS COMMONALITIES • Different, but also have similar structure • Not very pipeable because 2 paths • …lots of duplicate code • Why limit to only to two ways?
ABSTRACTION & DSLS S TA R T F R O M R U L E S
ABSTRACTION & DSLS S TA R T F R O M R U L E S • Describe what the overall solution looks like
ABSTRACTION & DSLS S TA R T F R O M R U L E S • Describe what the overall solution looks like • Choose how it gets run contextually
ABSTRACTION & DSLS TWO-PHASE
ABSTRACTION & DSLS TWO-PHASE • Always a two-phase process • Abstract, then concrete • Do concretion at application boundary
ABSTRACTION & DSLS TWO-PHASE • Always a two-phase process • Abstract, then concrete • Do concretion at application boundary
ABSTRACTION & DSLS TWO-PHASE • Always a two-phase process • Abstract, then concrete • Do concretion at application boundary
ABSTRACTION & DSLS IMPROVING Kernel
ABSTRACTION & DSLS IMPROVING Kernel • Fallback keys • Bang-functions
ABSTRACTION & DSLS I M P R O V I N G K e r n e l — FA L L B A C K K E Y S
ABSTRACTION & DSLS I M P R O V I N G K e r n e l — FA L L B A C K K E Y S • Composition is at the heart of modularity
ABSTRACTION & DSLS I M P R O V I N G K e r n e l — FA L L B A C K K E Y S • Composition is at the heart of modularity • Orthogonality is at the heart of composition
ABSTRACTION & DSLS I M P R O V I N G K e r n e l — FA L L B A C K K E Y S • Composition is at the heart of modularity • Orthogonality is at the heart of composition
ABSTRACTION & DSLS I M P R O V I N G K e r n e l — FA L L B A C K K E Y S • Composition is at the heart of modularity • Orthogonality is at the heart of composition
ABSTRACTION & DSLS I M P R O V I N G K e r n e l — FA L L B A C K K E Y S • Composition is at the heart of modularity • Orthogonality is at the heart of composition • Let’s abstract default values!
ABSTRACTION & DSLS I M P R O V I N G K e r n e l — FA L L B A C K K E Y S • Composition is at the heart of modularity • Orthogonality is at the heart of composition • Let’s abstract default values!
ABSTRACTION & DSLS I M P R O V I N G K e r n e l — FA L L B A C K K E Y S • Composition is at the heart of modularity • Orthogonality is at the heart of composition • Let’s abstract default values!
ABSTRACTION & DSLS I M P R O V I N G K e r n e l — FA L L B A C K K E Y S • Composition is at the heart of modularity • Orthogonality is at the heart of composition • Let’s abstract default values!
ABSTRACTION & DSLS I M P R O V I N G K e r n e l — FA L L B A C K K E Y S • Composition is at the heart of modularity • Orthogonality is at the heart of composition • Let’s abstract default values! 🔁
ABSTRACTION & DSLS I M P R O V I N G K e r n e l — FA L L B A C K K E Y S • Composition is at the heart of modularity • Orthogonality is at the heart of composition • Let’s abstract default values! • More focused (does one thing) 🔁
ABSTRACTION & DSLS I M P R O V I N G K e r n e l — FA L L B A C K K E Y S • Composition is at the heart of modularity • Orthogonality is at the heart of composition • Let’s abstract default values! • More focused (does one thing) • More general (works everywhere) 🔁
ABSTRACTION & DSLS I M P R O V I N G K e r n e l — FA L L B A C K K E Y S • Composition is at the heart of modularity • Orthogonality is at the heart of composition • Let’s abstract default values! • More focused (does one thing) • More general (works everywhere) • Ad hoc function extension 🔁
ABSTRACTION & DSLS IMPROVING Kernel — BANG FUNCTIONS
ABSTRACTION & DSLS IMPROVING Kernel — BANG FUNCTIONS
ABSTRACTION & DSLS IMPROVING Kernel — BANG FUNCTIONS
ABSTRACTION & DSLS IMPROVING Kernel — BANG FUNCTIONS 💣
ABSTRACTION & DSLS IMPROVING Kernel — BANG FUNCTIONS 💣
ABSTRACTION & DSLS IMPROVING Kernel — BANG FUNCTIONS 💣 Abstracted out 💣 foo!/* from foo/*
ABSTRACTION & DSLS IMPROVING Kernel — BANG FUNCTIONS 💣 Abstracted out 💣 foo!/* from foo/*
ABSTRACTION & DSLS IMPROVING Kernel — BANG FUNCTIONS 💣 Abstracted out 💣 foo!/* from foo/* 🆗 ⏩
ABSTRACTION & DSLS IMPROVING Kernel — BANG FUNCTIONS 💣 Abstracted out 💣 foo!/* from foo/* 🆗 ⏩ 💣
ABSTRACTION & DSLS IMPROVING Kernel — BANG FUNCTIONS 💣 Works everywhere Any data Any error struct Abstracted out 💣 foo!/* from foo/* 🆗 ⏩ 💣 Any flow (esp. pipes) Super easy to test
ABSTRACTION & DSLS IMPROVING Kernel — BANG FUNCTIONS 💣 Works everywhere Any data Any error struct Abstracted out 💣 foo!/* from foo/* 🆗 ⏩ 💣 Any flow (esp. pipes) Super easy to test BONUS Disambiguate between nil value and actual errors
ABSTRACTION & DSLS N O T E : M E TA P H O R
ABSTRACTION & DSLS N O T E : M E TA P H O R Because it’s easier now
ABSTRACTION & DSLS N O T E : M E TA P H O R • Concept: Flow-ability is very core to Elixir’s ethos • Kernel.|>/2 Because it’s easier now
ABSTRACTION & DSLS N O T E : M E TA P H O R • Concept: Flow-ability is very core to Elixir’s ethos • Kernel.|>/2 • Consistent flow metaphor / punning on existing metaphor • Exceptional: ~>/2 and >>>/2 Because it’s easier now
ABSTRACTION & DSLS N O T E : M E TA P H O R • Concept: Flow-ability is very core to Elixir’s ethos • Kernel.|>/2 • Consistent flow metaphor / punning on existing metaphor • Exceptional: ~>/2 and >>>/2 Because it’s easier now
ABSTRACTION & DSLS N O T E : M E TA P H O R • Concept: Flow-ability is very core to Elixir’s ethos • Kernel.|>/2 • Consistent flow metaphor / punning on existing metaphor • Exceptional: ~>/2 and >>>/2 Because it’s easier now
ABSTRACTION & DSLS N O T E : M E TA P H O R • Concept: Flow-ability is very core to Elixir’s ethos • Kernel.|>/2 • Consistent flow metaphor / punning on existing metaphor • Exceptional: ~>/2 and >>>/2 Because it’s easier now
ABSTRACTION & DSLS N O T E : M E TA P H O R • Concept: Flow-ability is very core to Elixir’s ethos • Kernel.|>/2 • Consistent flow metaphor / punning on existing metaphor • Exceptional: ~>/2 and >>>/2 Because it’s easier now
ABSTRACTION & DSLS W H AT ’ S G A I N E D ?
ABSTRACTION & DSLS W H AT ’ S G A I N E D ? • Clear • Composable • Greater reuse ♻ • User choice • Increased testability • Simple example: is_exception?/1 • Could still add protocol to get even more power
ABSTRACTION STO RY T E L L I N G 📚
ABSTRACTION STO RY T E L L I N G 📚
ABSTRACTION STO RY T E L L I N G 📚
ABSTRACTION & DSLS H O W T O E AT T H E E L E P H A N T
ABSTRACTION & DSLS H O W T O E AT T H E E L E P H A N T
ABSTRACTION & DSLS H O W T O E AT T H E E L E P H A N T • By feature?
ABSTRACTION & DSLS H O W T O E AT T H E E L E P H A N T • By feature? • By behaviour?
ABSTRACTION & DSLS H O W T O E AT T H E E L E P H A N T • By feature? • By behaviour? • By structure / properties!
ON STRUCTURE
ON STRUCTURE 🔺▫🔵
STRUCTURE T H E O N LY T H R E E R I G H T A N S W E R S
STRUCTURE T H E O N LY T H R E E R I G H T A N S W E R S 1
STRUCTURE T H E O N LY T H R E E R I G H T A N S W E R S 1 2
STRUCTURE T H E O N LY T H R E E R I G H T A N S W E R S 1 2 3
STRUCTURE T H E O N LY T H R E E R I G H T A N S W E R S 1 2 3
STRUCTURE A S S O C I AT I V I T Y
STRUCTURE A S S O C I AT I V I T Y • Not a data structure
STRUCTURE A S S O C I AT I V I T Y • Not a data structure • Not a function
STRUCTURE A S S O C I AT I V I T Y • Not a data structure • Not a function • An interface & rules!
STRUCTURE A S S O C I AT I V I T Y • Not a data structure • Not a function • An interface & rules!
STRUCTURE A S S O C I AT I V I T Y • Not a data structure • Not a function • An interface & rules! p a t e m w o l f e h t e t o (N ) r ho
STRUCTURE A SEMIGROUP ON…
STRUCTURE A SEMIGROUP ON…
STRUCTURE UNLAWFUL COUNTEREXAMPLE 🚨
STRUCTURE H OW TO C O M P O S E P RO P E RT I E S
STRUCTURE H OW TO C O M P O S E P RO P E RT I E S • A structure of structures • Keep it in your brain • Enforce with TypeClass
LE T ’S DO SOME THING WILD
LE T ’S DO SOME THING WILD ⚡🔥 POWER UP 🌪🌊
POWER UP EXPLICIT ASSUMPTIONS
POWER UP EXPLICIT ASSUMPTIONS • Parallel pipes
POWER UP EXPLICIT ASSUMPTIONS • Parallel pipes • Concurrency = partial order
POWER UP EXPLICIT ASSUMPTIONS • Parallel pipes • Concurrency = partial order t
POWER UP EXPLICIT ASSUMPTIONS • Parallel pipes • Concurrency = partial order t
POWER UP EXPLICIT ASSUMPTIONS • Parallel pipes • Concurrency = partial order t
POWER UP EXPLICIT ASSUMPTIONS • Parallel pipes • Concurrency = partial order t
POWER UP EXPLICIT ASSUMPTIONS • Parallel pipes • Concurrency = partial order t
POWER UP EXPLICIT ASSUMPTIONS • Parallel pipes • Concurrency = partial order • Monotonic • All loops must be linearized t
POWER UP EXPLICIT ASSUMPTIONS • Parallel pipes • Concurrency = partial order • Monotonic • All loops must be linearized • Properties • Serial composition • Parallel composition • Explicit evaluation strategy t
POWER UP PIPES++
POWER UP PIPES++
POWER UP PIPES++
POWER UP PIPES++
POWER UP CLEANUP
POWER UP CLEANUP
POWER UP C A R R I E R D ATA
POWER UP C A R R I E R D ATA
POWER UP C A R R I E R D ATA
POWER UP C A R R I E R D ATA
POWER UP C A R R I E R D ATA
POWER UP P ROTO C O L
POWER UP SIMPLE CASE
POWER UP SIMPLE CASE
POWER UP SIMPLE CASE
POWER UP ASYNC
POWER UP ASYNC
POWER UP ASYNC
POWER UP ASYNC
POWER UP U P S H OT
POWER UP U P S H OT • Higher semantic density (focused on meaning not mechanics)
POWER UP U P S H OT • Higher semantic density (focused on meaning not mechanics) • Declarative, configurable data flow 🤯
POWER UP U P S H OT • Higher semantic density (focused on meaning not mechanics) • Declarative, configurable data flow 🤯 • Extremely extensible • defimpl Dataflow, for: %Stream{} • defimpl Dataflow, for: %Distributed{} • defimpl Dataflow, for: %Broadway{}
POWER UP U P S H OT • Higher semantic density (focused on meaning not mechanics) • Declarative, configurable data flow 🤯 • Extremely extensible • defimpl Dataflow, for: %Stream{} • defimpl Dataflow, for: %Distributed{} • defimpl Dataflow, for: %Broadway{} • Model-testable
POWER UP U P S H OT • Higher semantic density (focused on meaning not mechanics) • Declarative, configurable data flow 🤯 • Extremely extensible • defimpl Dataflow, for: %Stream{} • defimpl Dataflow, for: %Distributed{} • defimpl Dataflow, for: %Broadway{} • Model-testable • Composable with other pipes and change evaluation strategies
A CALL FOR LIBRARIES
A CALL FOR LIBRARIES 📣
A CALL FOR LIBRARIES SUMMARY
A CALL FOR LIBRARIES SUMMARY • Can plug into / extend
A CALL FOR LIBRARIES SUMMARY • Can plug into / extend • Single-threaded context
A CALL FOR LIBRARIES SUMMARY • Can plug into / extend • Single-threaded context • Distributed context
A CALL FOR LIBRARIES SUMMARY • Can plug into / extend • Single-threaded context • Distributed context • Dynamic hybrid contexts
A CALL FOR LIBRARIES EXTEND RAILROAD PROGRAMMING
A CALL FOR LIBRARIES EXTEND RAILROAD PROGRAMMING Happy Path (Continue) Error Case (Skip) No Effect (Afterwards)
A CALL FOR LIBRARIES EXTEND RAILROAD PROGRAMMING Happy Path (Continue) Error Case (Skip) No Effect (Afterwards)
A CALL FOR LIBRARIES EXTEND RAILROAD PROGRAMMING Happy Path (Continue) Error Case (Skip) No Effect (Afterwards)
A CALL FOR LIBRARIES S U R P R I S I N G N U M B E R O F FA C T O R S
A CALL FOR LIBRARIES S U R P R I S I N G N U M B E R O F FA C T O R S Log Program
A CALL FOR LIBRARIES S U R P R I S I N G N U M B E R O F FA C T O R S Log Program
SUMMARY
SUMMARY KEEP IN MIND
SUMMARY KEEP IN MIND 1. Protocols-for-DDD (P4D3) 2. Add a semantic layer 3. How do you locally test your distributed system? Look at the properties! 4. Under which conditions does your code work? What are your assumptions? 5. Prop testing is useful for structured abstractions 6. You should be able to code half-asleep
https://fission.codes https://talk .fission.codes https://tools.fission.codes L B E D A N K T, A M ST E R D A M 🎉 brooklyn@fission.codes g i t h u b . c o m /e x p e d e @expede
REWRITING A PHOENIX APP
REWRITING A PHOENIX APP 🕚 A N T I - PAT T E R N S & B U Y I N G T I M E 🔥
REWRITING A PHOENIX APP THE PROBLEM
REWRITING A PHOENIX APP THE PROBLEM • Inherited project • First go at Phoenix • “We don’t believe in tests”
REWRITING A PHOENIX APP THE PROBLEM • Inherited project • First go at Phoenix • “We don’t believe in tests” • Very brittle, many bugs 😱
REWRITING A PHOENIX APP THE PROBLEM • Inherited project • First go at Phoenix • “We don’t believe in tests” • Very brittle, many bugs 😱 • Now tight deadline — no time to do a total rewrite • …or was there? 😏
REWRITING A PHOENIX APP S T R AT E G Y : G E N E R A L I Z AT I O N !
REWRITING A PHOENIX APP S T R AT E G Y : G E N E R A L I Z AT I O N ! • Deescalate • Running code • Get to “net new” features • Two things that people don’t generally associate with abstraction!
REWRITING A PHOENIX APP ⚠ A WORD OF WARNING ⚠
REWRITING A PHOENIX APP ⚠ A WORD OF WARNING ⚠
REWRITING A PHOENIX APP ⚠ A WORD OF WARNING ⚠ How you see yourself
REWRITING A PHOENIX APP ⚠ A WORD OF WARNING ⚠ How you see yourself How others see you
REWRITING A PHOENIX APP THE (TEMPORARY) SOLUTION
REWRITING A PHOENIX APP THE (TEMPORARY) SOLUTION
REWRITING A PHOENIX APP THE (TEMPORARY) SOLUTION
REWRITING A PHOENIX APP THE (TEMPORARY) SOLUTION
REWRITING A PHOENIX APP THE (TEMPORARY) SOLUTION •Generate all of MVVC •Compile-time functions •Incl. validation •Spend 90% time on macro •Write •Test!
REWRITING A PHOENIX APP TRADEOFFS
REWRITING A PHOENIX APP TRADEOFFS • Implement very fast • Flexible • Makes a lot of assumptions • Macro magic • Difficult to read, extend, &c • Low-level implementation • The macro swamp
REWRITING A PHOENIX APP TRADEOFFS
REWRITING A PHOENIX APP TRADEOFFS • Straightforward: MVVC! • Encode patterns directly in code 🎉
C R DT CAS E ST U DY
C R DT CAS E ST U DY ⏲ ELEGANT AND USEFUL ☮
C R DT CAS E ST U DY SCENARIO
C R DT CAS E ST U DY SCENARIO • Distributed / uncontrolled topology
C R DT CAS E ST U DY SCENARIO • Distributed / uncontrolled topology 💻 💻 💻 💻 💻 💻
C R DT CAS E ST U DY SCENARIO • Distributed / uncontrolled topology 💻 • Eventual consistency 💻 💻 💻 💻 💻
C R DT CAS E ST U DY SCENARIO • Distributed / uncontrolled topology 💻 • Eventual consistency 💻 💻 🚦 💻 💻 💻
C R DT CAS E ST U DY SCENARIO • Distributed / uncontrolled topology 💻 • Eventual consistency 💻 💻 🚦 💻 💻 💻
C R DT CAS E ST U DY SCENARIO • Distributed / uncontrolled topology 💻 • Eventual consistency 💻 💻 🚦 💻 💻 💻
C R DT CAS E ST U DY SCENARIO • Distributed / uncontrolled topology 💻 • Eventual consistency 💻 • High resilience • Infinite(ish) time tolerance • Decentralized (e.g. via gossip) 💻 🚦 💻 • Self-healing 💻 💻
C R DT CAS E ST U DY SCENARIO • Distributed / uncontrolled topology 💻 • Eventual consistency 💻 • High resilience • Infinite(ish) time tolerance • Decentralized (e.g. via gossip) 💻 💻 • Self-healing 💻 💻
C R DT CAS E ST U DY SCENARIO • Distributed / uncontrolled topology 💻 • Eventual consistency 💻 • High resilience • Infinite(ish) time tolerance • Decentralized (e.g. via gossip) 💻 💻 • Self-healing 💻 💻
C R DT CAS E ST U DY SCENARIO • Distributed / uncontrolled topology 💻 • Eventual consistency 💻 • High resilience • Infinite(ish) time tolerance • Decentralized (e.g. via gossip) 💻 💻 • Self-healing • Two variants • State-based • Operation-based ✅ 💻 💻
C R DT CAS E ST U DY B A S I C : P O S I T I V E / N E G AT I V E C O U N T E R
C R DT CAS E ST U DY B A S I C : P O S I T I V E / N E G AT I V E C O U N T E R • Two sets • Increments • Decrements • Operations • Internal • Increment • Decrement • Compare • External • Merge • Query
C R DT CAS E ST U DY B A S I C : P O S I T I V E / N E G AT I V E C O U N T E R • Two sets • Increments • Decrements • Operations • Internal • Increment • Decrement • Compare • External • Merge • Query • Challenges • Negative counts • Double deletes • Solutions • Only delete existing items • Commutativity
C R DT CAS E ST U DY OBSERVE/REMOVE SE T
C R DT CAS E ST U DY OBSERVE/REMOVE SE T • Two sub-sets • Add • Remove • Operations • Internal • Add • Remove • Compare • Tag • External • Merge — semigroup • Query
C R DT CAS E ST U DY OBSERVE/REMOVE SE T • Two sub-sets • Add • Remove • Operations • Internal • Add • Remove • Compare • Tag • External • Merge — semigroup • Query • Challenges • Add-after-remove • Unique tags vs re-add • Can only delete what you know about • But deleting an unknown element is the same as a noop, so great! • Remove has priority over add • Wall clock timestamps are unreliable • Need monotonically increasing structure • Solutions • Only delete existing items • Commutativity • Protocols • Collectable • Enumerable • Semigroup (merge) -> Commutative (CRDT) • “Meet semilattice”
W H AT E V E N I S E L I X I R ?
W H AT E V E N I S E L I X I R ? ⚗
W H AT E V E N I S E L I X I R ? SHOW OF HANDS ✋ Is Elixir a functional language?
W H AT E V E N I S E L I X I R ? STRUCTURAL INTEGRIT Y
W H AT E V E N I S E L I X I R ? STRUCTURAL INTEGRIT Y ☝ Production Elixir can actually be pretty unstructured
W H AT E V E N I S E L I X I R ? STRUCTURAL INTEGRIT Y ☝ Production Elixir can actually be pretty unstructured • Imperative • Lots of ambient state • Spread across processes • Ordering & timing bugs
W H AT E V E N I S E L I X I R ? STRUCTURAL INTEGRIT Y ☝ Production Elixir can actually be pretty unstructured • Imperative • Lots of ambient state • Spread across processes • Ordering & timing bugs • Functional • Well-defined structured functions (map, filter, reduce) • Immutable references
W H AT E V E N I S E L I X I R ? W H Y N O T F U L LY S T R U C T U R E D ?
W H AT E V E N I S E L I X I R ? W H Y N O T F U L LY S T R U C T U R E D ? • Naive structure can be rigid! • Composition • Orthogonality
W H AT E V E N I S E L I X I R ? W H Y N O T F U L LY S T R U C T U R E D ? • Naive structure can be rigid! • Composition • Orthogonality Credit: psihedelisto
W H AT E V E N I S E L I X I R ? H O W T O E AT T H E S T R U C T U R A L E L E P H A N T
POWER UP
POWER UP Data dominates. If you’ve chosen the right data structures and organized things well, the algorithms will almost always be selfevident. Data structures, not algorithms, are central to programming. ⚙ ~ROB PIKE, 5 RULES OF PROGRAMMING
ABSTRACTION & DSLS DEFINITIONS
ABSTRACTION & DSLS DEFINITIONS Abstract 📈 Concrete 🚚
ABSTRACTION & DSLS DEFINITIONS Abstract 📈 \ Specific General ]👪_ Concrete 🚚
ABSTRACTION & DSLS DEFINITIONS Abstract 📈 42 \ Specific “Community” General ]👪_ Conference speakers L Concrete 🚚
ABSTRACTION & DSLS DEFINITIONS Abstract 📈 42 \ Specific “Community” General ]👪_ Conference speakers L Concrete 🚚
ABSTRACTION & DSLS DEFINITIONS Abstract 📈 42 \ Specific “Community” General ]👪_ Conference speakers L Concrete 🚚
