A presentation at Code BEAM Virtual by Brooklyn Zelenka
T YRANNY THE OF STRUCTURELESSNESS N O I T I D E L UA H H O W M O R E M E A N I N G F U L C O D E C A N M A K E Y O U R P R O J E C T M O R E R E S I L I E N T & M A I N TA I N A B L E H T R I
THE T YRANNY OF STRUCTURELESSNESS For a number of years I have been familiar with the observation that the quality of programmers is a decreasing function of the density of GOTO statements in the programs they produce π€π§ EDSGER DIJKSTRA
THE T YRANNY OF STRUCTURELESSNESS For a number of years I have been familiar with the observation that the quality of programmers is a decreasing function of the density of GOTO statements in the programs they produce π€π§ EDSGER DIJKSTRA
THE T YRANNY OF STRUCTURELESSNESS Whatβs she on about? Elixir doesnβt have GOTOsβ¦ π€ THIS AUDIENCE
THE T YRANNY OF STRUCTURELESSNESS B R O O K LY N Z E L E N K A , @ E X P E D E
THE T YRANNY OF STRUCTURELESSNESS B R O O K LY N Z E L E N K A , @ E X P E D E β’ Cofounder/CTO at Fission β’ https://fission.codes β’ Make DevOps & Backend obsolete π β’ Spending a lot of time with IPFS, DIDs, CRDTs β’ Want to hear more? Berlin FP online meetup June 2 β’ PLT & VM enthusiast β’ Prev. Ethereum Core Developer β’ Primary author of Witchcraft Suite & Exceptional β’ This is a version of CodeBEAM Amsterdam 2019 keynote β’ Whova app for Q&A afterwards
THE BIG IDEA
THE BIG IDEA πβπ
THE BIG IDEA ONE-LINER
THE BIG IDEA ONE-LINER β Work at a higher level πΈ
THE BIG IDEA LANGUAGE DESIGN REFLECTS INTENDED USE π
THE BIG IDEA W H O βS O RG LO O KS L I K E T H I S ?
THE BIG IDEA W H O βS O RG LO O KS L I K E T H I S ? π§ π¨π» π©π» π©π» π¨π» β π©π» π¨π»
THE BIG IDEA W H O βS O RG LO O KS L I K E T H I S ? π§ π¨π» π©π» π©π» π¨π» β π©π» π¨π»
THE BIG IDEA HOW ABOUT THIS? π©π» π©π» π¨π» β π¨π» π©π» π¨π»
THE BIG IDEA HOW ABOUT THIS? π©π» π©π» π¨π» β π¨π» π©π» π¨π»
THE BIG IDEA QUESTIONS
THE BIG IDEA QUESTIONS We want more type of features over time. As a result, complexity grows at an exponential rate.
THE BIG IDEA QUESTIONS We want more type of features over time. As a result, complexity grows at an exponential rate. How do you make Elixir code more flexible and easier to reason about at scale?
THE BIG IDEA QUESTIONS We want more type of features over time. As a result, complexity grows at an exponential rate. How do you make Elixir code more flexible and easier to reason about at scale? Do you think that the patterns we use today are the best possible patterns for software?
THE BIG IDEA QUESTIONS We want more type of features over time. As a result, complexity grows at an exponential rate. How do you make Elixir code more flexible and easier to reason about at scale? Do you think that the patterns we use today are the best possible patterns for software? How will you write code in 2025, 2030, and 2050?
THE BIG IDEA CORE We need to evolve our approach: focus on domain and structure!
THE BIG IDEA CORE We need to evolve our approach: focus on domain and structure! β¨ π¦ π
IN THE LARGE
IN THE LARGE π
IN THE LARGE C O D E YO U U S E D TO W R I T E Imperative
IN THE LARGE C O D E YO U U S E D TO W R I T E Imperative
IN THE LARGE βGOODβ ELIXIR Imperative
IN THE LARGE βGOODβ ELIXIR Imperative Ξ»
IN THE LARGE Imperative 3 - L AY E R π Ξ»
IN THE LARGE Imperative 3 - L AY E R π Semantic DSL / OO Ξ»
IN THE LARGE PROP + MODEL TEST
GOTO CONSIDERED HARMFUL
G OTO S C O N S I D E R E D H A R M F U L W H AT β S S O B A D A B O U T H AV I N G C O N T R O L? π¦Ά π«
G OTO S C O N S I D E R E D H A R M F U L W H AT β S S O B A D A B O U T H AV I N G C O N T R O L? π¦Ά π« β’ GOTOs β’ Low level instruction β’ Literally how the machine is going to see it β’ Extremely flexible β’ Highly concrete β’ Huge number of implicit states
G OTO S C O N S I D E R E D H A R M F U L W H AT β S S O B A D A B O U T H AV I N G C O N T R O L? π¦Ά π« β’ GOTOs β’ Low level instruction Line 1 Line 2 β’ Literally how the machine is going to see it Line 3 β’ Extremely flexible Line 4 β’ Highly concrete β’ Huge number of implicit states L i n e 5 β G OTO Line 6
G OTO S C O N S I D E R E D H A R M F U L W H AT β S S O B A D A B O U T H AV I N G C O N T R O L? π¦Ά π« β’ GOTOs β’ Low level instruction Line 1 Line 2 β’ Literally how the machine is going to see it Line 3 β’ Extremely flexible Line 4 β’ Highly concrete β’ Huge number of implicit states L i n e 5 β G OTO Line 6
G OTO S C O N S I D E R E D H A R M F U L W H AT β S S O B A D A B O U T H AV I N G C O N T R O L? π¦Ά π« β’ GOTOs β’ Low level instruction Line 1 Line 2 β’ Literally how the machine is going to see it Line 3 β’ Extremely flexible Line 4 β’ Highly concrete β’ Huge number of implicit states L i n e 5 β G OTO Line 6
G OTO S C O N S I D E R E D H A R M F U L W H AT β S S O B A D A B O U T H AV I N G C O N T R O L? π¦Ά π« β’ GOTOs β’ Low level instruction Line 1 Line 2 β’ Literally how the machine is going to see it Line 3 β’ Extremely flexible Line 4 β’ Highly concrete β’ Huge number of implicit states L i n e 5 β G OTO Line 6 π₯
G OTO S C O N S I D E R E D H A R M F U L STRUCTURED PROGRAMMING while
G OTO S C O N S I D E R E D H A R M F U L STRUCTURED PROGRAMMING β’ Subroutines β’ Loops β’ Switch/branching β’ Named routines while
G OTO S C O N S I D E R E D H A R M F U L T H E N E X T G E N E R AT I O N π
G OTO S C O N S I D E R E D H A R M F U L T H E N E X T G E N E R AT I O N π β’ Functions β’ Map β’ Reduce β’ Filter β’ Constraint solvers
G OTO S C O N S I D E R E D H A R M F U L TRADEOFFS
G OTO S C O N S I D E R E D H A R M F U L TRADEOFFS β’ Exchange granular control for structure β’ Meaning over mechanics β’ More human than machine β’ Safer!
G OTO S C O N S I D E R E D H A R M F U L TRADEOFFS β’ Exchange granular control for structure β’ Spectrum β’ Meaning over mechanics β’ Turing Tarpit β’ More human than machine β’ Church Chasm β’ Safer! β’ Haskell Fan Fiction
COMPLEXITY ACTO R A BY S S π³
COMPLEXITY ACTO R A BY S S π³ 1. Each step is very simple 2. Reasoning about dynamic organisms is hard 1. Remember to store your data for crash recovery 2. Called collaborator may not be there 3. Complexity grows faster than linear 4. Find common factors β your abstraction
COMPLEXITY ACTO R A BY S S π³ 1. Each step is very simple 2. Reasoning about dynamic organisms is hard 1. Remember to store your data for crash recovery 2. Called collaborator may not be there 3. Complexity grows faster than linear 4. Find common factors β your abstraction
250 G OTO S C O N S I D E R E D H A R M F U L PAY O F F Structured Unstructured
250 G OTO S C O N S I D E R E D H A R M F U L COMPLEXITY PAY O F F Structured Unstructured TIME
G OTO S C O N S I D E R E D H A R M F U L PAY O F F 1000 COMPLEXITY 750 500 Unstructured Structured 250 TIME
G OTO S C O N S I D E R E D H A R M F U L PAY O F F 1000 COMPLEXITY 750 500 Unstructured Structured 250 TIME
G OTO S C O N S I D E R E D H A R M F U L PAY O F F 1000 COMPLEXITY 750 500 Unstructured Structured 250 TIME
COMPLEXITY ORTHOGONAL COMPLECTING
COMPLEXITY ORTHOGONAL COMPLECTING
COMPLEXITY ORTHOGONAL COMPLECTING
COMPLEXITY ORTHOGONAL COMPLECTING
COMPLEXITY ORTHOGONAL COMPLECTING
COMPLEXITY ORTHOGONAL COMPLECTING
COMPLEXITY ORTHOGONAL COMPLECTING
COMPLEXITY ORTHOGONAL COMPLECTING
COMPLEXITY ORTHOGONAL COMPLECTING
COMPLEXITY ORTHOGONAL COMPLECTING
COMPLEXITY ORTHOGONAL COMPLECTING π
COMPLEXITY ORTHOGONAL COMPLECTING π Structures: 4
COMPLEXITY ORTHOGONAL COMPLECTING π Structures: 4 Results: effectively limitless
ON ABSTRACTION & DSLS
ON ABSTRACTION & DSLS N O T G E T T I N G T R A P P E D I N T H E D E TA I L S
ABSTRACTION & DSLS COMMONALITIES
ABSTRACTION & DSLS COMMONALITIES
ABSTRACTION & DSLS COMMONALITIES β’ They clearly have a similar structure β’ NOT equally expressive β’ Enumerable β’ Always converted to List β’ Witchcraft.Functor
ABSTRACTION & DSLS COMMONALITIES
ABSTRACTION & DSLS COMMONALITIES
ABSTRACTION & DSLS COMMONALITIES
ABSTRACTION & DSLS COMMONALITIES β’ Different, but also have similar structure β’ Not very pipeable because 2 paths β’ β¦lots of duplicate code
ABSTRACTION & DSLS COMMONALITIES β’ Different, but also have similar structure β’ Not very pipeable because 2 paths β’ β¦lots of duplicate code β’ Why limit to only to two ways?
ABSTRACTION & DSLS S TA R T F R O M R U L E S
ABSTRACTION & DSLS S TA R T F R O M R U L E S β’ Describe what the overall solution looks like β βfront endβ interface
ABSTRACTION & DSLS S TA R T F R O M R U L E S β’ Describe what the overall solution looks like β βfront endβ interface β’ Choose how it gets run contextually β βback endβ runner
ABSTRACTION & DSLS TWO-PHASE
ABSTRACTION & DSLS TWO-PHASE β’ Always a two-phase process β’ Abstract, then concrete β’ Do concretion at application boundary
ABSTRACTION & DSLS TWO-PHASE β’ Always a two-phase process β’ Abstract, then concrete β’ Do concretion at application boundary
ABSTRACTION & DSLS TWO-PHASE d n e t n Fro β’ Always a two-phase process β’ Abstract, then concrete β’ Do concretion at application boundary B d n e k c a
ABSTRACTION & DSLS IMPROVING Kernel
ABSTRACTION & DSLS IMPROVING Kernel β’ Fallback keys β’ Bang-functions
ABSTRACTION & DSLS I M P R O V I N G K e r n e l β FA L L B A C K K E Y S
ABSTRACTION & DSLS I M P R O V I N G K e r n e l β FA L L B A C K K E Y S β’ Composition is at the heart of modularity
ABSTRACTION & DSLS I M P R O V I N G K e r n e l β FA L L B A C K K E Y S β’ Composition is at the heart of modularity β’ Orthogonality is at the heart of composition
ABSTRACTION & DSLS I M P R O V I N G K e r n e l β FA L L B A C K K E Y S β’ Composition is at the heart of modularity β’ Orthogonality is at the heart of composition
ABSTRACTION & DSLS I M P R O V I N G K e r n e l β FA L L B A C K K E Y S β’ Composition is at the heart of modularity β’ Orthogonality is at the heart of composition
ABSTRACTION & DSLS I M P R O V I N G K e r n e l β FA L L B A C K K E Y S β’ Composition is at the heart of modularity β’ Orthogonality is at the heart of composition β’ Letβs abstract default values!
ABSTRACTION & DSLS I M P R O V I N G K e r n e l β FA L L B A C K K E Y S β’ Composition is at the heart of modularity β’ Orthogonality is at the heart of composition β’ Letβs abstract default values!
ABSTRACTION & DSLS I M P R O V I N G K e r n e l β FA L L B A C K K E Y S β’ Composition is at the heart of modularity β’ Orthogonality is at the heart of composition β’ Letβs abstract default values!
ABSTRACTION & DSLS I M P R O V I N G K e r n e l β FA L L B A C K K E Y S β’ Composition is at the heart of modularity β’ Orthogonality is at the heart of composition β’ Letβs abstract default values!
ABSTRACTION & DSLS I M P R O V I N G K e r n e l β FA L L B A C K K E Y S β’ Composition is at the heart of modularity β’ Orthogonality is at the heart of composition β’ Letβs abstract default values! π
ABSTRACTION & DSLS I M P R O V I N G K e r n e l β FA L L B A C K K E Y S β’ Composition is at the heart of modularity β’ Orthogonality is at the heart of composition β’ Letβs abstract default values! β’ More focused (does one thing) π
ABSTRACTION & DSLS I M P R O V I N G K e r n e l β FA L L B A C K K E Y S β’ Composition is at the heart of modularity β’ Orthogonality is at the heart of composition β’ Letβs abstract default values! β’ More focused (does one thing) β’ More general (works everywhere) π
ABSTRACTION & DSLS I M P R O V I N G K e r n e l β FA L L B A C K K E Y S β’ Composition is at the heart of modularity β’ Orthogonality is at the heart of composition β’ Letβs abstract default values! β’ More focused (does one thing) β’ More general (works everywhere) β’ Ad hoc function extension π
ABSTRACTION & DSLS IMPROVING Kernel β BANG FUNCTIONS
ABSTRACTION & DSLS IMPROVING Kernel β BANG FUNCTIONS
ABSTRACTION & DSLS IMPROVING Kernel β BANG FUNCTIONS π£
ABSTRACTION & DSLS IMPROVING Kernel β BANG FUNCTIONS π£
ABSTRACTION & DSLS IMPROVING Kernel β BANG FUNCTIONS π£ Abstracted out π£ foo!/* from foo/*
ABSTRACTION & DSLS IMPROVING Kernel β BANG FUNCTIONS π£ Abstracted out π£ foo!/* from foo/*
ABSTRACTION & DSLS IMPROVING Kernel β BANG FUNCTIONS π£ Abstracted out π£ foo!/* from foo/* π β©
ABSTRACTION & DSLS IMPROVING Kernel β BANG FUNCTIONS π£ Abstracted out π£ foo!/* from foo/* π β© π£
ABSTRACTION & DSLS IMPROVING Kernel β BANG FUNCTIONS π£ Works everywhere Any data Any error struct Abstracted out π£ foo!/* from foo/* π β© π£ Any flow (esp. pipes) Super easy to test
ABSTRACTION & DSLS IMPROVING Kernel β BANG FUNCTIONS π£ Works everywhere Any data Any error struct Abstracted out π£ foo!/* from foo/* π β© π£ Any flow (esp. pipes) Super easy to test BONUS Disambiguate between nil value and actual errors
ABSTRACTION STO RY T E L L I N G π
ABSTRACTION STO RY T E L L I N G π
ABSTRACTION STO RY T E L L I N G π
FIGHTING GenSoup
FIGHTING GenSoup π«π²β
FIGHTING GenSoup G O O D I N T E R FA C E S ! = G O O D A B S T R A C T I O N S
FIGHTING GenSoup G O O D I N T E R FA C E S ! = G O O D A B S T R A C T I O N S β’ GenServer & co are actually pretty low level β’ Add some semantics! β’ Donβt reinvent the wheel every time π‘ β’ Letβs look at a very common example
FIGHTING GenSoup A B S T R A C T I O N β I N T E R FA C E / F R O N T E N D
FIGHTING GenSoup A B S T R A C T I O N β I N T E R FA C E / F R O N T E N D
FIGHTING GenSoup SIMPLE SYNCHRONOUS CASE (BACK END)
FIGHTING GenSoup A S Y N C C A S E β U N D E R LY I N G M E C H A N I C S
FIGHTING GenSoup A S Y N C C A S E β I M P L E M E N TAT I O N ( B A C K E N D )
FIGHTING GenSoup W H AT D I D W E G E T ?
FIGHTING GenSoup W H AT D I D W E G E T ? β’ Common interface β’ Encapsulate the detail β’ Donβt have to think about mechanics anymore
FIGHTING GenSoup ABSTRACTION = FOCUS/ESSENCE
FIGHTING GenSoup ABSTRACTION = FOCUS/ESSENCE
FIGHTING GenSoup ABSTRACTION = FOCUS/ESSENCE π
LE T βS DO SOME THING WILD
LE T βS DO SOME THING WILD β‘π₯ POWER UP πͺπ
POWER UP EXPLICIT ASSUMPTIONS
POWER UP EXPLICIT ASSUMPTIONS β’ Parallel pipes
POWER UP EXPLICIT ASSUMPTIONS β’ Parallel pipes β’ Concurrency = partial order
POWER UP EXPLICIT ASSUMPTIONS β’ Parallel pipes β’ Concurrency = partial order t
POWER UP EXPLICIT ASSUMPTIONS β’ Parallel pipes β’ Concurrency = partial order t
POWER UP EXPLICIT ASSUMPTIONS β’ Parallel pipes β’ Concurrency = partial order t
POWER UP EXPLICIT ASSUMPTIONS β’ Parallel pipes β’ Concurrency = partial order t
POWER UP EXPLICIT ASSUMPTIONS β’ Parallel pipes β’ Concurrency = partial order t
POWER UP EXPLICIT ASSUMPTIONS β’ Parallel pipes β’ Concurrency = partial order β’ Monotonic t
POWER UP EXPLICIT ASSUMPTIONS β’ Parallel pipes β’ Concurrency = partial order β’ Monotonic β’ Properties β’ Serial composition β’ Parallel composition β’ Explicit evaluation strategy t
POWER UP PIPES++
POWER UP PIPES++
POWER UP PIPES++
POWER UP PIPES++
POWER UP P ROTO C O L / β F RO N T E N D β
POWER UP CLEANUP
POWER UP CLEANUP
POWER UP C A R R I E R D ATA
POWER UP C A R R I E R D ATA
POWER UP C A R R I E R D ATA
POWER UP C A R R I E R D ATA
POWER UP C A R R I E R D ATA
POWER UP SIMPLE CASE BACK END
POWER UP SIMPLE CASE BACK END
POWER UP SIMPLE CASE BACK END
POWER UP ASYNC BACK END
POWER UP ASYNC BACK END
POWER UP ASYNC BACK END
POWER UP ASYNC BACK END
POWER UP U P S H OT
POWER UP U P S H OT β’ Higher semantic density (focused on meaning not mechanics)
POWER UP U P S H OT β’ Higher semantic density (focused on meaning not mechanics) β’ Declarative, configurable data flow π€―
POWER UP U P S H OT β’ Higher semantic density (focused on meaning not mechanics) β’ Declarative, configurable data flow π€― β’ Extremely extensible β’defimpl Dataflow, for: %Stream{} β’defimpl Dataflow, for: %Distributed{} β’defimpl Dataflow, for: %Broadway{}
POWER UP U P S H OT β’ Higher semantic density (focused on meaning not mechanics) β’ Declarative, configurable data flow π€― β’ Extremely extensible β’defimpl Dataflow, for: %Stream{} β’defimpl Dataflow, for: %Distributed{} β’defimpl Dataflow, for: %Broadway{} β’ Model-testable
POWER UP U P S H OT β’ Higher semantic density (focused on meaning not mechanics) β’ Declarative, configurable data flow π€― β’ Extremely extensible β’defimpl Dataflow, for: %Stream{} β’defimpl Dataflow, for: %Distributed{} β’defimpl Dataflow, for: %Broadway{} β’ Model-testable β’ Composable with other pipes and change evaluation strategies
A CALL FOR LIBRARIES
A CALL FOR LIBRARIES π£
A CALL FOR LIBRARIES EXTEND RAILROAD PROGRAMMING
A CALL FOR LIBRARIES EXTEND RAILROAD PROGRAMMING Happy Path (Continue) Error Case (Skip) No Effect (Afterwards)
A CALL FOR LIBRARIES EXTEND RAILROAD PROGRAMMING Happy Path (Continue) Error Case (Skip) No Effect (Afterwards)
A CALL FOR LIBRARIES EXTEND RAILROAD PROGRAMMING Happy Path (Continue) Error Case (Skip) No Effect (Afterwards)
A CALL FOR LIBRARIES S U R P R I S I N G N U M B E R O F FA C T O R S
A CALL FOR LIBRARIES S U R P R I S I N G N U M B E R O F FA C T O R S Log Program
A CALL FOR LIBRARIES S U R P R I S I N G N U M B E R O F FA C T O R S Log Program
SUMMARY
SUMMARY KEEP IN MIND
SUMMARY KEEP IN MIND 1. Protocols-for-DDD 2. Add a semantic layer 3. How do you locally test your distributed system? Look at the properties! 4. Under which conditions does your code work? What are your assumptions? 5. Prop testing is useful for structured abstractions 6. You should be able to code half-asleep
https://fission.codes https://talk .fission.codes https://tools.fission.codes π THANK YOU, INTERNE T π brooklyn@fission.codes g i t h u b . c o m /e x p e d e @expede
Dijkstra banished the GOTO 50 years ago, but unstructured programming continues to lurk in the background of every module we write. As our software gets larger and does more, that complexity resurfaces in new and less obvious ways. As much as the actor model is lauded for being like an organism, organic code can become full of structural complexity.
Is there a way to strengthen our code, while retaining its flexibility and making it even clearer to communicate our intent? There is! The decades since, weβve developed new techniques and patterns to structure our code. And best of all, we can have them encoded in libraries and modules directly! These techniques are starting to become enshrined in standard libraries of other languages β can they be brought to Elixir?
for free. You
can too.