Refactoring to modules: Why and how – all you need to know in (less than) an hour

Refactoring to modules: Why and how – all you need to know in (less than) an hour @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

shownotes http://jfrog.com/shownotes Slides Video Links Comments, Ratings Raffle @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

The speakers @jbaruch @jbaruch @eyalbe4 #GopherConIL @eyalbe4 http://jfrog.com/shownotes

Let’s go back in time

Poll time! Pre 1.0 (2012) 1.2 (2013) 1.5 (2015) 1.8 (2017) 1.11 (2019) @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

Why we have a problem? @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

Why we have a problem? @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

Simple solution! Dependencies are sources Remote import is a VCS path Dump everything together into one source tree (GOPATH) Compile Profit @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

@jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

But… how do i… Know which dependencies do I use? Know which dependencies did you use? Know which dependencies should I use? Know is it our code that I am editing right now? WTF is going on?! @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

Yeah… “ To date, we’ve resorted to an email semaphore whenever someone fixes a bug a package, imploring everyone else to run go get -u. You can probably imagine how successful this is, and how much time is being spent chasing bugs that were already fixed. Dave Cheney @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

Duplicate your dependencies “ Check your dependencies to your own VCS. Brad Firzpatrick @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

Build your own dependency manager “ It’s not the role of the tooling provided by the language to dictate how you manage your code in the production sense. Andrew Gerrand @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

We expect you to already have a homegrown dependency manager “ If you need to build any tooling around what Go uses (Git, Mercurial, Bazaar), you already understand those tools, so it should be straightforward to integrate with whatever system you have. Andrew Gerrand @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

Don’t trust what we’ve built “ go-get is nice for playing around, but if you do something serious, like deploying to production, your deploy script now involves fetching some random dude’s stuff on GitHub. Brad Firzpatrick @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

quiz time! godeps.json dependencies.tsv govendor, govend, goven, gv trash, garbage, rubbish Weapons manufacturer @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

GOPATH + VENDORING = ! @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

GOPATH, The proud son of the monorepo

Two huge problems with gopath It only allows a single version of any given package to exist at once (per GOPATH) We cannot programmatically differentiate between code the user is working on and code they merely depend on @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

vendoring “ Copy all of the files at some version from one version control repository and paste them into a different version control repository @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

What’s wrong with it (well, what’s not) History, branch, and tag information is lost Pulling updates is impossible It invites modification, divergence, and bad fork It wastes space Good luck finding which version of the code you forked @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

Still wrong! You still have no idea what version are you using You have to connect each dependency as a submodule manually Switching branches and forks LOL Working on modules with other teams ROFL @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

@jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

The go dep experiment

@jbaruch #dockercon jfrog.com/shownotes

Proper dependency management? Working in project directories Local cache for dependencies Version declarations Conflict resolution @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

Conflict on the conflict resolution SAT/SMT vs MVS/SIV

Enter Go modules

Enter go modules @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

Backwards compatibility and migration go mod init go.mod is created The rest is the same: imports in code just work @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

That’s some serious dark magic… @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

Go modules convert everything (almost?) @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

What happens to go.mod when you add import (and run go get/go build) Go checks the URL: If it’s Go Proxy (module repository), it gets the module If it’s a VCS it clones and builds the module locally If it’s a web page, looks for go-import meta tag Selects the latest compatible version tag Semantic import versioning @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

@jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

Compatible?! Let’s assume SemVer works (LOL) The latest version of v1.x.x is compatible with v1.0.0 and up Premise: import path string should always be backwards compatible @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

What about version 2?! Incompatible code can’t use the same import path Add /v2/ to the module path Use /v2/ in the import path import “github.com/my/module/v2/mypkg” @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

What if it doesn’t have any semver tags?! Pseudo version v0.0.0-yyyymmddhhmmss-abcdefabcdef @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

What if (when) I want to ban a version?! You can specify “version X or later”: >= x.y.z You can use exclude or replace for better control @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

From vendoring to hierarchy of module repositories

Go modules define an hierarchy of caches Public Modules Repository GoCenter Organizational Modules Repository Project Athens JFrog Artifactory Local cache on the developer’s machine $GOPATH/pkg/mod

Local cache on the developer’s machine After the mods are resolved (or built) they are cached in $GOPATH/pkg/mod Provides immediate access Not shared Not reliable (can be wiped at any moment) @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

Organizational modules repository JFrog Artifactory or Athens Provides faster (Intranet) access Provides reproducible builds as it caches the dependencies used once for build reproduction Requires team infrastructure and maintenance (SaaS offers exist) @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

public modules repositories GoCenter Google announced a vision for a federation of public repositories Provides fast access Provides reproducible builds as it caches the popular and requested dependencies from version control Highly available, requires no maintenance, free @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

Demo time! @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes

Twitter ads and q&A jfrog.com/shownotes @eyalbe4 @jbaruch #GopherConIL @jbaruch @eyalbe4 #GopherConIL http://jfrog.com/shownotes