A presentation at Client event (commissioned keynote) by Holly Cummins
the adventurer’s guide to breaking production Holly Cummins @holly_cummins @holly_cummins #IBM
me: “innovation leader” at IBM @holly_cummins #IBM
me: “innovation leader” at IBM translation: getting into trouble with technology. @holly_cummins #IBM
me: “innovation leader” at IBM translation: getting into trouble with technology. … for 20 years @holly_cummins #IBM
how often do we get to go someplace truly new? @holly_cummins #IBM
see things we’ve never seen before? @holly_cummins #IBM
in software we do it all the time @holly_cummins #IBM
ancient landscapes @holly_cummins #IBM
legacy environments @holly_cummins #IBM
client name redacted @holly_cummins new territories #IBM
@holly_cummins #IBM
what could possibly go wrong? @holly_cummins #IBM
um … what problem were we trying to solve? @holly_cummins #IBM
George Mallory Died on Everest… 30 years before Edmund Hillary because it’s there @holly_cummins #IBM new for the sake of it
surprise inside @holly_cummins #IBM
the pace of change is fast @holly_cummins #IBM
the pace of change is fast @holly_cummins #IBM
the landscape is complicated @holly_cummins #IBM
the old stuff hasn’t gone away @holly_cummins #IBM
stuff slows us down @holly_cummins #IBM
@holly_cummins #IBM
@holly_cummins #IBM
and … @holly_cummins #IBM
ok but what’s the worst that could go wrong? @holly_cummins #IBM
Knight Capital $460 million loss @holly_cummins #IBM
Knight Capital $460 million loss in 45 minutes @holly_cummins #IBM
@holly_cummins #IBM
the million-dollar frozen database @holly_cummins #IBM
true story, unfortunately Hey boss, I created a Kubernetes cluster. Holly @holly_cummins #IBM
true story, unfortunately Hey boss, I created a Kubernetes cluster. I forgot it for 2 months. Holly @holly_cummins #IBM
true story, unfortunately Hey boss, I created a Kubernetes cluster. I forgot it for 2 months. … and it’s £1000 a month. Holly @holly_cummins #IBM
bugs, but in space Phobos 1 @holly_cummins #IBM
“we couldn’t get the automated checks to work, so we bypassed them” @holly_cummins #IBM
“the space probe is bricked.” @holly_cummins #IBM
what causes bugs? @holly_cummins #IBM
other people @holly_cummins #IBM
us @holly_cummins #IBM
@holly_cummins #IBM
interactions @holly_cummins #IBM
“every time we change one microservice, another breaks” #IBM @holly_cummins
distributed != decoupled @holly_cummins #IBM
managing bugs @holly_cummins #IBM
breaking production isn’t the worst thing @holly_cummins #IBM
as long as it’s a small break @holly_cummins #IBM
as long as it’s a tiny break @holly_cummins #IBM
limit blast radius @holly_cummins #IBM
canary deploys @holly_cummins #IBM
breaking production isn’t the worst thing the important thing is how fast you can unbreak production @holly_cummins #IBM
recoverability @holly_cummins #IBM
unrecoverable @holly_cummins #IBM
unbreak @holly_cummins #IBM
unbreak diagnose @holly_cummins #IBM
unbreak deploy diagnose @holly_cummins #IBM
unbreak deploy diagnose observability @holly_cummins #IBM
unbreak deploy diagnose observability @holly_cummins #IBM devops
unbreak deploy diagnose observability @holly_cummins #IBM devops
my most embarrassing break of production @holly_cummins #IBM
my most embarrassing break of production @holly_cummins #IBM
my most embarrassing break of production @holly_cummins #IBM
most problems are harder to diagnose @holly_cummins #IBM
observability @holly_cummins #IBM
observability what you don’t have to do … if you have observability @holly_cummins #IBM
unbreak deploy diagnose observability @holly_cummins #IBM devops
make releases deeply boring @holly_cummins #IBM
so you can do them make all the time releases deeply boring @holly_cummins #IBM
CI/CD rst code second #IBM fi @holly_cummins
GitOps @holly_cummins #IBM
GitOps infrastructure as code @holly_cummins #IBM
ok but preventing problems? @holly_cummins #IBM
pair programming @holly_cummins #IBM
test-driven development (TDD) @holly_cummins #IBM
if you care about it, automate it @holly_cummins #IBM
integrate early and often @holly_cummins #IBM
integrate early and often many times a day @holly_cummins #IBM
contract test your interactions @holly_cummins #IBM
the problem with mocks @holly_cummins our code #IBM their code
the problem with mocks @holly_cummins our code #IBM our mock
the problem with mocks our code tests ✔ @holly_cummins #IBM our mock
the problem with mocks our code our mock tests ✔ @holly_cummins #IBM our code their actual code
the problem with mocks our code our mock tests ✔ reality ✘ @holly_cummins #IBM our code their actual code
the problem with mocks @holly_cummins our code #IBM their code
the problem with mocks @holly_cummins our code #IBM contract test their code
mock the problem with mocks @holly_cummins our code #IBM contract test their code
mock the problem with mocks @holly_cummins our code #IBM functional test contract test their code
mock the problem with mocks our code functional test contract test our tests ✔ their tests ✔ reality ✔ @holly_cummins #IBM their code
mock the problem with mocks @holly_cummins our code #IBM functional test contract test their code
mock the problem with mocks our code functional test contract test our tests ✔ their tests ✘ reality ✘ @holly_cummins #IBM their code
mock the problem with mocks @holly_cummins our code #IBM functional test contract test their code
mock the problem with mocks our code functional test contract test our tests ✘ their tests ✔ reality ✘ @holly_cummins #IBM their code
@holly_cummins #IBM
demo @holly_cummins #IBM
2014 2021 Ant Tekton Java 7 OSGi WebSphere Kubernetes OpenShift Node.js React.js locally deployed on public cloud my stack @holly_cummins #IBM
your ability to learn is a key professional asset @holly_cummins #IBM
teach people the stuff you’re learning @holly_cummins #IBM
@holly_cummins #IBMGarage
2008: a developer had a lot of fun with Groovy. @holly_cummins #IBMGarage
2008: a developer had a lot of fun with Groovy. 2009: he left the company; the others who had to maintain his code had less fun. @holly_cummins #IBMGarage
pair programming @holly_cummins #IBM
the value of discomfort @holly_cummins #IBM
@holly_cummins #IBM
TDD (test driven development) @holly_cummins #IBM
TDD (test driven development) BDD (behaviour driven development) @holly_cummins #IBM
TDD (test driven development) BDD (behaviour driven development) CDD (cake driven development) @holly_cummins #IBM
TDD (test driven development) BDD (behaviour driven development) CDD (cake driven development) PDD (pain driven development) @holly_cummins #IBM
harness discomfort to drive innovation @holly_cummins #IBM
learning comes from failure @holly_cummins #IBM
success comes from learning @holly_cummins #IBM
thank you! (and have fun at the rest of the event) Holly Cummins @holly_cummins @holly_cummins #IBM
This was a keynote written for a private client conference.
for free. You
can too.