Devel-ops: Bringing development practices to operations tasks

Slide 1

Slide 2

Slide 3

Slide 4

Jonathan Relf Devel-Ops - Bringing Development Practices to Operations Tasks

Slide 5

27th June 2019 Jonathan Relf DEVEL-OPS

Slide 6

Commify is the team behind a global portfolio of business messaging brands. We work with more than 45,000 companies, helping them transform their mobile communications with their customers and staff.

Slide 7

We provide SMS, voice, web, IP/OTT, email and intelligent multichannel messaging services both on a self-serve basis (through an online platform or API), and as tailored solutions to more complex needs. www.commify.com

Slide 8

BRINGING DEVELOPMENT PRACTICES TO OPERATIONS TASKS DEVEL-OPS

Slide 9

THIS TALK IS NOT: VS Devs Ops

Slide 10

THIS TALK IS MORE:

Slide 11

@jbjon

Slide 12

“THE DEFINITION OF INSANITY IS REPEATING THE SAME MISTAKES OVER AND OVER AGAIN AND EXPECTING DIFFERENT RESULTS” Unknown @jbjon

Slide 13

“THE DEFINITION OF INSANITY IN I.T. OPERATIONS IS MANUALLY REPEATING A TASK OVER AND OVER AGAIN AND EXPECTING THE SAME RESULTS” Jonathan Relf @jbjon

Slide 14

Slide 15

THE IRON AGE OF I.T.

Slide 16

THE CLOUD AGE OF I.T.

Slide 17

“UNRELIABLE SOFTWARE DEPENDING ON RELIABLE HARDWARE TO RELIABLE SOFTWARE RUNNING ON UNRELIABLE HARDWARE” Infrastructure As Code @jbjon

Slide 18

2017

Slide 19

2018

Slide 20

Slide 21

Slide 22

Slide 23

Slide 24

Slide 25

Slide 26

Slide 27

Slide 28

Slide 29

Slide 30

https://landscape.cncf.io/

Slide 31

THOUGHTWORKS RADAR @jbjon https://www.thoughtworks.com/radar/tools

Slide 32

https://blogs.technet.microsoft.com/xdot509/2013/07/24/getting-started-with-windows-azure-part-2-what-are-cloud-services/

Slide 33

WHY BOTHER DEFINING YOUR OWN INFRASTRUCTURE IN CODE? @jbjon

Slide 34

REGULATORY REQUIREMENTS

Slide 35

3RD PARTY SOLUTION UNCERTAINTY

Slide 36

HARDEN APPLICATION HOSTING

Slide 37

AVOID VENDOR LOCK-IN

Slide 38

Slide 39

MAINTAINING SERVERS OVER TIME OWN SOFTWARE CONFIGURATION UPDATES 3RD PARTY SOFTWARE CONFIGURATION UPDATES OPERATING SYSTEM FEATURES CONFIGURATION HYPERVISOR CONFIGURATION UPDATES PHYSICAL SERVER CONFIGURATION UPDATES UPDATES

Slide 40

THE FOUR STAGES OF INFRASTRUCTURE PROVISIONING CONFIGURING MAINTENANCE TERMINATION @jbjon

Slide 41

PROVISIONING ▸ Since virtualisation, one of the easiest of the phases ▸ Traditionally based off ISO images ▸ Out of date, unpatched images @jbjon

Slide 42

CONFIGURATION ▸ Can be where most configuration drift is added if not done through automation ▸ It may involve ‘tinkering’ to get a server working ▸ Can end up with ‘Snowflake servers’ ▸ Configuration Management software like Puppet & Chef @jbjon

Slide 43

MAINTENANCE ▸ Updates or upgrades of software components ▸ From security patches to in-place upgrades of O.S. ▸ Ordering of patches may affect outcome ▸ Scripting can help ensure consistency @jbjon

Slide 44

TERMINATION ▸ Fear of shutting off servers ▸ Treating servers like “pets, not cattle” ▸ Anti-pattern: Celebrating up-time ▸ Plan for the fact an instance could disappear @jbjon

Slide 45

Slide 46

GOALS OF INFRASTRUCTURE AS CODE @jbjon

Slide 47

IT INFRASTRUCTURE SUPPORTS & ENABLES CHANGE Kief Morris @jbjon

Slide 48

CHANGES TO THE SYSTEM ARE ROUTINE WITHOUT DRAMA OR STRESS Kief Morris @jbjon

Slide 49

IT STAFF SPEND THEIR TIME ON VALUABLE THINGS… NOT REPETITIVE TASKS Kief Morris @jbjon

Slide 50

USERS ARE ABLE TO DEFINE, PROVISION, AND MANAGE THE RESOURCES THEY NEED WITHOUT I.T. STAFF TO DO IT FOR THEM Kief Morris @jbjon

Slide 51

DEFINITION TOOLS ▸ Good Infrastructure As Code tools ▸ have scriptable interfaces ▸ can be run unattended ▸ can be tailored through config ▸ allow tasks to be defined in code ▸ the definition files become ‘living documentation’ @jbjon

Slide 52

Slide 53

VERSION CONTROL ▸ Natural part of development workflow ▸ branching, rollbacks, ownership ▸ Single point of truth ▸ Living documentation @jbjon

Slide 54

CONTINUOUS INTEGRATION ▸ Early feedback about potentially breaking changes ▸ Changes tested in isolation from production ▸ Can apply to infrastructure, server, and configuration changes ▸ “Does this produce the instance I expect?” ▸ “Does this instance have all the features I expect?” ▸ “Is this instance configured for its role correctly?” @jbjon

Slide 55

BUILD PIPELINES ▸ Avoid ‘automation fear’ ▸ Build regularly as well as on changes ▸ Pipelines maintaining templates ensures up-to-date images ▸ Reduces manual knowledge fading ▸ Services used to build can be provisioned temporarily ▸ Packer supports building machine images @jbjon

Slide 56

AUTOMATED TESTING ▸ One of the best practices to borrow from Development ▸ Not relying on ‘Green builds’ ▸ Frameworks like ServerSpec (http://serverspec.org) @jbjon

Slide 57

CONTINUOUS DELIVERY ▸ Not ‘Continuous Deployment’ ▸ Being able to update Test / Lab environments regularly ▸ Risk increases with ‘Time Since Last Success’ @jbjon

Slide 58

CODE ANALYSIS ▸ Emerging area drawing from Dev practices ▸ Config management declarative languages have ‘lint’ tools @jbjon

Slide 59

“A NETFLIX TEAM KNEW THAT A PERCENTAGE OF AWS INSTANCES, WHEN PROVISIONED, WILL PERFORM MUCH WORSE THAN THE AVERAGE INSTANCE SO THEY WROTE THEIR PROVISIONING SCRIPTS TO IMMEDIATELY TEST THE PERFORMANCE OF EACH NEW INSTANCE. IF IT DOESN’T MEET THEIR STANDARDS, THE SCRIPT DESTROYS THE INSTANCE AND TRIES AGAIN WITH A NEW INSTANCE.” Kief Morris @jbjon

Slide 60

WAYS TO START INTRODUCING THIS ▸ Start small ▸ Script everything ▸ Automate the process ▸ Run it regularly ▸ Test the changes in a safe environment ▸ Monitor all the things @jbjon

Slide 61

Slide 62

QUESTIONS?

Slide 63

ABOUT ME ▸ Jonathan Relf ▸ Solutions Architect @ Commify ▸ about.me/jbjon @jbjon