9th October 2018 Jonathan Relf DEVEL-OPS
A presentation at The IT Crewe October 2018 in October 2018 in Crewe, UK by Jonathan Relf
9th October 2018 Jonathan Relf DEVEL-OPS
BRINGING DEVELOPMENT PRACTICES TO OPERATIONS TASKS DEVEL-OPS
THIS TALK IS NOT: Devs Vs Ops
THIS TALK IS MORE: embracing the chance to work together more. To be collaborative.
DEVEL-OPS “THE DEFINITION OF INSANITY IS REPEATING THE SAME MISTAKES OVER AND OVER AGAIN AND EXPECTING DIFFERENT RESULTS” Unknown
DEVEL-OPS “THE DEFINITION OF INSANITY IN I.T. OPERATIONS IS MANUALLY REPEATING A TASK OVER AND OVER AGAIN AND EXPECTING THE SAME RESULTS” Jonathan Relf
THE IRON AGE OF I.T.
THE CLOUD AGE OF I.T.
DEVEL-OPS “UNRELIABLE SOFTWARE DEPENDING ON RELIABLE HARDWARE TO RELIABLE SOFTWARE RUNNING ON UNRELIABLE HARDWARE” Infrastructure As Code
2017
https://blogs.technet.microsoft.com/xdot509/2013/07/24/getting-started-with-windows-azure-part-2-what-are-cloud-services/
May have regulatory or data sovereignty limitations ▸ Unsure of what you are getting using 3rd party solutions ▸ Allowing you to ‘harden’ application hosting ▸ Reducing the risk of ‘vendor lock’
OWN SOFTWARE CONFIGURATION UPDATES 3RD PARTY SOFTWARE CONFIGURATION UPDATES OPERATING SYSTEM FEATURES CONFIGURATION HYPERVISOR CONFIGURATION UPDATES PHYSICAL SERVER CONFIGURATION UPDATES UPDATES
PROVISIONING CONFIGURING MAINTENANCE TERMINATION
▸ Since virtualisation, one of the easiest of the phases ▸ Traditionally based off ISO images ▸ Out of date, unpatched images
▸ Can be where most configuration drift is added if not done through automation ▸ It may involve ‘tinkering’ to get a server working ▸ Can end up with ‘Snowflake servers’ ▸ Configuration Management software like Puppet & Chef
▸ Updates or upgrades of software components ▸ From security patches to in-place upgrades of O.S. ▸ Ordering of patches may affect outcome ▸ Scripting can help ensure consistency
▸ Fear of shutting off servers ▸ Treating servers like “pets, not cattle” ▸ Anti-pattern: Celebrating up-time ▸ Plan for the fact an instance could disappear
▸ IT infrastructure supports & enables change ▸ Changes to the system are routine, without drama or stress ▸ IT staff spend their time on valuable things.. not repetitive tasks ▸ Users are able to define, provision, and manage the resources they need, without needing IT staff to do it for them
▸ Good Infrastructure As Code tools ▸ have scriptable interfaces ▸ can be run unattended ▸ can be tailored through config ▸ allow tasks to be defined in code ▸ the definition files become ‘living documentation’
▸ Version Control ▸ Continuous Integration ▸ Build Pipelines ▸ Automated Testing ▸ Continuous Delivery ▸ Code Analysis
▸ Natural part of development workflow ▸ branching, rollbacks, ownership ▸ Single point of truth ▸ Living documentation
▸ Early feedback about potentially breaking changes ▸ Changes tested in isolation from production ▸ Can apply to infrastructure, server, and configuration changes ▸ "Does this produce the instance I expect?" ▸ "Does this instance have all the features I expect?" ▸ "Is this instance configured for its role correctly?"
▸ Avoid 'automation fear' ▸ Build regularly as well as on changes ▸ Pipelines maintaining templates ensures up-to-date images ▸ Reduces manual knowledge fading ▸ Services used to build can be provisioned temporarily ▸ Packer supports building machine images
▸ One of the best practices to borrow from Development ▸ Not relying on ‘Green builds’ ▸ Frameworks like ServerSpec (http://serverspec.org)
▸ Not ‘Continuous Deployment’ ▸ Being able to update Test / Lab environments regularly ▸ Risk increases with ‘Time Since Last Success’
▸ Emerging area drawing from Dev practices ▸ Config management declarative languages have ‘lint’ tools
“A NETFLIX TEAM KNEW THAT A PERCENTAGE OF AWS INSTANCES, WHEN PROVISIONED, WILL PERFORM MUCH WORSE THAN THE AVERAGE INSTANCE SO THEY WROTE THEIR PROVISIONING SCRIPTS TO IMMEDIATELY TEST THE PERFORMANCE OF EACH NEW INSTANCE. IF IT DOESN’T MEET THEIR STANDARDS, THE SCRIPT DESTROYS THE INSTANCE AND TRIES AGAIN WITH A NEW INSTANCE.” Kief Morris
▸ Start small ▸ Script everything ▸ Automate the process ▸ Run it regularly ▸ Test the changes in a safe environment ▸ Monitor all the things
▸ Jonathan Relf ▸ Solutions Architect @ Commify ▸ https://about.me/jbjon @jbjon
