A presentation at DevOps Vision in in Florida, USA by Sasha Czarkowski (Rosenbaum)
Making DevOps Valuable Sasha Rosenbaum @DivineOps COPYRIGHT 2024. ALL RIGHTS RESERVED.
Sasha Czarkowski (Rosenbaum) @DivineOps Dev Ops Product Sales Management
Ergonautic Stop Chasing Buzzwords Start Working Better @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
Ergonautic Stop Chasing Buzzwords Start Working Better @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
Once upon a time Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Traditional IT dev ops wall of confusion Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Dev Ops Speed Reliability
DevOpsDays Ghent 2009 Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
Jez Humble and Dave Farley: 2010 COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
@ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
“DevOps” Google Search Trend 2009-Present Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
State of DevOps Report 2019 Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
@ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
We have the technology… Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Just move faster! Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
You cannot CI/CD Without Testing Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
Did we get DevOps wrong? Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
The future is already here. It’s just not evenly distributed ~ William Gibson Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
CALMS Sasha Czarkowski Culture Automation Lean Measurement Sharing COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
CALMS Sasha Czarkowski Culture Automation Lean Measurement Sharing COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
Culture Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
Popular Perception of Culture Change Assumptions Sasha Czarkowski Values COPYRIGHT 2023. ALL RIGHTS RESERVED. Artifacts (behavior) www.ergonautic.ly
“Anyone wanting to change a culture needs to define the actions and behaviors they desire, then design the work processes that are necessary to reinforce those behaviors.” - John Shook, MIT Sloan Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
Culture Change (Schein) Artifacts (behavior) Sasha Czarkowski Values COPYRIGHT 2023. ALL RIGHTS RESERVED. Assumptions www.ergonautic.ly
Motivation Matching: Help people do what they already want to do. - BJ Fogg Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
Automation Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
@ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
Naive automation Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
This new tool is going to solve ALL of our problems! Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Automation Drift @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
The irony of manual takeover Source: Ironies of Automation. https://ckrybus.com/static/papers/Bainbridge_1983_Automatica.pdf Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
All systems are sociotechnical Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Entropy always wins @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
Inevitably, the friction with the machine is getting blamed on human error Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Let’s talk about the forgotten pillar Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Measurement Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
The aim of the system must be clear to everyone in the system. Without an aim, there is no system. The aim is a value-judgment. - W. Edwards Deming Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
We want to measure Value Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Measuring Value delivery is Hard Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Just measure things! Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Metrics Done Wrong ● Subjective ● Imbalanced ● Only a Proxy Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
Velocity ● Subjective ● Was there a story-point inflation? ● Imbalanced ● Did we sacrifice quality? ● Only a Proxy ● Did we deliver more units of value? Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
What gets measured, gets gamed Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
COPYRIGHT 2024. ALL RIGHTS RESERVED.
You want a balanced system of metrics Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Process Quality Creativity Reuse Speed
Let’s all use the 4 Golden Signals! Can you connect them to revenue, costs or customer experience? @ergonauticly Latency Traffic Errors Saturation COPYRIGHT 2023. ALL RIGHTS RESERVED. www.ergonautic.ly
Google: 500ms of latency dropped traffic by 20% Amazon: 100ms of latency dropped sales by 1% https://www.gigaspaces.com/blog/amazon-found-every-100ms-of-latency-cost-them-1-in-sales/
Ergonautic milliseconds of latency do not matter!
The narrative must be meaningful to your business @ergonauticly COPYRIGHT 2023. ALL RIGHTS RESERVED. Ergonautic
The true goal Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
Measuring flow of Value through the system Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
What is Reliability? @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Reliability vs Availability @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Everything is 200 OK! @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
The five nines 99.999% @ergonauticly 5.26 mins / year COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Will your users even notice? @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
The ISP background error rate is 0.01% - 1% @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
100% availability is… ● Unattainable ● Unnecessary ● Extremely expensive Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
Will your users care? Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
COPYRIGHT 2024. ALL RIGHTS RESERVED.
Reliability is a Feature Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Reliability Is my service doing what the users expect it to do? Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
SLA, SLI, SLO Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
SLA = Financially-backed Availability @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Monthly downtime > 1.5 days means 100% refund COPYRIGHT 2024. ALL RIGHTS RESERVED.
SLAs are about aligning incentives between Vendor & Customer @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
SLA usually includes a single metric For financial and reputational reasons, companies prefer to underpromise and overdeliver @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
SLI @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
SLI = Actual Reliability @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Monitoring @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Without monitoring, you have no way to tell whether your service even works! @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
Good Monitoring @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Without good monitoring, you don’t know that the service does what users expect it to do! @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
SLO @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
SLO = target reliability @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
SLO = Business-approved reliability @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
SLOs are about explicitly aligning incentives between Business & Engineering @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Error Budgets @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Error Budgets Acceptable level of unreliability Error budget = 1 - SLO EB = 1 – 99.99% = 0.01% ≃ 13 mins /quarter @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
Error budgets are about aligning incentives between Dev & Ops @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
If developers are measured on the same SLO, then when the error budget is drained developers shift focus from delivering new features to improving reliability @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
What happens when you blow the Error Budget? Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Nothing @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
@ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
SLO = Targeted Reliability Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Define SLOs to be the best proxies of user value Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Reliability is a Feature Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Start with SLIs Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Without good monitoring, you don’t know that the service does what users expect it to do! @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
Information -> Insight Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
What’s the action? What’s the action?
ODIM Objective, Decisions, Insights, Measurements Working backwards from intended outcomes Given [Objective] what decisions do we need to make? Given the need to [Decision] what insights do we need have to improve it? Given the need for [Insight] what do we need to [Measure]? By: Larry Maccherone, https://medium.com/@lmaccherone/odim-12d80823222 @ergonauticly COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Desired State Current State Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
Models the System Control Loop Controller Decision Information Sensor Actuator Acts on the System Measures the System Action Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. www.ergonautic.ly
Improving Productivity: Controller
It is control loops all the way down Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Making your DevOps valuable Align Measurements and Incentives to delivering Value to the user Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
The journey of a thousand miles begins with a single step Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
The future is already here. It’s just not evenly distributed ~ William Gibson Sasha Czarkowski COPYRIGHT 2024. ALL RIGHTS RESERVED. Ergonautic
Thank you! Sasha Czarkowski (Rosenbaum) COPYRIGHT 2024. ALL RIGHTS RESERVED.
In the past decade, most organizations have invested in DevOps, and by now many of those efforts have failed to meet expectations of the executive sponsors. Organizations have brought in the latest and greatest tools and often sped up their development process without seeing a clear benefit. Some invested in culture only to see collaboration wane and any progress stall. The DevOps pendulum swings between focus on technology and culture. The buzzwords, rituals and tools keep changing, but the results often seem to worsen over time.
Optimizing any one part of a system is often suboptimal for the whole system. The goal should never be to adopt but to adapt to the opportunities and challenges of our organizations, but what are those? This talk will give attendees a framework to analyze the flow of work through their unique sociotechnical system, to understand the specific flow of value and propose metrics to identify and prioritize iterative improvements. The presentation will be technology agnostic but will focus on mapping capabilities that appropriate technical tools and skills should achieve together.