Des silos au Platform Engineering en passant par le DevOps : adopter GitOps et aller au-delà de l’hype

Des silos au Platform Engineering en passant par le DevOps Adopter GitOps et aller au-delà de l’hype Horacio Gonzalez Clever Cloud @Sebi2706 - @LostInBrittany Sébastien Blanc Aiven DEVOXX FRANCE 2024

Who are we? Sébastien Blanc DevRel Aiven @sebi2706 @Sebi2706 - @LostInBrittany Horacio Gonzalez DevRel Clever Cloud @LostInBrittany DEVOXX FRANCE 2024

What are we going to see? 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. IT in the 90s Tooling evolves XP, agility and DevOps Enter the Cloud Declarative infrastructure Operators to the rescue GitOps? Platform Engineering? Build your own platform Some examples What about not using K8s? @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

IT in the 90s Once upon a time… @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

In a time almost forgotten When even internet was young… @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

When Windows 95 was the cutting edge And a 100 Mb disk was huge… @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Big companies still used mainframes Bigger, fancier, but still the same old IBM @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Bare-metal based IT reigned Control, reliability, security… But cost, rigidity, logistics… @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Applying the industrial model Trying to shoehorn IT into a model where it doesn’t fit @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Walls & Silos And procedures, and hierarchy, and corporate politics @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Tooling evolves CVS, Ant and (Leeeroy) Jenkins @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Old school procedures @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Tooling empowering changes Theory existed since 1999 But without the right tooling… @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Source control tools Better than copying and renaming folders… @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Dependency management & build Better than grabbing each dependency in their website and running javac by hand… @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Unit testing and continuous integration Hudson CI If Testing is Doubting, let’s doubt automatically @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Monitoring tools No more spending nights looking at a status screen @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Too many changes in a few years Old ways were difficult to change @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

@Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

XP, agility and DevOps Buzzwords that changed the IT @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Extreme Programming @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Manifesto for Agile Software Development @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Breaching walks, breaking down silos @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

The business of Agility The Dark Side rises @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Agile Tooling @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Back to industrial practices ? @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Is DevOps the same than Agility? Can you have one without the other? @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

You could have Agility without DevOps Even if I fail to see why you would want… @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

DevOps is a reaction to the wall of confusion Making the different stakeholders to work together in sync @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

You cannot have DevOps without agility DevOps is about shorter development sprints, increased focus on testing, increasing automation @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

DevOps comes with Agility DevOps is an extension of Agile that includes systems and operations @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Enter the Cloud Renting server time in other’s people infra @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

From virtualisation to the cloud How to use the infrastructure at its full capacity @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

The five pillars of the Cloud @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Cloud demands automation @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

It changes the way how IT works And it demands a mentality change @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Empowering developers Infrastructure is only a click away @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Distributed is the new black Cloud Native architectures and services @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Sysadmins who code Creating tools: automation, monitoring, observability… @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

New roles appear: SRE WTF is a System Reliability Engineer? @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

@Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Declarative Infrastructure The intern metaphor @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Containers make dev life easier @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Less simple if you must operate them Like in a production context @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

And what about microservices? Are you sure you want to operate them by hand? @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

And what about microservices? Are you sure you want to operate them by hand? @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Kubernetes: a full orchestrator @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Kubernetes - Desired State Management @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Terraform - Declarative infra as code @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Operators to the rescue Helping to tame the complexity of K8s and using K8s in heterogeneous systems @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Taming microservices with Kubernetes @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

What about complex deployments @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Tools like Helm helps with complexity @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Helm Charts are configuration Operating is more than installs & upgrades @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

What about legacy? Because not everything needs/wants to be in Kubernetes @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Kubernetes is about automation How about automating human operators? @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Kubernetes Operators A Kubernetes version of the human operator @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Building operators Basic K8s elements: Custom Resources & Controllers @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Custom Resource Definitions Extending Kubernetes API @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Extending Kubernetes API By defining new types of resources, internal or external to the cluster @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

With a CRD you can create CR in the cluster They are the blueprints of the Custom Resources @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Custom Resources are simply data All the logic must be in the Controller @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Kubernetes Controllers Keeping an eye on the resources @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

A reconcile loop Controllers watch the state of the cluster, and make or request changes where needed @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Kubernetes Operator Automating operations @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

What’s a Kubernetes Operator? @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Example: databases Things like adding an instance to a pool, doing a backup, sharding… @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Knowledge encoded in CRDs and Controllers @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Custom Controllers for Custom Resources Operators implement and manage Custom Resources using custom reconciliation logic @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Operator Capability Model Gauging the operator maturity @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

GitOps @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Les devs n’ont pas vocation à avoir accès au cluster Comment faire ? Les devs utilisent déjà Git, et c’est leur source de vérité. Utilisons Git comme source de vérité pour l’infra déclarative @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

A central source of truth @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Continuous Integration(CI) & Continuous Delivery (CD) Build Test Security Checks Release Deploy Stage Deploy Prod Continuous Integration Continuous Delivery @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

What is GitOps? Git is the single source of truth @Sebi2706 - @LostInBrittany Treat everything as code Operations through Git workflows DEVOXX FRANCE 2024

Platform What is it? Why is it useful? Why do we need Platform Engineers? @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

A fancy name for something already there Most companies already have some kind of platform Often homemade… @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

So many options … @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

So many options … @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Shift left and Cognitive Load @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Managing the self-service commodity @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

“The discipline of designing and building toolchains and workflows that enable self-service capabilities for software engineering organizations in the cloud-native era. Platform engineers provide an integrated product most often referred to as an “Internal Developer Platform” covering the operational necessities of the entire lifecycle of an application.” Lucas Galante @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

@Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

IDP is the new Facade Pattern @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Team Topologies “Organizations that consider establishing such a platform team should be very cautious not to accidentally create a separate DevOps team, nor should they simply relabel their existing hosting and operations structure as a platform.” TechRadar, October 2021 @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Team Topologies Stream-aligned team Enabling team Complicated Subsystem team Platform team @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Platform as a Product The Internal Dev Platform is the Product ● Conduct user research ○ Run friction logs ○ Empathy meetings ● Create a roadmap ● … @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

@Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Build your own Platform And becoming Platform Engineer @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

How to glue those all together ? @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Kubernetes Operators A Kubernetes version of the human operator @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

How to assemble all those building bricks ? @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Platform Building Frameworks Custom Resource / Promise Pipeline 2 Custom Resource X Pipeline 1 “A framework for building platforms” @Sebi2706 - @LostInBrittany Custom Resource Y DEVOXX FRANCE 2024

@Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Example: How to build an operator Because they are easier than you think… @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Example: Using several operators They are made to work together @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Example: Building a K8s-based Platform If you’re already a Kubernetes user @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024

Example: Using a PaaS provider There is a world outside Kubernetes @Sebi2706 - @LostInBrittany DEVOXX FRANCE 2024