There’s No Such Thing as Vanilla Kubernetes
A presentation at OpenShift Commons Briefing by Fen Aldrich
There is no such thing as vanilla Kubernetes Aaron Aldrich Managed OpenShift Black Belt 1
whoami DevOpsDays Hartford | NYC | Boston OSMIhelp.org Aaron Aldrich Managed OpenShift Black Belt @CrayZeigh 2 @TabletopDevOps twitch.tv/desertedislandtv actually my license plate
What is vanilla Kubernetes? 3
Look! It’s Vanilla Kubernetes!
What is vanilla kubernetes? Why bother? the myths of vanilla kubernetes Portability No “lock-in” Latest features 5
There are a LOT of different components just to get the control plane operational
What is vanilla kubernetes? Kubernetes requires a collection of services Control Plane Services: Node Services: ▸ kube-apiserver ▸ kubelet ▸ etcd ▸ kube-proxy ▸ kube-scheduler ▸ container runtime ▸ kube-controller-manager ▸ cloud-controller-manager* Add-Ons: ▸ DNS Services (required) ▸ Resource Monitoring** ▸ Logging*** 6
- Optional, but required if hosting with any CSP ** Optional, but required for horizontal-autoscaling *** Optional, but not really
Plus a whole bunch of extra services for practical application
Vanilla Kubernetes in practice Kubernetes in Practice ▸ Application Logs ▸ Infrastructure Management ▸ Application Metrics ▸ CI/CD ▸ Storage ▸ Container Registry ▸ Network ▸ Dependency Scanning ▸ Load Balancing ▸ Operating System ▸ Egress Routing ▸ Container Runtime ▸ Additional Authentication Providers ▸ Infrastructure ▸ Infrastructure Automation 7
Even with the underlying runtime a choice has to be made
Vanilla Kubernetes in practice 8 Container runtimes
And now docker is going away
Vanilla Kubernetes in practice 9 Container runtimes
Or that’s being pushed back a bit and really it’s not that big of a change…
Vanilla Kubernetes in practice 10 Container runtimes
So all of the purported benefits of “Vanilla” go away pretty quick once you look at practical implementation
Vanilla Kubernetes in practice Vanilla Kubernetes Reality Custom tuned infra Locked-in to custom build Full-stack maintenance 11
This is the real argument we’re having
Build v. Buy 12
This feels like it should be settled case law by now Always having this argument, especially in tech so lets look at it as it pertains to operational choices
Build v. Buy The Case of Build v. Buy Build Buy 13
Build
- Ultimate freedom, but with responsibility
- Free…. as in puppies
Build v Buy 14 Photo by Avel Chuklanov on Unsplash
Buy
- Opinionated, but supported by experts
- Expensive, but fast to implement
- Buying expertise
Build v Buy 15 Photo by Dillon Kydd on Unsplash
Build v. Buy How do you make a choice? min/maxing our business decisions
- Workaround Resource Constraints - Spend Money on implementation details - Spend Time on Market Differentiation 16 Photo by Jason Dent on Unsplash
Spoiler: You should probably buy it*
DataDog* is a good example of a company who made the right choice building it themselves and why that’s A Thing that might be good under certain circumstances. ** You are not DataDog.
Build v. Buy 17 “10 ways to shoot yourself in the foot with Kubernetes, number 9 will surprise you” - Laurent Bernaille, Robert Boll, Datadog ”Kubernetes the very hard way” - Robert Boll, Datadog
So What’s the alternative?
Kubernetes Distributions 18
Certification standards to ensure interoperability
Kubernetes distributions 19
Kubernetes distributions Kubernetes Conformance 20
Of course they’re all conformant
Kubernetes distributions Kubernetes Conformance 21
This is the actual bit that we’re talking about certifying, This is Kubernetes
Kubernetes distributions Upstream Kubernetes Kubernetes Ingress Kube Native App Helmed App Kubernetes CLI Kubernetes API Kubernetes 22
This is how a distribution adds on, abstractions and implementation details
Kubernetes distributions Amazon Elastic Kubernetes Service (EKS) Kubernetes Ingress Kube Native App Elastic Load Balancer S3 Helmed App Cloudwatch CodeStar Kubernetes CLI Container Registry Kubernetes API AWS Console Other AWS Services Kubernetes Amazon VPC EC2/Fargate Amazon Web Services 23 *not in anyway comprehensive Productive Abstractions
Even GKE, made by the originator of the project, doesn’t do vanilla kubernetes, it’s a distro, with implementation details and productive abstractions
Kubernetes distributions Google Kubernetes Engine (GKE) Kubernetes Ingress Kube Native App GCP Load Balancer Cloud Storage Helmed App Stackdriver Cloud Build Kubernetes CLI Container Registry Kubernetes API GCloud Console GCP Services and other Googly Things Kubernetes GCP VPC Container-Optimized OS GCloud 24 *not in anyway comprehensive Productive Abstractions
OpenShift is a little different being a full app platform built with and using k8s, but same idea Some of it is opinionated implementation details, the rest is expansion options
Kubernetes distributions Red Hat OpenShift OpenShift Routes OpenShift Mesh/Istio Kubernetes Ingress Kube Native App Operated/Helmed App Knative OpenShift Native App RHT Storage Che via CReW Telemetry (EFK/Profana) Templates/S2I/BC Red Hat MW Kubernetes CLI Quay OpenShift CLI Admin Console Kubernetes API OpenShift API OpenShift Kubernetes OVN RHEL Core OS Ansible/Terraform/Operators 25 *not in anyway comprehensive Productive Abstractions
So if they’re all certified fresh, we’re choosing implementation. OpenShift is a little different beast as a full platform Other two are more than a distro: control plane but also managed service for control plane. Gets into looking at the future…
Kubernetes distributions 26
Let’s do some future predictions based on the landscape so far and what we’re seeing
🔮🗺 27
SaaS and Moving Downmarket
MongoDB shifting from a few big customers buying on-prem, hard to implement software to easy to consume, big on end users Fewer big Customers, extremely long tail Makes sense, direction of evolution
Evolution and the ease of comsumption 28 https://www.moritzplassnig.com/saas-and-moving-downmarket-mongodbs-transformation/
Some brief tie in with wardley mapping Utility targets ease of consumption
Currently hybriding between product and utility, build v buy…. Rental… utility
Evolution and the ease of consumption 29 https://www.abusedbits.com/2018/04/descriptive-evolution-mapping.html
It’s OPS after all. Make the repeatable, boring stuff easy and automatic. Serverless is empowering code to run without worrying about implementation details: getting ideas into production faster
DevOps is about encoding knowledge in practice to help build on the shoulders of giants.
Evolution and the ease of consumption “Serverless is the target of every DevOps project” ~Andrew Clay Shafer, Map Camp, DevOps/Serverless Days 30
A platform for platforms 31
An icecream platform Kubernetes isn’t just an ice cream 32 Photo by Dovile Ramoskaite on Unsplash
A platform for building ice cream sundaes Kubernetes is part of a sundae 33
📽 Thank you speaking.crayzeigh.com linkedin.com/company/red-hat youtube.com/user/RedHatVideos Red Hat is the world’s leading provider of enterprise open source software solutions. Award-winning facebook.com/redhatinc support, training, and consulting services make Red Hat a trusted adviser to the Fortune 500. 34 twitter.com/RedHat
There’s no such thing as vanilla Kubernetes in the same vein that there is no vanilla Linux; even if you start from source, you must custom build everything around it to make it production ready. By looking at the past and the present, this talk will explore what it takes to build and operate Kubernetes in production, what alternatives you can buy, and extrapolates on industry trends towards “utility” services to explore what the future of Kubernetes might look like.
Video
Resources
The following resources were mentioned during the presentation or are useful additional information.
-
10 Ways to Shoot Yourself in the Foot with Kubernetes, Number 9 Will Surprise You
Talk from Datadog folks at Kubecon about implementing kubernetes and the challenges they have faced with extremely large scale implementation.
-
Kubernetes the Very Hard Way
Additional Datadog talk about how hard it was to implement Kubernetes early on
-
SaaS and Moving Downmarket: MongoDBs Transformation
-
Wardley Maps (Wikipedia)
Background on what is a Wardley Map
-
Learn Wardley Mapping
Maybe a more practical introduction from Ben Mosior (@hiredthought)
-
Descriptive Evolution Mapping
Where that Map comes from describing the evolution process and differences of Built for Purpose, built for manufacture and Built for operationalization
