A presentation at Agile India 2020 by Karthik Gaekwad
Kube Me This! Kubernetes best practices Karthik Gaekwad @iteration1 Head of Cloud Native Engineering
Hello • I’m Karthik Gaekwad • Head of Cloud Native Engineering @iteration1
Hello I’m Karthik Gaekwad Oracle Cloud: Developer on the Managed Kubernetes Team + Developer Relations Author of devops and Kubernetes courses on LinkedIn Learning Super popular helloworld docker container @iteration1
Chaos Engineering Want to learn more? Get the book in 5 minutes! verica.io/book Confidential – Oracle Internal/Restricted/Highly Restricted 4 https://verica.io/book/
Today we will We’re going to talk about Kubernetes… Break this down into 3 pillars… • Development and Architecture • Devops • Enterprise Transformation 5 @iteration1
Development & Architecture 6 @iteration1
7 @iteration1
Microservices Design • Start with Twelve-Factor App design • https://12factor.net • Based on the principals of software design and deployment at Heroku • Development best practice that synergizes with devops engineers 8 @iteration1
Kubernetes Design Patterns 10 @iteration1
Kubernetes Deployments What?? • Most common K8s object that is used for applications running in Kubernetes. • Deployment is a defined specification that is used to create replica sets and associated pods. 11 @iteration1
Kubernetes Deployments “I’m converting an application (monolith) to a Kubernetes based architecture, what should my deployment look like?” 12 @iteration1
Kubernetes Deployments “I’m converting an application (monolith) to a Kubernetes based architecture, what should my deployment look like?” 2 Choices: Single deployment model Multi deployment model 13 @iteration1
Kubernetes Deployments Single Multi • One single object for your whole application, backed by multiple pods behind the scenes. • Think of this like one Java WAR file for all parts of your application. • Multiple independent deployments for a larger application. • Components must work together (microservices based architecture) 14 @iteration1 • This is like having many WAR files for your app.
Working with multiple deployments Sometimes this can get hairy…. 18 @iteration1
Working with multiple deployments • Consider using liveness and readiness probes. • Readiness probe • User defined health check that tells Kubernetes when the container is ready to serve request. • K8s will route traffic to it once it’s “ready” • Liveness probe: • User defined health check to indicate whether a container is running. • If probe fails, K8s will kill the container and spawn a new one based on the restart policy • Read more: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#containerprobes 19 @iteration1
Working with multiple deployments • Consider using version endpoints for your pods/containers. • Can be any defined version strategy= git hash or a user defined version. • Assists with identification of what is actually running, and reference back to the source code. • Allows easier debugging, especially when there are multiple teams working on a single deployment, with frequent independent releases. 20 @iteration1
Authentication and Authorization 21 @iteration1
Authentication and Authorization • Problem: The concept of a user doesn’t exist in K8s. You have to self manage.. • Do you know how you are authenticating with Kubernetes? • Many ways to Authenticate • • • • • • Client Certs Static token file Service Account tokens OpenID Webhook Mode And more (https://kubernetes.io/docs/reference/access-authn-authz/authentication/) @iteration1
Goal: Pick a strategy that fits your use case Whatever you do, DO NOT YOLO!
If you DO NOT YOLO… You can pick an authz strategy..
Authentication and Authorization https://kubernetes.io/docs/reference/access-authnauthz/authorization/ @iteration1
Authentication and Authorization • Pro tip: Nobody uses ABAC anymore. Don’t be that guy…. • RBAC is the defacto standard • Based on roles and role bindings • Good set of defaults: https://github.com/uruddarraju/kubernetes-rbac-policies • Can use multiple authorizers together, but can get confusing. • 1st authorizer to authorize passes authz @iteration1
Logging and Monitoring 27
Logging and Monitoring • kubectl logs and tail commands only takes you so far… • Invest in a logging and monitoring strategy early on before you go to production. • Gives engineers the expertise to debug and monitor applications. 28 @iteration1
More time up front to play with tooling == Less time learning tooling during prod issues 29 @iteration1
Logging and Monitoring • Your existing tooling most likely plays well with Kubernetes. • Open source is a viable option as well. • CNCF ecosystem: • EFK stack for logging • Prometheus and Grafana for monitoring 30 @iteration1
Containers… 31 Confidential – Oracle Internal/Restricted/Highly Restricted
Container Image Best Practices • Image Sizes GOAL: Smaller the image, the better • Less things for an attacker to exploit. • Quicker to push, quicker to pull. 32 @iteration1
Container Image Best Practices GOAL: Don’t rely on :latest tag • :latest image yesterday might not be :latest image tomorrow • Instead, you’d want to know what specific version you’re operating with. 33 @iteration1
Container Image Best Practices GOAL: Consider using a private registry Enterprise concerns for data storage. Registry physically closer to your Kubernetes cluster Quicker image pulls = faster deployments to Kubernetes Consider using the your cloud provider for the registry • • • • 34 @iteration1
Devops 35
Managed Kubernetes Services Should I install my own, or use a managed service? 36 @iteration1
Managed Kubernetes Services Pros: Offload control plane management to the provider. Less maintenance headache. Spend time working on your apps, and required infrastructure. Cons: Not 100% customizable. Hidden costs. General guidance: Use it, unless you have a non standard usecase. 37 @iteration1
Cluster Management Dev/Test/Production clusters strategies? 38
Cluster Management Strategies • Two primary strategies in play: • Utilize different namespaces in single cluster • Utilize different clusters for dev/test/prod 39 @iteration1
Cluster Management Strategies (Namespaces) • Single cluster, with multiple namespaces • “dev/test/prod” Access control via kubeconfig to only have rights to a single namespace. Typically used in startups or companies with smaller ops teams. Pro: single cluster, so lesser management. Con: cluster issues will cause all environments to experience issues. Read more: https://kubernetes.io/blog/2016/08/kubernetes-namespaces-use-cases-insights/ • • • • • 40 @iteration1
Cluster Management Strategies (Separate clusters) • Multiple unique clusters for separation of concerns • Unique dev, test and production clusters • Easier to implement with managed services (one click) • Access control can be implemented in cloud layer, flows down to kubeconfig file. • Recommended approach for enterprises. • Cons: • More environments to manage 41 @iteration1
Tagging nodes 42 @iteration1
Tag your nodes • Multiple clusters = multiple nodes. • Be diligent about labeling nodes on creation so that you have better control over your cloud infrastructure. • Kubernetes has concept of labels, use it!! • https://kubernetes.io/docs/concepts/overview/working-withobjects/labels/ 43 @iteration1
Pipelines 44 @iteration1
Pipelines K8s allows you to build a devops pipeline. 1. 2. 3. 4. Build artifacts Test code Push to registry Optionally Deploy to K8s Accomplish with CI/CD Tooling Chance to modernize infra if you haven’t already 45 @iteration1
Cloud Native Enterprise Transformation 46 @iteration1
How to start? Where to start? 47 @iteration1
5 easy steps! Start small… Step 1: Get experience with K8s clusters. Step 2: Take one application or microservice and convert to K8s. Step 3: Run this application in a production setting. Step 4: Understand how to manage and firefight. Step 5: Goto step 2. 48 @iteration1
Know your teams Every organization is different. Build your cloud native transformation around your teams. Need equal expertise in development, operations and firefighting. Organize into development, devops and SRE teams. • • • • • Leverage OCI + Opensource technologies. • Watch CNCF space because landscape changes (https://www.cncf.io/) 49 @iteration1
KEEP CALM AND KUBE ON @iteration1
Kubernetes is a force to be reckoned with for enterprises, and there’s been a major shift to move their infrastructure to use Kubernetes. This session covers best practices you’ll want to keep in mind when making a shift from deploying applications to web servers and moving to a microservices model and Kubernetes. You’ll learn about topics you should consider while moving to Kubernetes and principles you should follow when building out your Kubernetes-based applications or infrastructure. The ideas presented are from war stories and experiences of working in the trenches with Kubernetes for the managed service for Oracle Cloud.
for free. You
can too.