A presentation at Continuous Delivery NYC Meetup in in New York, NY, USA by Jesse Butler
ive Deploying to a Service Mesh Using Istio to Simplify Kubernetes Deployments Jesse Butler Cloud Native Advocate, Oracle Cloud Infrastructure. cloudnative.oracle.com @jlb13
Level Set • Microservices • Kubernetes • Service Mesh Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 2
DevOps, Mother of Invention • Continuous Integration • Continuous Delivery • Microservices • Containers • Orchestration Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13
Let’s Talk About Istio A service mesh that allows us to connect, secure, control and observe services at scale, often requiring no service code modification Though other options exist, we’ll focus on Istio Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 4
Docker • Docker changed the way we build and ship software • Application and host are decoupled, making application services portable • Containers are an implementation detail, but a critical one Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 5
Docker Is a Start But it’s not the best at running at scale. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 6
Docker Is a Start And once we abstract the host away by using containers, we no longer have our hands on an organized platform. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 7
Kubernetes • Kubernetes to the rescue. • Scheduling and organization we need for deploying containers at scale • Really great abstractions of our resources and workloads Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 8
Migration from the Old World… Users Application Database Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 9
…to Cloud Native Kubernetes Hotness • Microservices running in orchestrated containers • Everybody’s happy • What happens now? Load balancer Service Service Service Database Queue Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 10
Day Two Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 11
Table Stakes for Services at Cloud Scale • We require a method to simply and repeatably deploy software, and simply and recoverably modify deployments • We require telemetry, observability, and diagnosability for our software if we hope to run at cloud scale Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 12
Day 2 Solutions • Ingress and Traffic Management • Tracing and Observability • Metrics and Analytics • Identity and Security Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 13
Abstract Requirements • Traffic Management • Observability • Security • Policy Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 14
Service Mesh for All the Things • This is not a new solution which solves all the world’s problems, but a different way to apply existing solutions • Enables integration of existing (as well as future) best-in-class solutions for All The Things Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 15
What Is a Service Mesh? • Infrastructure layer for controlling and monitoring service-to-service traffic • A data plane deployed alongside application services, creating a mesh • A control plane used to manage the mesh Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 16
Benefits of a Service Mesh • Provides DevOps teams a stable and extensible platform to monitor and maintain deployed services • Simplifies service implementation via service discovery, automated retries, circuit breaking, timeouts and more Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 17
Service Mesh is Not an API Gateway API Gateways deal with north-south traffic, inbound to your cluster Service Mesh is concerned with east-west traffic, between your services within your cluster Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 18
Back to Istio • Envoy proxy instances in the data plane to own the traffic and create the mesh • Leverages a sidecar pattern; each service added has a proxy injected into its pod • This vantage is what gives a service mesh its power, it sees and understands all Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 19
Sidecar Proxy Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 20
Sidecar Proxy HTTP/1.1, HTTP/2 gRPC or TPC With or without mTLS Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 21
Istio Features • Traffic Management – Fine-grained control with rich routing rules, retries, failovers, and fault injection • Observability – Automatic metrics, logs, and traces for all traffic within a cluster, including cluster ingress and egress Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 22
Istio Features • Security – Strong identity-based AuthN and AuthZ layer, secure by default for ingress, egress and service-to-service traffic • Policy – Extensible policy engine supporting access controls, rate limits and quotas Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 23
Istio Components • Envoy – Sidecar proxy • Pilot – Propagates rules to sidecars • Mixer – Enforces access control, collects telemetry data • Citadel – Service-to-service and end-user AuthN and AuthZ Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 24
Envoy High performance proxy which mediates inbound and outbound traffic. • HTTP/2 and gRPC proxies • Dynamic service discovery • Load balancing • TLS termination • Circuit breakers • Health checks • Split traffic • Fault injection • Rich metrics Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 25
Istio Architecture Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 26
Istio Architecture Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 27
Istio Architecture Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 28
Istio Architecture Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 29
Using Istio • istioctl, cli for mesh administration • Kiali – BUI Dashboard and Control • Configure services with typical Kubernetes workflows - CRDs • Sidecar auto-injection is optional on a per-namespace basis Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 30
Demo • Integrated observability – Kiali – Grafana – Jaeger Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 31
Kubernetes Objects • Pods – Unit of deployment • Deployments – What and how many to deploy • Services – Abstraction of a set of pods and access to them Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 32
Istio Objects • Virtual Service – Routes mesh traffic to a destination or subset • Destination Rules – Sets policies on routed traffic e.g TLS or LB • Gateways – Route ingress or egress traffic to and from mesh Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 33
Traffic Shifting Basics • Identify subsets in Deployments by using labels (e.g. a subset per version) • Configure Virtual Service to route traffic based upon the subsets • Use Destination Rules to set policies related to the traffic, such as load balancing Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 34
Simple Traffic Shifting v1->v2 foo:v1 Deployment VirtualService Pod foo:v1 DestinationRule foo:v2 Pod foo:v2 • ‘foo’ service routed through ‘foo’ VirtualService • DestinationRules for ‘foo:v1’ and ‘foo:v2’ Pods Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 35
Updates with Kubernetes • RollingUpdate rolls out new pods and kills off old pods, tunable with maxSurge and friends • It does this by thumping them on the head with SIGTERM • And continues to throw work at them • And throws work at new pods before they are ready • And the LoadBalancer configuration is updated concurrently or slightly later and yep this isn’t so hot Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 36
Updates with Kubernetes The Right Way • Implement Readiness Probes • Ensure everyone can handle a SIGTERM • Use a preStop lifecycle hook • Automate all of this and don’t break it Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 37
Simplifying CD Life with Istio • Create a Service, and a Deployment for each version of software • Use version labels to create subsets for each of the desired versions • Set up DestinationRules to route traffic to the each of the versions • Route the traffic based upon needs, using destination subset in VirtualService Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 38
Demo • Simple Traffic Shifting – Migrate traffic from v1 to v2 Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 39
Leveraging the Pattern • We can manage traffic in an informed way • We can take advantage of zero-downtime changes in routing between versions • We can automate deployments of any kind – Canary deployments – Blue/Green deployments – Whatever we want Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 40
Demo • Blue/Green Deployment – Move traffic from v1 to v2 gradually using weights Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 41
A Full Solution: Flagger • Kubernetes operator that automates promotion of canary deployments • Uses Istio for traffic shifting between service versions • Uses Prometheus metrics for canary analysis • Extension via webhooks for additional acceptance test, load test, etc Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 42
Demo • Canary Deployments – Automated Canary Deployments with Flagger Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 43
Test in Prod with Dark Launches • Istio offers Traffic Mirroring on a weighted basis as well, providing for Dark Launches • An under-test version of a service can be deployed and production traffic routed to it • Testing in production with production data Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 44
Demo • Dark Launch – Use Traffic Mirroring for a Dark Launch Copyright © 2019, Oracle and/or its affiliates. All rights reserved. @jlb13 45
Thanks! Twitter: @jlb13 cloudnative.oracle.com cloud.oracle.com/tryit Sign-up available for $500 trial (for a start)
Using Istio to Simplify Kubernetes Deployments
A service mesh is a configurable infrastructure component that’s aware of the internal state of our cluster. It’s architecture provides us with very powerful abstractions which can then be built upon. One of the core features of a service mesh is to simplify traffic management, and this can be leveraged to simplify common deployment actions. We’ll see how Istio and Linkerd can help simplify canary deployments, blue/green deployments and A/B testing. We’ll also touch on testing, tracing and dark launches.
for free. You
can too.