ive
Deploying to a Service Mesh Using Istio to Simplify Kubernetes Deployments
Jesse Butler Cloud Native Advocate, Oracle Cloud Infrastructure.
cloudnative.oracle.com
@jlb13
Level Set
• Microservices • Kubernetes • Service Mesh
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
2
DevOps, Mother of Invention • Continuous Integration • Continuous Delivery • Microservices • Containers • Orchestration
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
Let’s Talk About Istio A service mesh that allows us to connect, secure, control and observe services at scale, often requiring no service code modification Though other options exist, we’ll focus on Istio
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
4
Docker • Docker changed the way we build and ship software • Application and host are decoupled, making application services portable • Containers are an implementation detail, but a critical one
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
5
Docker Is a Start
But it’s not the best at running at scale.
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
6
Docker Is a Start And once we abstract the host away by using containers, we no longer have our hands on an organized platform.
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
7
Kubernetes • Kubernetes to the rescue. • Scheduling and organization we need for deploying containers at scale • Really great abstractions of our resources and workloads
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
8
Migration from the Old World… Users
Application
Database
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
9
…to Cloud Native Kubernetes Hotness • Microservices running in orchestrated containers • Everybody’s happy • What happens now?
Load balancer Service
Service
Service
Database
Queue
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
10
Day Two
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
11
Table Stakes for Services at Cloud Scale • We require a method to simply and repeatably deploy software, and simply and recoverably modify deployments • We require telemetry, observability, and diagnosability for our software if we hope to run at cloud scale
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
12
Day 2 Solutions
• Ingress and Traffic Management • Tracing and Observability • Metrics and Analytics • Identity and Security
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
13
Abstract Requirements • Traffic Management • Observability • Security • Policy
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
14
Service Mesh for All the Things • This is not a new solution which solves all the world’s problems, but a different way to apply existing solutions • Enables integration of existing (as well as future) best-in-class solutions for All The Things
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
15
What Is a Service Mesh? • Infrastructure layer for controlling and monitoring service-to-service traffic • A data plane deployed alongside application services, creating a mesh • A control plane used to manage the mesh
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
16
Benefits of a Service Mesh • Provides DevOps teams a stable and extensible platform to monitor and maintain deployed services • Simplifies service implementation via service discovery, automated retries, circuit breaking, timeouts and more
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
17
Service Mesh is Not an API Gateway API Gateways deal with north-south traffic, inbound to your cluster Service Mesh is concerned with east-west traffic, between your services within your cluster
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
18
Back to Istio • Envoy proxy instances in the data plane to own the traffic and create the mesh • Leverages a sidecar pattern; each service added has a proxy injected into its pod • This vantage is what gives a service mesh its power, it sees and understands all
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
19
Sidecar Proxy
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
20
Sidecar Proxy
HTTP/1.1, HTTP/2 gRPC or TPC With or without mTLS
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
21
Istio Features • Traffic Management – Fine-grained control with rich routing rules, retries, failovers, and fault injection
• Observability – Automatic metrics, logs, and traces for all traffic within a cluster, including cluster ingress and egress
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
22
Istio Features • Security – Strong identity-based AuthN and AuthZ layer, secure by default for ingress, egress and service-to-service traffic
• Policy – Extensible policy engine supporting access controls, rate limits and quotas
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
23
Istio Components • Envoy – Sidecar proxy
• Pilot – Propagates rules to sidecars
• Mixer – Enforces access control, collects telemetry data
• Citadel – Service-to-service and end-user AuthN and AuthZ
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
24
Envoy High performance proxy which mediates inbound and outbound traffic.
• HTTP/2 and gRPC proxies • Dynamic service discovery • Load balancing • TLS termination • Circuit breakers • Health checks • Split traffic • Fault injection • Rich metrics
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
25
Istio Architecture
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
26
Istio Architecture
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
27
Istio Architecture
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
28
Istio Architecture
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
29
Using Istio • istioctl, cli for mesh administration • Kiali – BUI Dashboard and Control • Configure services with typical Kubernetes workflows - CRDs • Sidecar auto-injection is optional on a per-namespace basis
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
30
Demo • Integrated observability – Kiali – Grafana – Jaeger
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
31
Kubernetes Objects • Pods – Unit of deployment
• Deployments – What and how many to deploy
• Services – Abstraction of a set of pods and access to them
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
32
Istio Objects • Virtual Service – Routes mesh traffic to a destination or subset
• Destination Rules – Sets policies on routed traffic e.g TLS or LB
• Gateways – Route ingress or egress traffic to and from mesh
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
33
Traffic Shifting Basics • Identify subsets in Deployments by using labels (e.g. a subset per version) • Configure Virtual Service to route traffic based upon the subsets • Use Destination Rules to set policies related to the traffic, such as load balancing
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
34
Simple Traffic Shifting v1->v2 foo:v1
Deployment
VirtualService
Pod foo:v1
DestinationRule foo:v2
Pod foo:v2
• ‘foo’ service routed through ‘foo’
VirtualService
• DestinationRules for ‘foo:v1’ and ‘foo:v2’ Pods Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
35
Updates with Kubernetes • RollingUpdate rolls out new pods and kills off old pods, tunable with maxSurge and friends • It does this by thumping them on the head with SIGTERM • And continues to throw work at them • And throws work at new pods before they are ready • And the LoadBalancer configuration is updated concurrently or slightly later and yep this isn’t so hot
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
36
Updates with Kubernetes The Right Way • Implement Readiness Probes • Ensure everyone can handle a SIGTERM • Use a preStop lifecycle hook • Automate all of this and don’t break it
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
37
Simplifying CD Life with Istio • Create a Service, and a Deployment for each version of software • Use version labels to create subsets for each of the desired versions • Set up DestinationRules to route traffic to the each of the versions • Route the traffic based upon needs, using destination subset in VirtualService
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
38
Demo • Simple Traffic Shifting – Migrate traffic from v1 to v2
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
39
Leveraging the Pattern • We can manage traffic in an informed way • We can take advantage of zero-downtime changes in routing between versions • We can automate deployments of any kind – Canary deployments – Blue/Green deployments – Whatever we want
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
40
Demo • Blue/Green Deployment – Move traffic from v1 to v2 gradually using weights
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
41
A Full Solution: Flagger • Kubernetes operator that automates promotion of canary deployments • Uses Istio for traffic shifting between service versions • Uses Prometheus metrics for canary analysis • Extension via webhooks for additional acceptance test, load test, etc
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
42
Demo • Canary Deployments – Automated Canary Deployments with Flagger
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
43
Test in Prod with Dark Launches • Istio offers Traffic Mirroring on a weighted basis as well, providing for Dark Launches • An under-test version of a service can be deployed and production traffic routed to it • Testing in production with production data
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
44
Demo • Dark Launch – Use Traffic Mirroring for a Dark Launch
Copyright © 2019, Oracle and/or its affiliates. All rights reserved.
@jlb13
45
Thanks!
Twitter: @jlb13 cloudnative.oracle.com cloud.oracle.com/tryit Sign-up available for $500 trial (for a start)
ive
Deploying to a Service Mesh Using Istio to Simplify Kubernetes Deployments
Jesse Butler Cloud Native Advocate, Oracle Cloud Infrastructure.
cloudnative.oracle.com
@jlb13