A presentation at Oracle OpenWorld in in London, UK by Jesse Butler
Microservices at Scale Next Steps with Kubernetes and Istio Jesse Butler Cloud Native Advocate Oracle Cloud Native Labs January 16, 2019 Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted
Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, timing, and pricing of any features or functionality described for Oracle’s products may change and remains at the sole discretion of Oracle Corporation. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 3
About Me • • • • Oracle via Sun Microsystems Responsible for Docker on Solaris, later on Oracle Linux Some work with Open Containers and CNCF WGs Now a Cloud Native Advocate @ Oracle Cloud • @jlb13 on Twitter Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 4
About OCI • • • Next-generation Cloud Infrastructure Highly performant, very affordable Managed Cloud Native Services • Check out OCI: https://cloud.oracle.com/tryit – OKE & OCIR at the core – Many managed services in the pipeline Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 5
Level Set • Containers in Production • Microservices • Docker, Kubernetes Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 6
Monolithic Applications Users Application Database Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 7
Monolithic Applications Users Application Database Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 8
Microservices • Microservices are the de facto standard for cloud native software • Microservices allow development teams to deploy portable and scalable applications Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 9
Microservices Users Cart Orders Reports Database Cluster Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 10
Microservices • Microservices can put a significant burden on Ops and DevOps teams Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 11
Microservices • Or, put another way Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 12
Let’s Talk About Istio Istio a service mesh that allows us to connect, secure, control and observe services at scale, often requiring no service code modification Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 13
The Old World • Once upon a time, proprietary systems and software were bundled and sold as a unit • This created independent silos per vendor, each with ecosystems of tools and service vendors • Systems analysts surfaced system data and implemented improvements Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 14
More Recent History • There were a lot of moving parts in the typical Old World IT organization • The advent of web applications made time to market a far more sensitive metric • DevOps arose as a means of reducing friction between where software is created and where it is deployed Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 15
Advent of DevOps • DevOps brings the concerns of development and operations closer together • Ideally we preserve meaningful historical expertise from both high level disciplines • DevOps is as much a cultural shift as it is technical Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 16
DevOps, Mother of Invention • Microservices • CI / CD • Cloud Adoption • Containers Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 17
Docker • Docker changed the way we build and ship software • Application and host are decoupled, making application services portable • Containers are an implementation detail, but a critical one Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 18
Docker Is a Start But, once we abstract the host away by using containers, we no longer have our hands on an organized platform. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 19
Kubernetes Kubernetes provides abstractions for deploying software in containers at scale Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 20
Kubernetes as a Platform • Infrastructure resource abstraction • Cluster software where one or more masters control worker nodes • Scheduler deploys work to the nodes • Work is deployed in groups of containers Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 21
Migration from the Old World… Users Application Database Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 22
…to Cloud Native Kubernetes Hotness • Microservices running in orchestrated containers • Everybody’s happy • What happens now? Load balancer Service Service Service Database Queue Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 23
…to Cloud Native Kubernetes Hotness • Microservices running in orchestrated containers • Everybody’s happy Load balancer Service Service Service Service Database Queue • What happens now? Service Service Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 24
Day Two Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 25
Table Stakes for Services at Cloud Scale • We require a method to simply and repeatably deploy software, and simply and recoverably modify deployments • We require telemetry, observability, and diagnosability for our software if we hope to run at cloud scale Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 26
Day 2 Solutions • Ingress and Traffic Management • Tracing and Observability • Metrics and Analytics • Identity and Security Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 27
Abstract Requirements • Traffic Management • Observability • Security • Policy Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 28
Hard Things are Hard These are Hard Problems™, and some software may address one of them well. Service mesh addresses them all. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 29
What Is a Service Mesh? • Infrastructure layer for controlling and monitoring service-to-service traffic • A data plane deployed alongside application services, and a control plane used to manage the mesh Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 30
Service Mesh • Provides DevOps teams a stable and extensible platform to monitor and maintain deployed services • For the most part, invisible to development teams Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 31
Service Mesh • This is not a new solution which solves all the world’s problems, but a different way to apply existing solutions • Enables integration of existing (as well as future) best-in-class solutions for All The Things Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 32
Let’s Get Back To Istio Istio a service mesh that allows us to connect, secure, control and observe services at scale, often requiring no service code modification. Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 33
Istio Features • Traffic Management – Fine-grained control with rich routing rules, retries, failovers, and fault injection • Observability – Automatic metrics, logs, and traces for all traffic within a cluster, including cluster ingress and egress Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 34
Istio Features • Security – Strong identity-based AuthN and AuthZ layer, secure by default for ingress, egress and service-to-service traffic • Policy – Extensible policy engine supporting access controls, rate limits and quotas Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 35
Sidecar Proxy Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 36
Sidecar Proxy Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 37
Envoy High performance proxy which mediates inbound and outbound traffic. • Dynamic service discovery • Load balancing • TLS termination • HTTP/2 and gRPC proxies • Circuit breakers • Health checks • Split traffic • Fault injection • Rich metrics Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 38
Istio Components • Envoy – Sidecar proxy • Pilot – Propagates rules to sidecars • Mixer – Enforces access control, collects telemetry data • Citadel – Service-to-service and end-user AuthN and AuthZ Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 39
Istio Architecture Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 40
Istio Architecture Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 41
Istio Architecture Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 42
Istio Architecture Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 43
Telemetry • Istio’s Mixer is stateless and does not manage any persistent storage of its own • Capable of accumulating a large amount of transient ephemeral state • Designed to be a highly reliable, goal is > 99.999% uptime for any individual instance • Many adapters available: Prometheus, Cloud providers, Datadog, Solarwinds… Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 44
Traffic Management • Integrated Ingress and Egress • Error handling, retries, circuit breaking • Application knowledge can be leveraged for intelligent routing • Fault injection for end-to-end testing Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 45
Performance and Scalability • Code level micro-benchmarks • Synthetic end-to-end benchmarks across various scenarios • Realistic complex app end-to-end benchmarks across various settings • Automation to ensure performance doesn’t regress Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 46
Security • Traffic encryption to defend against the manin-the-middle attacks • Mutual TLS and fine-grained access policies to provide flexible access control • Auditing tools to monitor all of it Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 47
Service Mesh Adoption • Service mesh provides features that make life easier for DevOps and Ops teams • Benefits are becoming apparent to developers, simplified services allowing the mesh to take are of things like retries, circuit breakers, etc • Istio is a great place to start • https://cloudnative.oracle.com/learn.html has an Istio 101 Tutorial if you are interested! Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 48
Questions? Thank you! Check out OCI: https://cloud.oracle.com/tryit Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted
• Backup Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 50
Grafana Istio Mesh Dashboard Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 51
Service Dashboard Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 52
Workload Dashboard Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 53
Inbound Workload Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 54
Outbound Workload Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 55
Service Tracing Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 56
Service Drilldown Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 57
Service Graph Copyright © 2019, Oracle and/or its affiliates. All rights reserved. | Confidential – Oracle Internal/Restricted/Highly Restricted 58
Microservices are the de facto standard architecture of cloud native software. With Docker and Kubernetes, it’s fairly simple to get a few services up and running. When this scales up to hundreds or thousands of containers and services, things can get complicated. Istio provides solutions within the service mesh, offering load balancing, traffic control, policy management features and easily enabling best-in-class observability, metrics and service graphing solutions. In this session, we’ll take a tour of Istio and discuss how its rich feature set helps us run cloud native at scale.
for free. You
can too.