Using Spinnaker to Create a Development Workflow on Kubernetes

A presentation at Spring One Tour - Singapore in November 2018 in Singapore by Paul Czarkowski

Slide 1

Slide 1

Continuous Deployment With Spinnaker & Kubernetes Paul Czarkowski Twitter: @pczarkowski © Copyright 2018 Pivotal Software, Inc. All rights Reserved. Version 1.0

Slide 2

Slide 2

Continuous Deployment With Spinnaker & Kubernetes Paul Czarkowski Twitter: @pczarkowski © Copyright 2018 Pivotal Software, Inc. All rights Reserved. Version 1.0

Slide 3

Slide 3

Agenda ■ Who am I ? ■ Pivotal Container Service (PKS) ■ Kubernetes 101 ■ Kubernetes Manifests ■ Helm Charts ■ Spinnaker ■ Q+A Cover w/ Image

Slide 4

Slide 4

Pivotal Container Service

Slide 5

Slide 5

WE build the container YOU build the container Application Code & Frameworks Continuous delivery Github Pivotal Network Concourse Pivotal Application Service (PAS) Pivotal Container Service (PKS)

cf push kubectl run Java | .NET | NodeJS Product Updates Marketplace Pivotal and Partner Products v1 CVEs Pivotal Services Open Service Broker API Buildpacks | Spring Boot | Spring Cloud | Steeltoe Public Cloud Services Customer Managed Services Elastic | Packaged Software | Spark v2 “3Rs” v3 Embedded OS ... NSX-T (Windows & Linux) Concourse CPI (15 methods) vSphere Openstack AWS Google Cloud Azure & Azure Stack Repair — CVEs Repave Rotate — Credhub

Slide 6

Slide 6

Enterprise-Grade Kubernetes GCP Service Broker Built with open-source Kubernetes Use the PKS CLI and API to create, operate, and scale your clusters. GCP Service Broker NSX-T NSX-T PKS Control Plane Kubernetes PKS Control Plane Constant compatibility with the latest stable release of Google Kubernetes Engine—no proprietary extensions. Bring Google Cloud Platform services to your containers. Extend your workloads with machine learning, AI, and data services from Google. K8s Cluster K8s Cluster K8s Cluster Network management, security, and load balancing out-of-the-box with VMware NSX-T. Multi-cloud, multi-hypervisor. Harbor Harbor An enterprise-class container registry. Includes vulnerability scanning, identity management, and more. BOSH VMware GCP Azure Openstack BOSH Reliable and consistent operational experience for any cloud. AWS

Slide 7

Slide 7

Slide 8

Slide 8

App / Container Deployment, Services & Routing App kubectl run cf push Pivotal Cloud Foundry Elastic Runtime myapp.mydomain.net Pivotal Container Service ???? ????

Slide 9

Slide 9

Slide 10

Slide 10

App / Container Deployment, Services & Routing docker build docker push kubectl run kubectl expose App cf push Pivotal Cloud Foundry Elastic Runtime myapp.mydomain.net Pivotal Container Service ???? ????

Slide 11

Slide 11

Slide 12

Slide 12

Kubernetes 101

Slide 13

Slide 13

Logical Kubernetes Architecture K8s Master Controller Manager Etcd Kube Scheduler API Server K8s Worker K8s Worker K8s Worker Kubelet Kubelet Pod Pod Pod Docker Kubelet Pod Pod Pod Docker Kube-proxy Kube-proxy CNI Pod Pod Pod Docker Kube-proxy CNI CNI

Slide 14

Slide 14

$ kubectl --help kubectl controls the Kubernetes cluster manager. Find more information at: https://kubernetes.io/docs/reference/kubectl/overview/ Basic Commands (Beginner): create Create a resource from a file or from stdin. expose Take a replication controller, service, deployment or pod and expose it as a new Kubernetes Service run Run a particular image on the cluster set Set specific features on objects

Slide 15

Slide 15

Kubernetes Manifest apiVersion: kind: metadata: spec:

Slide 16

Slide 16

Kubernetes Manifest http 80 apiVersion: apps/v1beta1 apiVersion: v1 kind: Deployment kind: Service metadata: metadata: labels: app: hello-world name: hello-app Load Balancer name: hello-svc spec: hello-svc Service ports: app=hello-world

  • port: 80 spec: replicas: 2 protocol: TCP template: targetPort: 8080 metadata: labels: app: hello-world spec: containers: - image: paulczar/hello-world name: hello-world selector: app: hello-world type: LoadBalancer http 8080 - load balanced hello-app Pod hello-app Pod app=hello-world app=hello-world

Slide 17

Slide 17

POD

Slide 18

Slide 18

$ kubectl run hello \ --image=paulczar/go-hello-world

Slide 19

Slide 19

● kubectl run created a deployment “deployments.apps/hello” NAME deployment.apps/hello ● DESIRED 1 CURRENT 1 AVAILABLE 1 AGE 1m The deployment created a replicaset “replicaset.apps/hello-64f6bf9dd4” NAME replicaset.apps/hello-64f6bf9dd4 ● UP-TO-DATE 1 DESIRED 1 CURRENT 1 READY 1 AGE 1m Which created a pod “pod/hello-64f6bf9dd4-tq5dq” NAME pod/hello-64f6bf9dd4-tq5dq READY 1/1 STATUS Running RESTARTS 0 AGE 2s

Slide 20

Slide 20

Pod Container Container Pod one or more application containers that are tightly coupled, sharing network and storage. Example: a web front-end Pod that consists of an NGINX container and a PHP-FPM container with a shared unix socket and a “init” container to transform their config files based on environment variables.

Slide 21

Slide 21

Container Container Container Container Pod ReplicaSet Pod ReplicaSet Extends Pod resource to run and maintain a specific number of copies of a pod.

Slide 22

Slide 22

Deployment Container Container Container Container Pod ReplicaSet Deployment Pod a controller that ensures a set number of replicas of a Pod is running and provides update and upgrade workflows for your Pods. Example: cloud native Node app that scales horizontally and upgrades 2 pods at a time.

Slide 23

Slide 23

statefulset Container Container Container Container Pod ReplicaSet Deployment Statefulset Pod a controller that manages stateful application Deployments by providing sticky identity for pods and strict ordering and uniqueness. Example: Cassandra database. First pod is ‘cassandra-0’ thus all other pods in the set can be told to cluster to ‘cassandra-0’ and it will form a ring, plus the storage will survive pod restarts.

Slide 24

Slide 24

$ kubectl scale --replicas=3 \ deployment/hello

Slide 25

Slide 25

$ kubectl scale --replicas=3 deployment/hello deployment.extensions/hello scaled $ kubectl get all NAME pod/hello-64f6bf9dd4-2bndq pod/hello-64f6bf9dd4-4kq9l pod/hello-64f6bf9dd4-8lkcs NAME deployment.apps/hello READY 1/1 0/1 1/1 DESIRED 3 NAME replicaset.apps/hello-64f6bf9dd4 STATUS Running ContainerCreating Running CURRENT 3 DESIRED 3 UP-TO-DATE 2 RESTARTS 0 0 0 AVAILABLE 3 CURRENT 3 READY 2 AGE 15m 2s 5s AGE 16m AGE 16m

Slide 26

Slide 26

$ kubectl edit deployment hello ... spec: containers: - env: - name: MESSAGE value: HELLO I LOVE YOU!!!! image: paulczar/go-hello-world imagePullPolicy: Always name: hello

Slide 27

Slide 27

$ kubectl get all NAME pod/hello-5c75b546c7-4lwnn pod/hello-5c75b546c7-bwxxq pod/hello-5c75b546c7-sl2pg NAME deployment.apps/hello READY 1/1 1/1 1/1 DESIRED 3 NAME replicaset.apps/hello-5c75b546c7 replicaset.apps/hello-64f6bf9dd4 STATUS Running Running Running CURRENT 3 DESIRED 3 0 RESTARTS 0 0 0 UP-TO-DATE 3 CURRENT 3 0 AGE 1m 1m 1m AVAILABLE 3 READY 3 0 AGE 1m 23m AGE 23m

Slide 28

Slide 28

$ kubectl port-forward deployment/hello 8080 Forwarding from 127.0.0.1:8080 -> 8080 $ curl localhost:8080 <html><head><title>HELLO I LOVE YOU!!!!</title></head><body>HELLO I LOVE YOU!!!!!</body></html>

Slide 29

Slide 29

Service

Slide 30

Slide 30

$ kubectl expose deployment \ hello --type=LoadBalancer \ --port 80 --target-port 8080

Slide 31

Slide 31

kubectl expose deployment hello ● creates a service with a ClusterIP that acts as an internal loadbalancer to all pods in the “hello” deployment --type=LoadBalancer ● ● Creates a NodePort Configures a LoadBalancer to access the pods via the NodePort $ kubectl get services NAME TYPE hello LoadBalancer CLUSTER-IP 10.39.248.123 EXTERNAL-IP 35.184.17.129 PORT(S) 80:30468/TCP $ curl 35.184.17.129 <html><head><title>HELLO I LOVE YOU!!!!</title></head><body>HELLO I LOVE YOU!!!!!</body></html> AGE 5m

Slide 32

Slide 32

Service app=bacon app=bacon 10.3.55.7 Service app=bacon track Pods based on metadata and provides connectivity and service discovery (DNS, Env variables) for them. Container Container Type ClusterIP (default) exposes service on a cluster-internal IP. Container Pod Container Pod

Slide 33

Slide 33

192.168.0.5:4530 192.168.0.6:4530 K8s Worker K8s Worker Service app=bacon 10.3.55.7 Service track Pods based on metadata and provides connectivity and service discovery (DNS, Env variables) for them. Type app=bacon Pod app=bacon Container Container Container Container Pod NodePort extends ClusterIP to expose services on each node’s IP via a static port.

Slide 34

Slide 34

33.6.5.22:80 Load Balancer 192.168.0.5:4530 192.168.0.6:4530 K8s Worker K8s Worker track Pods based on metadata and provides connectivity and service discovery (DNS, Env variables) for them. Service app=bacon Service 10.3.55.7 Type app=bacon Pod app=bacon Container Container Container Container Pod LoadBalancer extends NodePort to configure a cloud provider’s load balancer using the cloud-controller-manager.

Slide 35

Slide 35

https://example.com Ingress /bacon Service app=bacon eggs Service app=eggs Ingress a controller that manages an external entity to provide load balancing, SSL termination and name-based virtual hosting to services based on a set of rules.

Slide 36

Slide 36

Volume

Slide 37

Slide 37

Volume Is [effectively] a Directory, possibly with data in it, available to all containers in a Pod. Container Usually Shares lifecycle of a Pod (Created when Pod is created, destroyed when Pod is destroyed). Persistent Volumes outlive Pods. Container Pod Can be mounted from local disk, or from a network storage device such as a EBS volume, iscsi, NFS, etc.

Slide 38

Slide 38

Config Map / Secret

Slide 39

Slide 39

$ kubectl create configmap hello \ --from-file=index.html

Slide 40

Slide 40

kubectl create configmap hello --from-file=index.html ● creates a configmap called “hello” containing the contents index.html $ kubectl get configmap hello -o yaml apiVersion: v1 kind: ConfigMap metadata: name: hello data: index.html: "<html>\n<head>\n\t<title>Hello to my friends</title>\n</head>\n<body>\n\tHello to my friends\n</body>\n</html>\n\n"

Slide 41

Slide 41

kubectl create secret generic hello --from-file=index.html ● creates a secret called “hello” containing a base64 hash of contents index.html $ kubectl get secret hello -o yaml apiVersion: v1 kind: Secret metadata: name: hello data: index.html: PGh0bWw+CjxoZWFkPgoJPHRpdGxlPkhlbGxvIHRvIG15IGZyaWVuZHM8L3RpdGxlPgo8L2hlYWQ+Cjxib2R5 PgoJSGVsbG8gdG8gbXkgZnJpZW5kcwo8L2JvZHk+CjwvaHRtbD4KCg==

Slide 42

Slide 42

ConfigMaps/Secrets (user-data) Provides key-value pairs to be injected into a pod much like user-data is injected into a Virtual Machine in the cloud. Allows you to do last minute configuration of applications running on Kubernetes such as setting a database host, or a admin password. ConfigMaps store values as strings, Secrets store them as byte arrays (serialized as base64 encoded strings). Secrets are [currently] not encrypted by default. This is likely to change. Can be injected as files in a Volume, or as Environment Variables.

Slide 43

Slide 43

Helm

Slide 44

Slide 44

Helm is the best way to find, share, and use software built for Kubernetes

Slide 45

Slide 45

services db custom Chart.yaml Values.yaml templates/ ci load balancer

Slide 46

Slide 46

Secure | https://hub.kubeapps.com Discover & launch great Kubernetes-ready apps Search charts Wordpress, Jenkins, Kubeless... 231 charts ready to deploy

Slide 47

Slide 47

apiVersion: v1 apiVersion: apps/v1beta1 kind: Service kind: Deployment metadata: metadata: name: {{ .Chart.name }}-svc name: {{ .Chart.name }}-app labels: labels: app: {{ .Chart.name }} ... apiVersion: v1 kind: ConfigMap ... metadata: spec: containers: - image: paulczar/hello-world name: hello-world volumeMounts: - name: config mountPath: /etc/hello volumes: - name: config configMap: name: {{ .Chart.name }}-cm name: {{ .Chart.name }}-cm data: db: {{ .Value.db }} app: {{ .Chart.name }}-world spec: ports: - port: {{ .Value.port }} protocol: TCP targetPort: 8080 selector: app: {{ .Chart.name }}-world type: NodePort

Slide 48

Slide 48

$ helm install --name staging . \ --set db=’user:pass@staging.mysql/dbname’ $ helm install --name production . \ --set db=’user:pass@production.mysql/dbname’

Slide 49

Slide 49

Spinnaker

Slide 50

Slide 50

https://medium.com/netflix-techblog/announcing-ribbon-tying-the-netflix-mid -tier-services-together-a89346910a62

Slide 51

Slide 51

https://giphy.com/gifs/frustrated-keyboard-g8GfH3i5F0hby

Slide 52

Slide 52

https://unsplash.com/photos/WHWYBmtn3_0

Slide 53

Slide 53

Slide 54

Slide 54

APP APP APP APP Gitlab Concourse Spinnaker

Slide 55

Slide 55

Cluster Management Pipelines ● Server Group ● Pipeline ● Cluster ● Stage ● Applications ● Deployment Strategies ● Load Balancer ● Firewall

Slide 56

Slide 56

Multi-Cloud Inventory Actions and Reactions ● Server Group ● Pipeline ● Cluster ● Stage ● Applications ● Deployment Strategies ● Load Balancer ● Firewall

Slide 57

Slide 57

Slide 58

Slide 58

Cluster Management Deployment Management Deployment Strategies ● Server Group ● Pipeline ● Cluster ● Stage ● Applications ● Deployment Strategies ● Load Balancer ● Firewall

Slide 59

Slide 59

Spinnaker Cloud API App App App

Slide 60

Slide 60

Slide 61

Slide 61

Halyard https://en.wikipedia.org/wiki/Halyard

Slide 62

Slide 62

Slide 63

Slide 63

Slide 64

Slide 64

Transforming How The World Builds Software © Copyright 2018 Pivotal Software, Inc. All rights Reserved.