Using Spinnaker to Create a Development Workflow on Kubernetes
A presentation at Spring One Tour - Seoul in in Seoul, South Korea by Paul Czarkowski
Continuous Deployment With Spinnaker & Kubernetes Paul Czarkowski Twitter: @pczarkowski © Copyright 2018 Pivotal Software, Inc. All rights Reserved. Version 1.0
Agenda ■ Who am I ? ■ Pivotal Container Service (PKS) ■ Kubernetes 101 ■ Kubernetes Manifests ■ Helm Charts ■ Spinnaker ■ Q+A Cover w/ Image
Pivotal Container Service
WE build the container YOU build the container Application Code & Frameworks Continuous delivery Github Pivotal Network Concourse Pivotal Application Service (PAS) Pivotal Container Service (PKS)
cf push kubectl run Java | .NET | NodeJS Product Updates Marketplace Pivotal and Partner Products v1 CVEs Pivotal Services Open Service Broker API Buildpacks | Spring Boot | Spring Cloud | Steeltoe Public Cloud Services Customer Managed Services Elastic | Packaged Software | Spark v2 “3Rs” v3 Embedded OS ... NSX-T (Windows & Linux) Concourse CPI (15 methods) vSphere Openstack AWS Google Cloud Azure & Azure Stack Repair — CVEs Repave Rotate — Credhub
Enterprise-Grade Kubernetes GCP Service Broker Built with open-source Kubernetes Use the PKS CLI and API to create, operate, and scale your clusters. GCP Service Broker NSX-T NSX-T PKS Control Plane Kubernetes PKS Control Plane Constant compatibility with the latest stable release of Google Kubernetes Engine—no proprietary extensions. Bring Google Cloud Platform services to your containers. Extend your workloads with machine learning, AI, and data services from Google. K8s Cluster K8s Cluster K8s Cluster Network management, security, and load balancing out-of-the-box with VMware NSX-T. Multi-cloud, multi-hypervisor. Harbor Harbor An enterprise-class container registry. Includes vulnerability scanning, identity management, and more. BOSH VMware GCP Azure Openstack BOSH Reliable and consistent operational experience for any cloud. AWS
App / Container Deployment, Services & Routing App kubectl run cf push Pivotal Cloud Foundry Elastic Runtime myapp.mydomain.net Pivotal Container Service ???? ????
App / Container Deployment, Services & Routing docker build docker push kubectl run kubectl expose App cf push Pivotal Cloud Foundry Elastic Runtime myapp.mydomain.net Pivotal Container Service ???? ????
Kubernetes 101
Logical Kubernetes Architecture K8s Master Controller Manager Etcd Kube Scheduler API Server K8s Worker K8s Worker K8s Worker Kubelet Kubelet Pod Pod Pod Docker Kubelet Pod Pod Pod Docker Kube-proxy Kube-proxy CNI Pod Pod Pod Docker Kube-proxy CNI CNI
$ kubectl --help kubectl controls the Kubernetes cluster manager. Find more information at: https://kubernetes.io/docs/reference/kubectl/overview/ Basic Commands (Beginner): create Create a resource from a file or from stdin. expose Take a replication controller, service, deployment or pod and expose it as a new Kubernetes Service run Run a particular image on the cluster set Set specific features on objects
Kubernetes Manifest apiVersion: kind: metadata: spec:
Kubernetes Manifest http 80 apiVersion: apps/v1beta1 apiVersion: v1 kind: Deployment kind: Service metadata: metadata: labels: app: hello-world name: hello-app Load Balancer name: hello-svc spec: hello-svc Service ports: app=hello-world
- port: 80 spec: replicas: 2 protocol: TCP template: targetPort: 8080 metadata: labels: app: hello-world spec: containers: - image: paulczar/hello-world name: hello-world selector: app: hello-world type: LoadBalancer http 8080 - load balanced hello-app Pod hello-app Pod app=hello-world app=hello-world
RESOURCES
POD
$ kubectl run hello \ --image=paulczar/go-hello-world
● kubectl run created a deployment “deployments.apps/hello” NAME deployment.apps/hello ● DESIRED 1 CURRENT 1 AVAILABLE 1 AGE 1m The deployment created a replicaset “replicaset.apps/hello-64f6bf9dd4” NAME replicaset.apps/hello-64f6bf9dd4 ● UP-TO-DATE 1 DESIRED 1 CURRENT 1 READY 1 AGE 1m Which created a pod “pod/hello-64f6bf9dd4-tq5dq” NAME pod/hello-64f6bf9dd4-tq5dq READY 1/1 STATUS Running RESTARTS 0 AGE 2s
Pod Container Container Pod one or more application containers that are tightly coupled, sharing network and storage. Example: a web front-end Pod that consists of an NGINX container and a PHP-FPM container with a shared unix socket and a “init” container to transform their config files based on environment variables.
Container Container Container Container Pod ReplicaSet Pod ReplicaSet Extends Pod resource to run and maintain a specific number of copies of a pod.
Deployment Container Container Container Container Pod ReplicaSet Deployment Pod a controller that ensures a set number of replicas of a Pod is running and provides update and upgrade workflows for your Pods. Example: cloud native Node app that scales horizontally and upgrades 2 pods at a time.
statefulset Container Container Container Container Pod ReplicaSet Deployment Statefulset Pod a controller that manages stateful application Deployments by providing sticky identity for pods and strict ordering and uniqueness. Example: Cassandra database. First pod is ‘cassandra-0’ thus all other pods in the set can be told to cluster to ‘cassandra-0’ and it will form a ring, plus the storage will survive pod restarts.
$ kubectl scale --replicas=3 \ deployment/hello
$ kubectl scale --replicas=3 deployment/hello deployment.extensions/hello scaled $ kubectl get all NAME pod/hello-64f6bf9dd4-2bndq pod/hello-64f6bf9dd4-4kq9l pod/hello-64f6bf9dd4-8lkcs NAME deployment.apps/hello READY 1/1 0/1 1/1 DESIRED 3 NAME replicaset.apps/hello-64f6bf9dd4 STATUS Running ContainerCreating Running CURRENT 3 DESIRED 3 UP-TO-DATE 2 RESTARTS 0 0 0 AVAILABLE 3 CURRENT 3 READY 2 AGE 15m 2s 5s AGE 16m AGE 16m
$ kubectl edit deployment hello ... spec: containers: - env: - name: MESSAGE value: HELLO I LOVE YOU!!!! image: paulczar/go-hello-world imagePullPolicy: Always name: hello
$ kubectl get all NAME pod/hello-5c75b546c7-4lwnn pod/hello-5c75b546c7-bwxxq pod/hello-5c75b546c7-sl2pg NAME deployment.apps/hello READY 1/1 1/1 1/1 DESIRED 3 NAME replicaset.apps/hello-5c75b546c7 replicaset.apps/hello-64f6bf9dd4 STATUS Running Running Running CURRENT 3 DESIRED 3 0 RESTARTS 0 0 0 UP-TO-DATE 3 CURRENT 3 0 AGE 1m 1m 1m AVAILABLE 3 READY 3 0 AGE 1m 23m AGE 23m
$ kubectl port-forward deployment/hello 8080 Forwarding from 127.0.0.1:8080 -> 8080 $ curl localhost:8080 <html><head><title>HELLO I LOVE YOU!!!!</title></head><body>HELLO I LOVE YOU!!!!!</body></html>
Service
$ kubectl expose deployment \ hello --type=LoadBalancer \ --port 80 --target-port 8080
kubectl expose deployment hello ● creates a service with a ClusterIP that acts as an internal loadbalancer to all pods in the “hello” deployment --type=LoadBalancer ● ● Creates a NodePort Configures a LoadBalancer to access the pods via the NodePort $ kubectl get services NAME TYPE hello LoadBalancer CLUSTER-IP 10.39.248.123 EXTERNAL-IP 35.184.17.129 PORT(S) 80:30468/TCP $ curl 35.184.17.129 <html><head><title>HELLO I LOVE YOU!!!!</title></head><body>HELLO I LOVE YOU!!!!!</body></html> AGE 5m
Service app=bacon app=bacon 10.3.55.7 Service app=bacon track Pods based on metadata and provides connectivity and service discovery (DNS, Env variables) for them. Container Container Type ClusterIP (default) exposes service on a cluster-internal IP. Container Pod Container Pod
192.168.0.5:4530 192.168.0.6:4530 K8s Worker K8s Worker Service app=bacon 10.3.55.7 Service track Pods based on metadata and provides connectivity and service discovery (DNS, Env variables) for them. Type app=bacon Pod app=bacon Container Container Container Container Pod NodePort extends ClusterIP to expose services on each node’s IP via a static port.
33.6.5.22:80 Load Balancer 192.168.0.5:4530 192.168.0.6:4530 K8s Worker K8s Worker track Pods based on metadata and provides connectivity and service discovery (DNS, Env variables) for them. Service app=bacon Service 10.3.55.7 Type app=bacon Pod app=bacon Container Container Container Container Pod LoadBalancer extends NodePort to configure a cloud provider’s load balancer using the cloud-controller-manager.
https://example.com Ingress /bacon Service app=bacon eggs Service app=eggs Ingress a controller that manages an external entity to provide load balancing, SSL termination and name-based virtual hosting to services based on a set of rules.
Volume
Volume Is [effectively] a Directory, possibly with data in it, available to all containers in a Pod. Container Usually Shares lifecycle of a Pod (Created when Pod is created, destroyed when Pod is destroyed). Persistent Volumes outlive Pods. Container Pod Can be mounted from local disk, or from a network storage device such as a EBS volume, iscsi, NFS, etc.
Config Map / Secret
$ kubectl create configmap hello \ --from-file=index.html
kubectl create configmap hello --from-file=index.html ● creates a configmap called “hello” containing the contents index.html $ kubectl get configmap hello -o yaml apiVersion: v1 kind: ConfigMap metadata: name: hello data: index.html: "<html>\n<head>\n\t<title>Hello to my friends</title>\n</head>\n<body>\n\tHello to my friends\n</body>\n</html>\n\n"
kubectl create secret generic hello --from-file=index.html ● creates a secret called “hello” containing a base64 hash of contents index.html $ kubectl get secret hello -o yaml apiVersion: v1 kind: Secret metadata: name: hello data: index.html: PGh0bWw+CjxoZWFkPgoJPHRpdGxlPkhlbGxvIHRvIG15IGZyaWVuZHM8L3RpdGxlPgo8L2hlYWQ+Cjxib2R5 PgoJSGVsbG8gdG8gbXkgZnJpZW5kcwo8L2JvZHk+CjwvaHRtbD4KCg==
ConfigMaps/Secrets (user-data) Provides key-value pairs to be injected into a pod much like user-data is injected into a Virtual Machine in the cloud. Allows you to do last minute configuration of applications running on Kubernetes such as setting a database host, or a admin password. ConfigMaps store values as strings, Secrets store them as byte arrays (serialized as base64 encoded strings). Secrets are [currently] not encrypted by default. This is likely to change. Can be injected as files in a Volume, or as Environment Variables.
Helm
Helm is the best way to find, share, and use software built for Kubernetes
services db custom Chart.yaml Values.yaml templates/ ci load balancer
Secure | https://hub.kubeapps.com Discover & launch great Kubernetes-ready apps Search charts Wordpress, Jenkins, Kubeless... 231 charts ready to deploy
apiVersion: v1 apiVersion: apps/v1beta1 kind: Service kind: Deployment metadata: metadata: name: {{ .Chart.name }}-svc name: {{ .Chart.name }}-app labels: labels: app: {{ .Chart.name }} ... apiVersion: v1 kind: ConfigMap ... metadata: spec: containers: - image: paulczar/hello-world name: hello-world volumeMounts: - name: config mountPath: /etc/hello volumes: - name: config configMap: name: {{ .Chart.name }}-cm name: {{ .Chart.name }}-cm data: db: {{ .Value.db }} app: {{ .Chart.name }}-world spec: ports: - port: {{ .Value.port }} protocol: TCP targetPort: 8080 selector: app: {{ .Chart.name }}-world type: NodePort
$ helm install --name staging . \ --set db=’user:pass@staging.mysql/dbname’ $ helm install --name production . \ --set db=’user:pass@production.mysql/dbname’
Spinnaker
https://medium.com/netflix-techblog/announcing-ribbon-tying-the-netflix-mid -tier-services-together-a89346910a62
https://giphy.com/gifs/frustrated-keyboard-g8GfH3i5F0hby
https://unsplash.com/photos/WHWYBmtn3_0
APP APP APP APP Gitlab Concourse Spinnaker
Cluster Management Pipelines ● Server Group ● Pipeline ● Cluster ● Stage ● Applications ● Deployment Strategies ● Load Balancer ● Firewall
Multi-Cloud Inventory Actions and Reactions ● Server Group ● Pipeline ● Cluster ● Stage ● Applications ● Deployment Strategies ● Load Balancer ● Firewall
Cluster Management Deployment Management Deployment Strategies ● Server Group ● Pipeline ● Cluster ● Stage ● Applications ● Deployment Strategies ● Load Balancer ● Firewall
Spinnaker Cloud API App App App
Halyard https://en.wikipedia.org/wiki/Halyard
Transforming How The World Builds Software © Copyright 2018 Pivotal Software, Inc. All rights Reserved.
Paul will give an introduction to Containers and Kubernetes and the default development/deployment workflows that it enables. He will then show you how you can use Spinnaker to simplify and streamline your workflow and help provide a full #gitops style CI/CD.
for free. You
can too.
Resources
The following resources were mentioned during the presentation or are useful additional information.
