Kubernetes Cookbook

2nd Edition

Building Cloud-Native Applications

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

Sameer Naik, Sébastien Goasguen, and Jonathan Michaux

Kubernetes Cookbook

by Sameer Naik, Sébastien Goasguen, and Jonathan Michaux

Copyright © 2023 CloudTank SARL, Sameer Naik and Jonathan Michaux. All rights reserved.

Printed in the United States of America.

Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.

O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles (http://oreilly.com). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com.

Editors: Jeff Bleiel

Production Editor: Ashley Stussy

Interior Designer: David Futato

Cover Designer: Karen Montgomery

Illustrator: Kate Dullea

March 2018: First Edition

December 2023: Second Edition

Revision History for the Early Release

2023-05-05: First Release

2023-06-26: Second Release

See http://oreilly.com/catalog/errata.csp?isbn=9781098142247 for release details.

The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. A Functional Approach to Java, the cover image, and related trade dress are trademarks of O’Reilly Media, Inc.

The views expressed in this work are those of the authors, and do not represent the publisher’s views. While the publisher and the authors have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights.

978-1-098-14218-6

Chapter 1. Getting Started with Kubernetes

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 1st chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the author at Oreilly@triggermesh.com.

In this first chapter we present recipes that will help you get started with Kubernetes. We show you how to use Kubernetes without installing it and introduce components such as the command-line interface (CLI) and the dashboard, which allow you to interact with a cluster, as well as Minikube, an all-in-one solution you can run on your laptop.

1.1 Using Kubernetes Without Installation

Problem

You want to try Kubernetes without installing it.

Solution

To use Kubernetes without installing it, follow the interactive tutorial on the Kubernetes website.

1.2 Installing the Kubernetes CLI, kubectl

Problem

You want to install the Kubernetes command-line interface so you can interact with your Kubernetes cluster.

Solution

Install kubectl in one of the following ways:

Download the source tarballs.

Use a package manager.

Build from source (see Recipe 13.1).

The documentation highlights a few mechanisms to get kubectl. The easiest is to download the latest official release. For example, on a Linux system, to get the latest stable version, enter:

$ wget https://dl.k8s.io/release/$(wget -qO - https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl

$ sudo install -m 755 kubectl /usr/local/bin/kubectl

Linux and macOS users can also install kubectl using the Homebrew package manager.

$ brew install kubectl

Google Kubernetes Engine users (see Recipe 13.1) will get kubectl as part of the gcloud command installation. For example, on Sébastien’s local machine:

$ which kubectl
/Users/sebgoa/google-cloud-sdk/bin/kubectl

Also note that the latest versions of Minikube (see Recipe 1.3) packages kubectl and will install it in your $PATH if it is not found.

Before you move on from this recipe, make sure you have a working kubectl by listing its version. This command will also try to get the version of the default Kubernetes cluster:

$ kubectl version --client --short
Flag --short has been deprecated, and will be removed in the future. The --short output will become the default.
Client Version: v1.25.4
Kustomize Version: v4.5.7

See Also

Documentation on installing kubectl

1.3 Installing Minikube to Run a Local Kubernetes Instance

Problem

You want to use Kubernetes for testing or development or for training purposes on your local machine.

Solution

Use Minikube. Minikube is a tool that lets you use Kubernetes on your local machine without any installation except for the minikube binary. It takes advantage of your local hypervisor (e.g., VirtualBox, Hyperkit, KVM, etc.) and launches a virtual machine that runs Kubernetes in a single node.

To install the Minikube CLI locally, you can get the latest release or build from source. To install the latest release of minikube on a Linux-based machine, do:

$ wget https://github.com/kubernetes/minikube/releases/latest/download/minikube-linux-amd64 -O minikube

$ sudo install -m 755 minikube /usr/local/bin/minikube

This will put the minikube binary in your path and make it accessible from everywhere.

Discussion

Once minikube is installed, you can verify the Minikube version with the following command:

$ minikube version
minikube version: v1.28.0
commit: 986b1e...

Now you can start a cluster with:

$ minikube start

Minikube can be deployed as a VM, a container, or bare-metal. This is configured using the --driver option. When no driver is specified Minikube will automatically select one to create the cluster. After the cluster has been created, your Kubernetes client, kubectl, will have a minikube context which will be configured to be the default context. Checking what nodes you have in your cluster will return the minikube hostname:

$ kubectl get nodes
NAME       STATUS   ROLES           AGE     VERSION
minikube   Ready    control-plane   2m45s   v1.25.3

See Also

Minikube Get Started Guide

Minikube Drivers

minikube source on GitHub

1.4 Using Minikube Locally for Development

Problem

You want to use Minikube locally for testing and development of your Kubernetes application. You have installed and started minikube (see Recipe 1.3) and want to know a few extra commands to simplify your development experience.

Solution

The Minikube CLI offers a few commands that make your life easier. The CLI has built-in help that you can use to discover the subcommands on your own—​here’s a snippet:

$ minikube
...
Basic Commands:
  start            Starts a local Kubernetes cluster
  status           Gets the status of a local Kubernetes cluster
  stop             Stops a running local Kubernetes cluster
  delete           Deletes a local Kubernetes cluster
...
Configuration and Management Commands:
  addons           Enable or disable a minikube addon
...

Aside from start, stop, and delete, you should become familiar with the ip, ssh, tunnel, dashboard, and docker-env commands.

Tip

Minikube runs a Docker engine to be able to start containers. In order to access this Docker engine from your local machine using your local Docker client, you’ll need to set up the correct Docker environment with eval $(minikube docker-env).

Discussion

The minikube start command starts the virtual machine (VM) that will run Kubernetes locally. By default it will allocate 2 GB of RAM, so when you are done, do not forget to stop it with minikube stop. Also, you can give the VM more memory and CPUs as well as pick a certain Kubernetes version to run—​for example:

$ minikube start --cpus=4 --memory=4000 --kubernetes-version=v1.7.2

For debugging the Docker daemon that is used inside Minikube, you might find minikube ssh handy; it will log you into the virtual machine. To get the IP address of the Minikube VM, use minikube ip. Finally, to launch the Kubernetes dashboard in your default browser, use minikube dashboard.

Tip

If for any reason your Minikube becomes unstable, or you want to start afresh, you can remove it with minikube stop and minikube delete. Then a minikube start will give you a fresh installation.

1.5 Starting Your First Application on Minikube

Problem

You’ve started Minikube (see Recipe 1.3), and now you want to launch your first application on Kubernetes.

Solution

As an example, you can start the Ghost microblogging platform on Minikube using two kubectl commands:

$ kubectl run ghost --image=ghost:0.9
$ kubectl expose deployments ghost --port=2368 --type=NodePort

Monitor the pod manually to see when it starts running and then use the minikube service command to open your browser automatically and access Ghost:

$ kubectl get pods
NAME                     READY     STATUS    RESTARTS   AGE
ghost-8449997474-kn86m   1/1       Running   0          2h

$ minikube service ghost

Discussion

The kubectl run command is called a generator; it is a convenience command to create a Deployment object (see Recipe 3.5). The kubectl expose command is also a generator, a convenience command to create a Service object (see Recipe 4.1) that routes network traffic to the containers started by your deployment.

1.6 Accessing the Dashboard in Minikube

Problem

You are using Minikube and want to access the Kubernetes dashboard to start your first application from a graphical user interface.

Solution

You can open the Kubernetes dashboard from Minikube with:

$ minikube dashboard

Click on the plus sign (+) at the top right of the UI that opens in your browser, and you will see the page depicted in Figure 1-1.

Figure 1-1. Snapshot of the dashboard application create view

Discussion

To create an application, click the Create button in the top-right corner, give the application a name, and specify the Docker image that you want to use. Then click the Deploy button and you will be presented with a new view that shows deployments and replica sets, and after a bit of time you will see a pod. These are some key API primitives we will deal with in greater detail in the rest of the book.

The snapshot in Figure 1-2 presents a typical dashboard view after having created a single application using the Redis container.

Figure 1-2. A dashboard overview with a Redis application

If you go back to a terminal session and use the command-line client, you will see the same thing:

$ kubectl get pods,rs,deployments
NAME                        READY     STATUS    RESTARTS   AGE
po/redis-3215927958-4x88v   1/1       Running   0          24m

NAME                  DESIRED   CURRENT   READY     AGE
rs/redis-3215927958   1         1         1         24m

NAME           DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/redis   1         1         1            1           24m

Your Redis pod will be running the Redis server, as the following logs show:

$ kubectl logs redis-3215927958-4x88v
...
                _._
           _.-``__ ''-._
      _.-``    `.  `_.  ''-._           Redis 3.2.9 (00000000/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 1
  `-._    `-._  `-./  _.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |           http://redis.io
  `-._    `-._`-.__.-'_.-'    _.-'
 |`-._`-._    `-.__.-'    _.-'_.-'|
 |    `-._`-._        _.-'_.-'    |
  `-._    `-._`-.__.-'_.-'    _.-'
      `-._    `-.__.-'    _.-'
          `-._        _.-'
              `-.__.-'

...
1:M 14 Jun 07:28:56.637 # Server started, Redis version 3.2.9
1:M 14 Jun 07:28:56.643 * The server is now ready to accept connections on
port 6379

Chapter 2. Learning to Use the Kubernetes Client

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 3rd chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the author at Oreilly@triggermesh.com.

This chapter gathers recipes around the basic usage of the Kubernetes command-line interface (CLI), kubectl. See Chapter 1 for how to install the CLI tool; for advanced use cases, see Chapter 5, where we show how to use the Kubernetes API.

2.1 Listing Resources

Problem

You want to list Kubernetes resources of a certain kind.

Solution

Use the get verb of kubectl along with the resource type. To list all pods:

$ kubectl get pods

To list all services and deployments:

$ kubectl get services,deployments

To list a specific deployment:

$ kubectl get deployment myfirstk8sapp

To list all resources:

$ kubectl get all

Note that kubectl get is a very basic but extremely useful command to get a quick overview what is going on in the cluster—​it’s essentially the equivalent to ps on Unix.

Tip

Many resources have short names you can use with kubectl, sparing your time and sanity. Here are some examples:

configmaps (aka cm)

daemonsets (aka ds)

deployments (aka deploy)

endpoints (aka ep)

events (aka ev)

horizontalpodautoscalers (aka hpa)

ingresses (aka ing)

namespaces (aka ns)

nodes (aka no)

persistentvolumeclaims (aka pvc)

persistentvolumes (aka pv)

pods (aka po)

replicasets (aka rs)

replicationcontrollers (aka rc)

resourcequotas (aka quota)

serviceaccounts (aka sa)

services (aka svc)

2.2 Deleting Resources

Problem

You no longer need resources and want to get rid of them.

Solution

Use the delete verb of kubectl along with the type and name of the resource you wish to delete.

To delete all resources in the namespace my-app, do:

$ kubectl get ns
NAME          STATUS    AGE
default       Active    2d
kube-public   Active    2d
kube-system   Active    2d
my-app        Active    20m

$ kubectl delete ns my-app
namespace "my-app" deleted

If you’re wondering how to create a namespace, see Recipe 5.3.

You can also delete specific resources and/or influence the process by which they are destroyed. To delete services and deployments labeled with app=niceone, do:

$ kubectl delete svc,deploy -l app=niceone

To force deletion of a pod, do:

$ kubectl delete pod hangingpod --grace-period=0 --force

To delete all pods in the namespace test, do:

$ kubectl delete pods --all --namespace test

Discussion

Do not delete supervised objects such as pods controlled by a deployment directly. Rather, kill their supervisors or use dedicated operations to get rid of the managed resources. For example, if you scale a deployment to zero replicas (see Recipe 9.1), then you effectively delete all the pods it looks after.

Another aspect to take into account is cascading versus direct deletion—​for example, when you delete a custom resource definition (CRD) as shown in Recipe 13.4, all its dependent objects are deleted too. To learn more about how to influence the cascading deletion policy, read Garbage Collection in the Kubernetes docs.

2.3 Watching Resource Changes with kubectl

Problem

You want to watch the changes to Kubernetes objects in an interactive manner in the terminal.

Solution

The kubectl command has a --watch option that gives you this behavior. For example, to watch pods:

$ kubectl get pods --watch

Note that this is a blocking and autoupdating command, akin to top.

Discussion

The --watch option is useful, but sometimes not very reliable, in terms of refreshing the screen correctly. Alternatively, you can use the watch command, as in:

$ watch kubectl get pods

2.4 Editing Resources with kubectl

Problem

You want to update the property of a Kubernetes resource.

Solution

Use the edit verb of kubectl along with the resource type:

$ kubectl run nginx --image=nginx
$ kubectl edit pod/nginx

Now edit the nginx pod in your editor—​for example, add a new label called mylabel with value true. Once you save, you’ll see something like:

pod/nginx edited

Discussion

If you have editor issues, use EDITOR=vi. Also be aware that not all changes trigger a deployment.

Some triggers have shortcuts, for example, if you want to change the image version a deployment uses, simply use kubectl set image, which updates the existing container images of resources (valid for deployments, replica sets/replication controllers, daemon sets, jobs, and simple pods).

2.5 Asking kubectl to Explain Resources and Fields

Problem

You want to gain a deeper understanding of a certain resource—for example, service—and/or understand what exactly a certain field in a Kubernetes manifest means, including default values and if it’s required or optional.

Solution

Use the explain verb of kubectl:

$ kubectl explain svc
DESCRIPTION:
Service is a named abstraction of software service (for example, mysql)
consisting of local port (for example 3306) that the proxy listens on, and the
selector that determines which pods will answer requests sent through the proxy.

FIELDS:
   status       <Object>
     Most recently observed status of the service. Populated by the system.
     Read-only. More info: https://git.k8s.io/community/contributors/devel/
     api-conventions.md#spec-and-status/

   apiVersion   <string>
     APIVersion defines the versioned schema of this representation of an
     object. Servers should convert recognized schemas to the latest internal
     value, and may reject unrecognized values. More info:
     https://git.k8s.io/community/contributors/devel/api-conventions.md#resources

   kind <string>
     Kind is a string value representing the REST resource this object
     represents. Servers may infer this from the endpoint the client submits
     requests to. Cannot be updated. In CamelCase. More info:
     https://git.k8s.io/community/contributors/devel/api-conventions
     .md#types-kinds

   metadata     <Object>
     Standard object's metadata. More info:
     https://git.k8s.io/community/contributors/devel/api-conventions.md#metadata

   spec <Object>
     Spec defines the behavior of a service. https://git.k8s.io/community/
     contributors/devel/api-conventions.md#spec-and-status/

$ kubectl explain svc.spec.externalIPs
FIELD: externalIPs <[]string>

DESCRIPTION:
     externalIPs is a list of IP addresses for which nodes in the cluster will
     also accept traffic for this service.  These IPs are not managed by
     Kubernetes.  The user is responsible for ensuring that traffic arrives at a
     node with this IP.  A common example is external load-balancers that are not
     part of the Kubernetes system.

Discussion

The kubectl explain command¹ pulls the descriptions of resources and fields from the Swagger/OpenAPI definitions,² exposed by the API server.

See Also

Ross Kukulinski’s blog post, “kubectl explain — #HeptioProTip”

¹ Kubernetes, “Kubectl Reference Docs: Explain”.

² Kubernetes, “The Kubernetes API”.

Chapter 3. Creating and Modifying Fundamental Workloads

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 4th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the author at Oreilly@triggermesh.com.

In this chapter, we present recipes that show you how to manage fundamental Kubernetes workload types: pods and deployments. We show how to create deployments and pods via CLI commands and from a YAML manifest, and explain how to scale and update a deployment.

3.1 Creating a pod Using kubectl run

Problem

You want to quickly launch a long-running application such as a web server.

Solution

Use the kubectl run command, a generator that creates a pod on the fly. For example, to create a pod that runs the Ghost microblogging platform do the following:

$ kubectl run nginx --image=nginx

$ kubectl get pod/nginx
NAME    READY   STATUS    RESTARTS   AGE
nginx   1/1     Running   0          3m55s

Discussion

The kubectl run command can take a number of arguments to configure additional parameters of the deployments. For example, you can do the following:

Set environment variables with --env

Define container ports with --port

Define a command to run using --command

Automatically create an associated service with --expose

Test a run without actually running anything with --dry-run

Typical usages are as follows. To launch nginx serving on port 2368 and create a service along with it, enter:

$ kubectl run nginx --image=nginx --port=2368 --expose

To launch MySQL with the root password set, enter:

$ kubectl run mysql --image=mysql --env=MYSQL_ROOT_PASSWORD=root

To launch a busybox container and execute the command sleep 3600 on start, enter:

$ kubectl run myshell --image=busybox --command -- sh -c "sleep 3600"

See also kubectl run --help for more details about the available arguments.

3.2 Creating a Deployment using kubectl create

Problem

You want to quickly launch a long-running application such as a content management system.

Solution

Use the kubectl create deployment to create a deployment manifest on the fly. For example, to create a deployment that runs the wordpress content management system, do the following:

$ kubectl create deployment wordpress --image wordpress

$ kubectl get deployments.apps/wordpress
NAME        READY   UP-TO-DATE   AVAILABLE   AGE
wordpress   1/1     1            1           90s

Discussion

The kubectl create deployment command can take a number of arguments to configure additional parameters of the deployments. For example, you can do the following:

Define container ports with --port

Define the number of replicas using --replicas

Test a run without actually running anything with --dry-run

Provide the created manifest using --output yaml

See also kubectl create deployment --help for more details about the available arguments.

3.3 Creating Objects from File Manifests

Problem

Rather than creating an object via a generator such as kubectl run, you want to explicitly state its properties and then create it.

Solution

Use kubectl create like so:

$ kubectl create -f <manifest>

In Recipe 5.3 you’ll see how to create a namespace using a YAML manifest. This is one of the simplest examples as the manifest is very short. It can be written in YAML or JSON—​for example, with a YAML manifest file myns.yaml like so:

apiVersion: v1
kind: Namespace
metadata:
  name: myns

You can create this object with kubectl create -f myns.yaml.

Discussion

You can point kubectl create to a URL instead, or a filename in your local filesystem. For example, to create the frontend for the canonical Guestbook application, get the URL of the raw YAML that defines the application in a single manifest and enter:

$ kubectl create -f https://raw.githubusercontent.com/kubernetes/master/ \
                    examples/guestbook/all-in-one/guestbook-all-in-one.yaml

3.4 Writing a Pod Manifest from Scratch

Problem

You want to write a pod manifest from scratch and not use a generator such as kubectl run.

Solution

A Pod is an /api/v1 object and, like any other Kubernetes object, its manifest file contains the following fields:

apiVersion, which specifies the API version

kind, which indicates the type of the object

metadata, which provides some metadata about the object

spec, which provides the object specification

The pod manifest contains an array of containers and an optional array of volumes (see Chapter 7). In its simplest form, with a single container and no volume, it looks as follows:

apiVersion:    v1
kind:          Pod
metadata:
  name:        oreilly
spec:
  containers:
  - name:      oreilly
    image:     nginx

Save this YAML manifest in a file called oreilly.yaml and then use kubectl to create it:

$ kubectl create -f oreilly.yaml

Discussion

The API specification of a pod is much richer than what is shown in the Solution, which is the most basic functioning pod. For example, a pod can contain multiple containers, as shown here:

apiVersion: v1
kind:       Pod
metadata:
  name:     oreilly
spec:
  containers:
  - name:   oreilly
    image:  nginx
  - name:   safari
    image:  redis

A pod can also contain volume definitions to load data in the containers (see Recipe 7.1), as well as probes to check the health of the containerized application (see Recipe 11.2 and Recipe 11.3).

A description of the thinking behind many of the specification fields and a link to the full API object specification is detailed in the documentation.

Note

Unless for very specific reasons, never create a pod on its own. Use a Deployment object (see Recipe 3.5) to supervise pods—​it will watch over the pods through another object called a ReplicaSet.

See Also

Kubernetes Pods reference documentation.

ReplicaSet documentation

3.5 Launching a Deployment Using a Manifest

Problem

You want to have full control over how a (long-running) app is launched and supervised.

Solution

Write a manifest using the Deployment object in it. For the basics, see also Recipe 3.4.

Let’s say you have manifest file called fancyapp.yaml with the following content:

apiVersion:       apps/v1
kind:             Deployment
metadata:
  name:           fancyapp
spec:
  replicas:       5
  selector:
    matchLabels:
      app: fancy
  template:
    metadata:
      labels:
        app:      fancy
        env:      development
    spec:
      containers:
      - name:     sise
        image:    mhausenblas/simpleservice:0.5.0
        ports:
        - containerPort: 9876
        env:
        - name:   SIMPLE_SERVICE_VERSION
          value:  "0.9"

As you can see, there are a couple of things you might want to do explicitly when launching the app:

Set the number of pods (replicas), or identical copies, that should be launched and supervised.

Label it, such as with env=development (see also Recipe 5.5 and Recipe 5.6).

Set environment variables, such as SIMPLE_SERVICE_VERSION.

Now let’s have a look at what the deployment entails:

$ kubectl create -f fancyapp.yaml
deployment.apps/fancyapp created

$ kubectl get deploy
NAME       READY   UP-TO-DATE   AVAILABLE   AGE
fancyapp   5/5     5            5           57s

$ kubectl get rs
NAME                  DESIRED   CURRENT   READY     AGE
fancyapp-1223770997   5         5         0         59s

$ kubectl get po
NAME                                 READY   STATUS    RESTARTS      AGE
fancyapp-74c6f7cfd7-98d97            1/1     Running   0             115s
fancyapp-74c6f7cfd7-9gm2l            1/1     Running   0             115s
fancyapp-74c6f7cfd7-kggsx            1/1     Running   0             115s
fancyapp-74c6f7cfd7-xfs6v            1/1     Running   0             115s
fancyapp-74c6f7cfd7-xntk2            1/1     Running   0             115s
frontend-79b6ddfbfc-22h2n            1/1     Running   0             19m
frontend-79b6ddfbfc-2grhr            1/1     Running   0             19m
frontend-79b6ddfbfc-s5sj7            1/1     Running   0             19m

Warning

When you want to get rid of a deployment, and with it the replica sets and pods it supervises, execute a command like kubectl delete deploy/fancyapp. Do not try to delete individual pods, as they will be recreated by the deployment. This is something that often confuses beginners.

Deployments allow you to scale the app (see Recipe 9.1) as well as roll out a new version or roll back to a previous version. They are, in general, good for stateless apps that require pods with identical characteristics.

Discussion

A deployment is a supervisor for pods and replica sets (RSs), giving you fine-grained control over how and when a new pod version is rolled out or rolled back to a previous state. The RSs and pods that a deployment supervises are generally of no interest to you unless, for example, you need to debug a pod (see Recipe 12.5). Figure 3-1 illustrates how you can move back and forth between deployment revisions.

Figure 3-1. Deployment revisions

To generate the manifest for a deployment, you can use the kubectl create command and the --dry-run option. It will allow you to generate the manifest in YAML or JSON format and save the manifest for later use. For example, to create the manifest of a deployment called fancy-app using the Docker image nginx, issue the following command:

$ kubectl create deployment fancyapp --image nginx -o json --dry-run=client
{
    "kind": "Deployment",
    "apiVersion": "apps/v1",
    "metadata": {
        "name": "fancyapp",
        "creationTimestamp": null,
        "labels": {
            "app": "fancyapp"
        }
    },
...

See Also

Kubernetes Deployments in documentation

3.6 Updating a Deployment

Problem

You have a deployment and want to roll out a new version of your app.

Solution

Update your deployment and let the default update strategy, RollingUpdate, automatically handle the rollout.

For example, suppose you create a new container image and want to update the deployment based on it:

kubectl create deployment myapp --image=mhausenblas/simpleservice:0.4.0

$ kubectl create deployment myapp --image=mhausenblas/simpleservice:0.4.0
deployment.apps/myapp created

$ kubectl set image deployment/myapp simpleservice=mhausenblas/simpleservice:0.5.0
deployment.apps/myapp image updated

$ kubectl rollout status deployment myapp
deployment "myapp" successfully rolled out

$ kubectl rollout history deployment sise
deployment.apps/myapp
REVISION        CHANGE-CAUSE
1               <none>
2               <none>

You’ve now successfully rolled out a new revision of your deployment where only the container image used has changed. All other properties of the deployment, such as the number of replicas, stay unchanged. But what if you want to update other aspects of the deployment, such as changing environment variables? You can use a number of kubectl commands to update the deployment. For example, to add a port definition to the current deployment, you can use kubectl edit:

$ kubectl edit deploy myapp

This command will open the current deployment in your default editor, or, when set and exported, in the editor specified by the environment variable KUBE_EDITOR.

Say you want to add the following port definition:

...
  ports:
  - containerPort: 9876
...

The result of the editing process (in this case, with KUBE_EDITOR set to vi) is shown in Figure 3-2.

Figure 3-2. Editing a deployment

Once you save and exit the editor, Kubernetes kicks off a new deployment, now with the port defined. Let’s verify that:

$ kubectl rollout history deployment myapp
deployments "sise"
REVISION        CHANGE-CAUSE
1               <none>
2               <none>
3               <none>

Indeed, we see that revision 3 has been rolled out with the changes we introduced with kubectl edit. The reason the CHANGE-CAUSE column is empty is that you didn’t use kubectl create with the --record option. If you want to see what triggered a revision, add this option.

As mentioned earlier, there are more kubectl commands that you can use to update your deployment:

Use kubectl apply to update a deployment (or create it if it doesn’t exist) from a manifest file—​for example, kubectl apply -f simpleservice.yaml.

Use kubectl replace to replace a deployment from a manifest file—​for example, kubectl replace -f simpleservice.yaml. Note that unlike apply, in order to use replace, the deployment must already exist.

Use kubectl patch to update a specific key—​for example:

kubectl patch deployment myapp -p '{"spec": {"template":
{"spec": {"containers":
[{"name": "sise", "image": "mhausenblas/simpleservice:0.5.0"}]}}}}'

What if you make a mistake or experience issues with the new version of the deployment? Luckily, Kubernetes makes it really easy to roll back to a known good state using the kubectl rollout undo command. For example, suppose the last edit was a mistake and you want to roll back to revision 2. You can do this with the following command:

$ kubectl rollout undo deployment myapp ‐‐to‐revision=2

You can then verify that the port definition has been removed with kubectl get deploy/myapp -o yaml.

Note

The rollout of a deployment is only triggered if parts of the pod template (that is, keys below .spec.template) are changed, such as environment variables, ports, or the container image. Changes to aspects of the deployments, such as the replica count, do not trigger a new deployment.

Chapter 4. Working with Services

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 5th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the author at Oreilly@triggermesh.com.

In this chapter, we discuss how pods communicate within the cluster, how applications discover each other, and how to expose pods so that they can be accessed from outside of the cluster.

The resource we will be using here is called a Kubernetes service, as depicted in Figure 4-1.

Figure 4-1. The Kubernetes service concept

A service provides a stable virtual IP (VIP) address for a set of pods. Though pods may come and go, services allow clients to reliably discover and connect to the containers running in the pods by using the VIP. The “virtual” in VIP means it’s not an actual IP address connected to a network interface; its purpose is purely to forward traffic to one or more pods. Keeping the mapping between the VIP and the pods up to date is the job of kube-proxy, a process that runs on every node on the cluster. This kube-proxy process queries the API server to learn about new services in the cluster and updates the node’s iptables rules (iptables) accordingly, to provide the necessary routing information.

4.1 Creating a Service to Expose Your Application

Problem

You want to provide a stable and reliable way to discover and access your application within the cluster.

Solution

Create a Kubernetes service for the pods that make up your application.

Assuming you created an nginx deployment with kubectl create deployment nginx --image nginx, you can automatically create a Service object using the kubectl expose command, like so:

$ kubectl expose deploy/nginx --port 80
service "nginx" exposed

$ kubectl describe svc/nginx
Name:              nginx
Namespace:         default
Labels:            app=nginx
Annotations:       <none>
Selector:          app=nginx
Type:              ClusterIP
IP Family Policy:  SingleStack
IP Families:       IPv4
IP:                10.97.137.240
IPs:               10.97.137.240
Port:              <unset>  80/TCP
TargetPort:        80/TCP
Endpoints:         172.17.0.3:80
Session Affinity:  None
Events:            <none>

You will then see the object appear when you list the service:

$ kubectl get svc nginx
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
nginx        ClusterIP   10.97.137.240   <none>        80/TCP    2s

Discussion

To access this service via your browser, run a proxy in a separate terminal, like so:

$ kubectl proxy
Starting to serve on 127.0.0.1:8001

Then open your browser with:

$ open http://localhost:8001/api/v1/namespaces/default/services/nginx/proxy/

If you wanted to write a Service object by hand for the same nginx deployment, you would write the following YAML file:

apiVersion:  v1
kind:        Service
metadata:
  name:      nginx
spec:
  selector:
    app:     nginx
  ports:
  - port:    80

The one thing to pay attention to in this YAML file is the selector, which is used to select all the pods that make up this microservice abstraction. Kubernetes uses the Service object to dynamically configure the iptables on all the nodes to be able to send the network traffic to the containers that make up the microservice. The selection is done as a label query (see Recipe 5.6) and results in a list of endpoints.

Tip

If your service does not seem to be working properly, check the labels used in the selector and verify that a set of endpoints is being populated with kubectl get endpoints. If not, this most likely means that your selector is not finding any matching pods.

Note

Pod supervisors, such as deployments or replication controllers, operate orthogonally to services. Both supervisors and services find the pods they’re looking after by using labels, but they have different jobs to do: supervisors monitor the health of and restart pods, and services make them accessible in a reliable way.

See Also

Kubernetes Services documentation

Kubernetes tutorial “Using a Service to Expose Your App”

4.2 Verifying the DNS Entry of a Service

Problem

You have created a service (see Recipe 4.1) and want to verify that your DNS registration is working properly.

Solution

By default Kubernetes uses ClusterIP as the service type, and that exposes the service on a cluster-internal IP. If the DNS cluster add-on is available and working properly, you can access the service via a fully qualified domain name (FQDN) in the form of $SERVICENAME.$NAMESPACE.svc.cluster.local.

To verify that this is working as expected, get an interactive shell within a container in your cluster. The easiest way to do this is to use kubectl run with the busybox image, like so:

$ kubectl run busybox --rm -it --image busybox  -- /bin/sh
If you don't see a command prompt, try pressing enter.

/ # nslookup nginx
Server:		10.96.0.10
Address:	10.96.0.10:53

Name:	nginx.default.svc.cluster.local
Address: 10.100.34.223

The IP address returned for the service should correspond to its cluster IP.

4.3 Changing the Type of a Service

Problem

You have an existing service, say of type ClusterIP, as discussed in Recipe 4.2 and you want to change its type so that you can expose your application as a NodePort or via a cloud provider load balancer using the LoadBalancer service type.

Solution

Use the kubectl edit command along with your preferred editor to change the service type. Suppose you have a manifest file called simple-nginx-svc.yaml with this content:

kind:       Service
apiVersion: v1
metadata:
  name:     webserver
spec:
  ports:
  - port:   80
  selector:
    app:    nginx

Create the webserver service and query for it:

$ kubectl create -f simple-nginx-svc.yaml

$ kubectl get svc/webserver
NAME        TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
webserver   ClusterIP   10.98.223.206   <none>        80/TCP    11s

Next, change the service type to, say, NodePort, like so:

$ kubectl edit svc/webserver

This command will download the current spec the API server has of the service and open it in your default editor (EDITOR=vi is set here).

# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
kind: Service
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"webserver","namespace":"default"},"spec":{"ports":[{"port":80}],"selector":{"app":"nginx"}}}
  creationTimestamp: "2023-03-01T14:07:55Z"
  name: webserver
  namespace: default
  resourceVersion: "1128"
  uid: 48daed0e-a16f-4923-bd7e-1d879dc2221f
spec:
  clusterIP: 10.98.223.206
  clusterIPs:
  - 10.98.223.206
  externalTrafficPolicy: Cluster
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - nodePort: 31275
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  sessionAffinity: None
  type: NodePort
status:
  loadBalancer: {}

Once you’ve saved the edits (changing type to NodePort), you can verify the updated service, like so:

$ kubectl get svc/webserver
NAME        TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
webserver   NodePort   10.98.223.206   <none>        80:31275/TCP   4m

$ kubectl get svc/webserver -o yaml
apiVersion: v1
kind: Service
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"webserver","namespace":"default"},"spec":{"ports":[{"port":80}],"selector":{"app":"nginx"}}}
  creationTimestamp: "2023-03-01T14:07:55Z"
  name: webserver
  namespace: default
  resourceVersion: "1128"
  uid: 48daed0e-a16f-4923-bd7e-1d879dc2221f
spec:
  clusterIP: 10.98.223.206
  clusterIPs:
  - 10.98.223.206
  externalTrafficPolicy: Cluster
  internalTrafficPolicy: Cluster
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - nodePort: 31275
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  sessionAffinity: None
  type: NodePort
status:
  loadBalancer: {}

Note that you can change the service type to whatever suits your use case; however, be aware of the implications of certain types, like LoadBalancer, which may trigger the provisioning of public cloud infrastructure components that can be costly if used without awareness and/or monitoring.

4.4 Deploying an Ingress Controller on Minikube

Problem

You want to deploy an ingress controller on Minikube to learn about Ingress objects. Ingress objects are of interest to you because you want to provide access to your applications running in Kubernetes from outside your Kubernetes cluster; however, you do not want to create a NodePort- or LoadBalancer-type service.

Solution

For Ingress objects (discussed in Recipe 4.5) to take effect and provide a route from outside the cluster to your pods, you need to deploy an ingress controller.

On Minikube, enable the ingress add-on like so:

$ minikube addons enable ingress

Once done, you should see ingress appear as enabled in the list of Minikube add-ons. Check that this is the case with:

$ minikube addons list | grep ingress
| ingress | minikube | enabled ✅ | unknown (third-party) |

After a minute or less, a new pod will start in the newly created your ingress-nginx namespace:

$ kubectl get pods -n ingress-nginx
NAME                                       READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create-7r9rs       0/1     Completed   0          3m39s
ingress-nginx-admission-patch-lstj2        0/1     Completed   0          3m39s
ingress-nginx-controller-cc8496874-lhwd5   1/1     Running     0          3m39s

You are now ready to create Ingress objects.

See Also

Ingress documentation

Nginx-based ingress controller

4.5 Making Services Accessible from Outside the Cluster

Problem

You want to access a Kubernetes service from outside of the cluster.

Solution

Use an ingress controller (see Recipe 4.4), which is configured by creating Ingress objects..

First we’ll deploy a simple service that can be invoked and returns hello world. We’ll start by creating the deployment:

$ kubectl create deployment web --image=gcr.io/google-samples/hello-app:1.0

Then we’ll expose the service:

$ kubectl expose deployment web --port=8080

Below is the manifest of an Ingress rule that configures a path to the hello-app service:

$ cat nginx-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-public
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx-example
  rules:
  - host:
    http:
      paths:
      - path: /web
        pathType: Prefix
        backend:
          service:
            name: web
            port:
              number: 8080

$ kubectl create -f nginx-ingress.yaml

Now you can see the Ingress object created for nginx in your Kubernetes dashboard (Figure 4-2).

Figure 4-2. Screenshot of the nginx ingress object

From the Kubernetes dashboard, you can see that nginx will be available via the IP address 192.168.49.2. Based on this information, you can access nginx from outside the cluster now as follows:

$ curl https://192.168.49.2/web
Hello, world!
Version: 1.0.0
Hostname: web-68487bc957-v9fj8

Discussion

In general, ingress works as depicted in Figure 4-3: the ingress controller listens to the /ingresses endpoint of the API server, learning about new rules. It then configures the routes so that external traffic lands at a specific (cluster-internal) service—service1 on port 9876 in the depicted example.

Figure 4-3. Ingress concept

See Also

The kubernetes/ingress-nginx repo on GitHub

Chapter 5. Exploring the Kubernetes API and Key Metadata

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 6th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the author at Oreilly@triggermesh.com.

In this chapter, we present recipes that address the basic interaction with Kubernetes objects as well as the API. Every object in Kubernetes, no matter if namespaced like a deployment or cluster-wide like a node, has certain fields available—​for example, metadata, spec, and status.¹ The spec describes the desired state for an object (the specification), and the status captures the actual state of the object, managed by the Kubernetes API server.

5.1 Discovering the Kubernetes API Server’s Endpoints

Problem

You want to discover the various API endpoints available on the Kubernetes API server.

Solution

Here we assume you’ve spun up a development cluster like Kind or Minikube locally. You can start kubectl proxy in a separate terminal. Proxy lets us easily access the Kubernetes server API with an HTTP client such as curl, without needing to worry about authentication and certificates. After running kubectl proxy, you should be able to reach the API server on port 8001, as shown here:

$ curl http://localhost:8001/api/v1/
{
  "kind": "APIResourceList",
  "groupVersion": "v1",
  "resources": [
    {
      "name": "bindings",
      "singularName": "",
      "namespaced": true,
      "kind": "Binding",
      "verbs": [
        "create"
      ]
    },
    {
      "name": "componentstatuses",
      "singularName": "",
      "namespaced": false,

...

This lists all the objects exposed by the Kubernetes API, and at the top of the list you can see an example of an object of kind Binding as well as the allowed operations on this subject, such as create.

Discussion

You can discover all the API groups by calling the following endpoint:

$ curl http://localhost:8001/apis/
{
  "kind": "APIGroupList",
  "apiVersion": "v1",
  "groups": [
    {
      "name": "apiregistration.k8s.io",
      "versions": [
        {
          "groupVersion": "apiregistration.k8s.io/v1",
          "version": "v1"
        }
      ],
      "preferredVersion": {
        "groupVersion": "apiregistration.k8s.io/v1",
        "version": "v1"
      }
    },
    {
      "name": "apps",
      "versions": [

...

Pick some API groups to explore from this list, such as:

/apis/apps

/apis/storage.k8s.io

/apis/flowcontrol.apiserver.k8s.io

/apis/autoscaling

Each of these endpoints corresponds to an API group. The core API objects are available in the v1 group at /api/v1, whereas other newer API objects are available in named groups under the /apis/` endpoint, such as storage.k8s.io/v1 and apps/v1. Within a group, API objects are versioned (e.g., v1, v2, v1alpha, v1beta1) to indicate the maturity of the objects. Pods, services, config maps, and secrets, for example, are all part of the /api/v1 API group, whereas the /apis/autoscaling group has v1, v2, and v2beta2 versions.

The group an object is part of is what is referred to as the apiVersion in the object specification, available via the API reference.

See Also

Kubernetes API Overview

Kubernetes API Conventions

5.2 Understanding the Structure of a Kubernetes Manifest

Problem

Although Kubernetes does have convenient generators like kubectl run and kubectl create, you must to learn how to write Kubernetes manifests in order to embrace the declarative nature of Kubernetes object specifications. To do this, you need to understand the general structure of manifests.

Solution

In Recipe 5.1, you learned about the various API groups and how to discover which group a particular object is in.

All API resources are either objects or lists. All resources have a kind and an apiVersion. In addition, every object kind must have metadata. The metadata contains the name of the object, the namespace it is in (see Recipe 5.3), and optionally some labels (see Recipe 5.6) and annotations (see Recipe 5.7).

A pod, for example, will be of kind Pod and apiVersion v1, and the beginning of a simple manifest written in YAML will look like this:

apiVersion: v1
kind: Pod
metadata:
  name: mypod
...

To complete a manifest, most objects will have a spec and, once created, will also return a status that describes the current state of the object:

apiVersion: v1
kind: Pod
metadata:
  name: mypod
spec:
  ...
status:
  ...

See Also

Understanding Kubernetes Objects

5.3 Creating Namespaces to Avoid Name Collisions

Problem

You want to create two objects with the same name but want to avoid naming collisions.

Solution

Create two namespaces and create one object in each.

If you don’t specify anything, objects are created in the default namespace. Try creating a second namespace called my-app, as shown here, and list the existing namespaces. You will see the default namespace, other namespaces that were created on startup (kube-system, kube-public, and kube-node-lease) and the my-app namespace you just created:

$ kubectl create namespace my-app
namespace/my-app created

$ kubectl get ns
NAME                 STATUS   AGE
default              Active   5d20h
kube-node-lease      Active   5d20h
kube-public          Active   5d20h
kube-system          Active   5d20h
my-app               Active   13s

Note

Alternatively, you can write a manifest to create your namespace. If you save the following manifest as app.yaml, you can then create the namespace with the kubectl create -f app.yaml command:

apiVersion: v1
kind: Namespace
metadata:
  name: my-app

Discussion

Attempting to start two objects with the same name in the same namespace (e.g., default) leads to a collision, and an error is returned by the Kubernetes API server. However, if you start the second object in a different namespace, the API server will create it:

$ kubectl run foobar --image=ghost:latest
pod/foobar created

$ kubectl run foobar --image=nginx:latest
Error from server (AlreadyExists): pods "foobar" already exists

$ kubectl run foobar --image=nginx:lastest --namespace my-app
pod/foobar created

Note

The kube-system namespace is reserved for administrators, whereas the kube-public namespace is meant to store public objects available to any users of the cluster.

5.4 Setting Quotas Within a Namespace

Problem

You want to limit the resources available in a namespace—​for example, the overall number of pods that can run in the namespace.

Solution

Use a ResourceQuota object to specify the limitations on a namespace basis.

We’ll start by creating a manifest for a resource quota and saving it in a file called resource-quota-pods.yaml:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: podquota
spec:
  hard:
    pods: "10"

Then we’ll create a new namespace and apply the quote to it:

$ kubectl create namespace my-app
namespace/my-app created

$ kubectl create -f resource-quota-pods.yaml --namespace=my-app
resourcequota/podquota created

$ kubectl describe resourcequota podquota --namespace=my-app
Name:       podquota
Namespace:  my-app
Resource    Used  Hard
--------    ----  ----
pods        1     10

Discussion

You can set a number of quotas on a per-namespace basis, including but not limited to pods, secrets, and config maps.

See Also

Configure Quotas for API Objects

5.5 Labeling an Object

Problem

You want to label an object so that you can easily find it later on. The label can be used for further end-user queries (see Recipe 5.6) or in the context of system automation.

Solution

Use the kubectl label command. For example, to label a pod named foobar with the key/value pair tier=frontend, do this:

$ kubectl label pods foobar tier=frontend
pod/foobar labeled

Tip

Check the complete help for the command (kubectl label --help). You can use it to find out how to remove labels, overwrite existing ones, and even label all resources in a namespace.

Discussion

In Kubernetes, you use labels to organize objects in a flexible, nonhierarchical manner. A label is a key/value pair without any predefined meaning for Kubernetes. In other words, the content of the key/value pair is not interpreted by the system. You can use labels to express membership (e.g., object X belongs to department ABC), environments (e.g., this service runs in production), or really anything you need to organize your objects. Note that labels do have restrictions concerning their length and allowed values.²

5.6 Using Labels for Queries

Problem

You want to query objects efficiently.

Solution

Use the kubectl get --selector command. For example, given the following pods:

$ kubectl get pods --show-labels
NAME                                       READY   STATUS    RESTARTS        AGE     LABELS
foobar                                     1/1     Running   0               18m     run=foobar,tier=frontend
nginx1                                     1/1     Running   0               72s     app=nginx,run=nginx1
nginx2                                     1/1     Running   0               68s     app=nginx,run=nginx2
nginx3                                     1/1     Running   0               65s     app=nginx,run=nginx3

You can select the pods that belong to the nginx app (app=nginx):

$ kubectl get pods --selector app=nginx
NAME     READY   STATUS    RESTARTS   AGE
nginx1   1/1     Running   0          3m45s
nginx2   1/1     Running   0          3m41s
nginx3   1/1     Running   0          3m38s

Discussion

Labels are part of an object’s metadata. Any object in Kubernetes can be labeled. Labels are also used by Kubernetes itself for pod selection by deployments (see Recipe 3.1) and services (see Chapter 4).

Labels can be added manually with the kubectl label command (see Recipe 5.5), or you can define labels in an object manifest, like so:

apiVersion: v1
kind: Pod
metadata:
  name: foobar
  labels:
    tier: frontend
...

Once labels are present, you can list them with kubectl get, noting the following:

-l is the short form of --selector and will query objects with a specified key=value pair.

--show-labels will show all the labels of each object returned.

-L will add a column to the results returned with the value of the specified label.

Many object kinds support set-based querying, meaning you can state a query in a form like “must be labelled with X and/or Y.” For example, kubectl get pods -l 'env in (production, development)' would give you pods that are in either the production or development environment.

With two pods running, one with label run=barfoo and the other with label run=foobar, you would get outputs similar to the following:

$ kubectl get pods --show-labels
NAME                      READY   ...    LABELS
barfoo-76081199-h3gwx     1/1     ...    pod-template-hash=76081199,run=barfoo
foobar-1123019601-6x9w1   1/1     ...    pod-template-hash=1123019601,run=foobar

$ kubectl get pods -L run
NAME                      READY   ...    RUN
barfoo-76081199-h3gwx     1/1     ...    barfoo
foobar-1123019601-6x9w1   1/1     ...    foobar

$ kubectl get pods -l run=foobar
NAME                      READY   ...
foobar-1123019601-6x9w1   1/1     ...

See Also

Kubernetes Labels documentation

5.7 Annotating a Resource with One Command

Problem

You want to annotate a resource with a generic, nonidentifying key/value pair, possibly using nonhuman readable data.

Solution

Use the kubectl annotate command:

$ kubectl annotate pods foobar \
description='something that you can use for automation'
pod/foobar annotated

Discussion

Annotations tend to be used for added automation of Kubernetes. For example, when you create a deployment with the kubectl create deployment command and you forget to use the --record option, you will notice that the change-cause column in your rollout history (see Recipe 3.6) is empty. As of Kubernetes v1.6.0, to start recording the commands that cause changes to the deployment, you can annotate it with the kubernetes.io/change-cause key. Given a deployment foobar, you might annotate it with:

$ kubectl annotate deployment foobar \
kubernetes.io/change-cause="Reason for creating a new revision"

Subsequent changes to the deployment will be recorded.

¹ Kubernetes, “Understanding Kubernetes Objects”.

² Kubernetes, “Labels and Selectors: Syntax and character set”.

Chapter 6. Managing Specialized Workloads

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 7th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the author at Oreilly@triggermesh.com.

In Chapter 3, we explored how to launch applications that are supposed to run forever, such as a web server or an app server. In this chapter, we will discuss workloads that are somewhat more specialized—​for example, ones that launch terminating processes such as batch jobs, run pods on certain nodes, or manage stateful and noncloud native apps.

6.1 Running a Batch Job

Problem

You want to run a process that runs for a certain time to completion, such as a batch conversion, backup operation, or database schema upgrade.

Solution

Use a Kubernetes job resource to launch and supervise the pod(s) that will carry out the batch process.¹

First, define the Kubernetes manifest for the job in a file called counter-batch-job.yaml:

apiVersion:          batch/v1
kind:                Job
metadata:
  name:              counter
spec:
  template:
    metadata:
      name:          counter
    spec:
      containers:
      - name:        counter
        image:       busybox
        command:
         - "sh"
         - "-c"
         - "for i in 1 2 3 ; do echo $i ; done"
      restartPolicy: Never

Then launch the job and have a look at its status:

$ kubectl apply -f counter-batch-job.yaml
job.batch/counter created

$ kubectl get jobs
NAME      COMPLETIONS   DURATION   AGE
counter   1/1           7s         12s

$ kubectl describe jobs/counter
Name:             counter
Namespace:        default
Selector:         controller-uid=2d21031e-7263-4ff1-becd-48406393edd5
Labels:           controller-uid=2d21031e-7263-4ff1-becd-48406393edd5
                  job-name=counter
Annotations:      batch.kubernetes.io/job-tracking:
Parallelism:      1
Completions:      1
Completion Mode:  NonIndexed
Start Time:       Mon, 03 Apr 2023 18:19:13 +0530
Completed At:     Mon, 03 Apr 2023 18:19:20 +0530
Duration:         7s
Pods Statuses:    0 Active (0 Ready) / 1 Succeeded / 0 Failed
Pod Template:
  Labels:  controller-uid=2d21031e-7263-4ff1-becd-48406393edd5
           job-name=counter
  Containers:
   counter:
    Image:      busybox
    Port:       <none>
    Host Port:  <none>
    Command:
      sh
      -c
      for i in 1 2 3 ; do echo $i ; done
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Events:
  Type    Reason            Age   From            Message
  ----    ------            ----  ----            -------
  Normal  SuccessfulCreate  30s   job-controller  Created pod: counter-5c8s5
  Normal  Completed         23s   job-controller  Job completed

Finally, you want to verify that it actually carried out the task (counting from 1 to 3):

$ kubectl logs jobs/counter
1
2
3

Indeed, as you can see, the counter job counted as expected.

If you don’t need the job anymore, use kubectl delete jobs/counter to remove it.

6.2 Running a Task on a Schedule Within a Pod

Problem

You want to run a task on a specific schedule within a pod managed by Kubernetes.

Solution

Use Kubernetes CronJob objects. The CronJob object is a derivative of the more generic Job object (see Recipe 6.1).

You can periodically schedule a job by writing a manifest similar to the one shown here. In the spec, you see a schedule section that follows the crontab format. The template section describes the pod that will run and the command that will get executed (it prints the current date and time every hour to stdout):

apiVersion:              batch/v1
kind:                    CronJob
metadata:
  name:                  hourly-date
spec:
  schedule:              "0 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name:        date
            image:       busybox
            command:
              - "sh"
              - "-c"
              - "date"
          restartPolicy: OnFailure

See Also

CronJob documentation

6.3 Running Infrastructure Daemons per Node

Problem

You want to launch an infrastructure daemon—​for example, a log collector or monitoring agent—​making sure that exactly one pod runs per node.

Solution

Use a DaemonSet to launch and supervise the daemon process. For example, to launch a Fluentd agent on each node in your cluster, create a file named fluentd-daemonset.yaml with the following content:

kind:                 DaemonSet
apiVersion:           apps/v1
metadata:
  name:               fluentd
spec:
  selector:
    matchLabels:
      app:            fluentd
  template:
    metadata:
      labels:
        app:          fluentd
      name:           fluentd
    spec:
      containers:
      - name:         fluentd
        image:        gcr.io/google_containers/fluentd-elasticsearch:1.3
        env:
         - name:      FLUENTD_ARGS
           value:     -qq
        volumeMounts:
         - name:      varlog
           mountPath: /varlog
         - name:      containers
           mountPath: /var/lib/docker/containers
      volumes:
         - hostPath:
             path:    /var/log
           name:      varlog
         - hostPath:
             path:    /var/lib/docker/containers
           name:      containers

Now launch the DaemonSet, like so:

$ kubectl apply -f fluentd-daemonset.yaml
daemonset.apps/fluentd created

$ kubectl get ds
NAME      DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
fluentd   1         1         1       1            1           <none>          60s

$ kubectl describe ds/fluentd
Name:           fluentd
Selector:       app=fluentd
Node-Selector:  <none>
Labels:         <none>
Annotations:    deprecated.daemonset.template.generation: 1
Desired Number of Nodes Scheduled: 1
Current Number of Nodes Scheduled: 1
Number of Nodes Scheduled with Up-to-date Pods: 1
Number of Nodes Scheduled with Available Pods: 1
Number of Nodes Misscheduled: 0
Pods Status:  1 Running / 0 Waiting / 0 Succeeded / 0 Failed
...

Discussion

Note that in the preceding output, because the command is executed on Minikube, you only see one pod running as there’s only one node in this setup. If you had 15 nodes in your cluster, you’d have 15 pods overall with 1 pod per node running. You can also restrict the daemon to certain nodes using the nodeSelector section in the spec of the DaemonSet manifest.

6.4 Influencing a Pods’ Startup Behavior

Problem

In order for your pod to function properly, it depends on some other service being available. You want to influence the pod’s startup behavior so that it only starts once the pods it depends on are available.

Solution

Use init containers to influence the startup behavior of a pod.

Imagine you want to launch an nginx web server that depends on a backend service to serve content. You therefore want to make sure that the nginx pod only starts up once the backend service is up and running.

First, create the backend service the web server depends on:

$ kubectl create deployment backend --image=mhausenblas/simpleservice:0.5.0
deployment.apps/backend created
$ kubectl expose deployment backend --port=80 --target-port=9876

Then you can use the following manifest, nginx-init-container.yaml, to launch the nginx instance and make sure it starts up only when the backend deployment serves data:

kind:                 Deployment
apiVersion:           apps/v1
metadata:
  name:               nginx
spec:
  replicas:           1
  selector:
    matchLabels:
      app:            nginx
  template:
    metadata:
      labels:
        app:          nginx
    spec:
      containers:
      - name:         webserver
        image:        nginx
        ports:
        - containerPort: 80
      initContainers:
      - name:         checkbackend
        image:        busybox
        command:      ['sh', '-c', 'until nslookup backend.default.svc.cluster.local; do echo
                       "Waiting for backend to come up"; sleep 3; done; echo
                       "Backend is up, ready to launch web server"']

Now you can launch the nginx deployment and verify whether the init container has done its job by looking at the logs of the pod it is supervising:

$ kubectl apply -f nginx-init-container.yaml
deployment.apps/nginx created

$ kubectl get po
NAME                       READY   STATUS    RESTARTS   AGE
backend-8485c64ccb-99jdh   1/1     Running   0          4m33s
nginx-779d9fcdf6-2ntpn     1/1     Running   0          32s

$ kubectl logs nginx-779d9fcdf6-2ntpn -c checkbackend
Server:   10.96.0.10
Address:  10.96.0.10:53

Name: backend.default.svc.cluster.local
Address: 10.101.119.67

Backend is up, ready to launch web server

As you can see, the command in the init container indeed worked as planned.

¹ Kubernetes, “Jobs - Run to Completion”.

Chapter 7. Volumes and Configuration Data

A Note for Early Release Readers

With Early Release ebooks, you get books in their earliest form—the authors’ raw and unedited content as they write—so you can take advantage of these technologies long before the official release of these titles.

This will be the 8th chapter of the final book. Please note that the GitHub repo will be made active later on.

If you have comments about how we might improve the content and/or examples in this book, or if you notice missing material within this chapter, please reach out to the author at Oreilly@triggermesh.com.

A volume in Kubernetes is a directory accessible to all containers running in a pod, with the additional guarantee that the data is preserved across restarts of individual containers.

We can distinguish between a few types of volumes:

Node-local emphemeral volumes, such as emptyDir or PersistentVolumeClaim

Generic networked volumes, such as nfs or cephfs

Cloud provider–specific volumes, such as AWS EBS or AWS EFS

Special-purpose volumes, such as secret or configMap

Which volume type you choose depends entirely on your use case. For example, for a temporary scratch space, an emptyDir would be fine, but when you need to make sure your data survives node failures you’ll want to look into more resilient alternatives or cloud-provider–specific solutions.

7.1 Exchanging Data Between Containers via a Local Volume

Problem

You have two or more containers running in a pod and want to be able to exchange data via filesystem operations.

Solution

Use a local volume of type emptyDir.

The starting point is the following pod manifest, exchangedata.yaml, which has two containers (c1 and c2) that each mount the local volume xchange into their filesystem, using different mount points:

apiVersion:          v1
kind:                Pod
metadata:
  name:              sharevol
spec:
  containers:
  - name:            c1
    image:           ubuntu:latest
    command:
      - "bin/bash"
      - "-c"
      - "sleep 10000"
    volumeMounts:
      - name:        xchange
        mountPath:   "/tmp/xchange"
  - name:            c2
    image:           ubuntu:latest
    command:
      - "bin/bash"
      - "-c"
      - "sleep 10000"
    volumeMounts:
      - name:        xchange
        mountPath:   "/tmp/data"
  volumes:
  - name:            xchange
    emptyDir:        {}

Now you can launch the pod, exec into it, create data from one container, and read it out from the other one:

$ kubectl create -f exchangedata.yaml
pod/sharevol created

$ kubectl exec sharevol -c c1 -i -t -- bash
[root@sharevol /]# mount | grep xchange
/dev/vda1 on /tmp/xchange type ext4 (rw,relatime)
[root@sharevol /]# echo 'some data' > /tmp/xchange/data
[root@sharevol /]# exit

$ kubectl exec sharevol -c c2 -i -t -- bash
[root@sharevol /]# mount | grep /tmp/data
/dev/vda1 on /tmp/data type ext4 (rw,relatime)
[root@sharevol /]# cat /tmp/data/data
some data
[root@sharevol /]# exit

Discussion

A local volume is backed by the node on which the pod and its containers are running. If the node goes down or you have to carry out maintenance on the it (see Recipe 12.8), then the local volume is gone and all the data is lost.

There are some use cases where local volumes are fine—​for example, for some scratch space or when the canonical state is obtained from somewhere else, such as an S3 bucket—​but in general you’ll want to use a persistent volume or one backed by networked storage (see Recipe 7.4).

See Also

Kubernetes Volumes documentation

7.2 Passing an API Access Key to a Pod Using a Secret

Problem

As an admin, you want to provide your developers with an API access key in a secure way; that is, without sharing it in clear text in your Kubernetes manifests.

Solution

Use a local volume of type secret.

Let’s say you want to give your developers access to an external service that is accessible via the passphrase open sesame.

First, create a text file called passphrase that holds the passphrase:

$ echo -n "open sesame" > ./passphrase

Next, create the secret, using the passphrase file:

$ kubectl create secret generic pp --from-file=./passphrase
secret/pp created

$ kubectl describe secrets/pp
Name:           pp
Namespace:      default
Labels:         <none>
Annotations:    <none>

Type:   Opaque

Data
====
passphrase:     11 bytes

From an admin point of view, you’re all set now and it’s time for your developers to consume the secret. So let’s switch hats and assume you’re a developer and want to use the passphrase from within a pod.

You would consume the secret, for example, by mounting it as a volume into your pod and then reading it out as a normal file. Create and save the following manifest called ppconsumer.yaml:

apiVersion:        v1
kind:              Pod
metadata:
  name:            ppconsumer
spec:
  containers:
  - name:          shell
    image:         busybox
    command:
      - "sh"
      - "-c"
      - "mount | grep access  && sleep 3600"
    volumeMounts:
      - name:      passphrase
        mountPath: "/tmp/access"
        readOnly:  true
  volumes:
  - name:          passphrase
    secret:
      secretName:  pp

Now launch the pod and have a look at its logs, where you would expect to see the pp secret file mounted as /tmp/access/passphrase:

$ kubectl create -f ppconsumer.yaml
pod/ppconsumer created

$ kubectl logs ppconsumer
tmpfs on /tmp/access type tmpfs (ro,relatime,size=7937656k)

To access the passphrase from within the running container, simply read out the passphrase file in /tmp/access, like so:

$ kubectl exec ppconsumer -i -t -- sh

/ # cat /tmp/access/passphrase
open sesame
/ # exit

Discussion

Secrets exist in the context of a namespace, so you need to take that into account when setting them up and/or consuming them.

You can access a secret from a container running in a pod via one of the following:

A volume (as shown in the Solution, where the content is stored in a tmpfs volume)

An environment variable

Also, note that the size of a secret is limited to 1 MB.

Tip

kubectl create secret deals with three types of secrets and, depending on your use case, you might want to choose different ones:

The docker-registry type is for use with a Docker registry.

The generic type is what we used in the Solution; it creates a secret from a local file, directory, or literal value (you need to base64-encode it yourself).

With tls you can create, for example, a secure SSL certificate for ingress.

kubectl describe doesn’t show the content of the secret in the plain text. This avoids “over-the-shoulder” password grabs. You can, however, easily decode it manually since it’s not encrypted—​only base64-encoded:

$ kubectl get secret pp -o yaml | \
  grep passphrase | \
  cut -d":" -f 2 | \
  awk '{$1=$1};1' | \
  base64 --decode
open sesame

In this command, the first line retrieves a YAML representation of the secret, and the second line with the grep pulls out the line passphrase: b3BlbiBzZXNhbWU= (note the leading whitespace here). Then, the cut extracts the content of the passphrase, and the awk command gets rid of the leading whitespace. Finally, the base64 command turns it into the original data again.

Tip

You have the option to encrypt secrets at rest by using the --encryption-provider-config option when launching the kube-apiserver.

See Also

Kubernetes Secrets documentation

Encrypting Secret Data at Rest

7.3 Providing Configuration Data to an Application

Problem

You want to provide configuration data to an application without storing it in the container image or hardcoding it into the pod specification.

Solution

Use a config map. These are first-class Kubernetes resources with which you can provide configuration data to a pod via environment variables or a file.

Let’s say you want to create a configuration with the key siseversion and the value 0.9. It’s as simple as this:

$ kubectl create configmap nginxconfig --from-literal=nginxgreeting="hello from nginx"
configmap/nginxconfig created

Now you can use the config map in a deployment—​say, in a manifest file with the following content:

apiVersion:    v1
kind:          Pod
metadata:
  name:        nginx
spec:
  containers:
  - name:      nginx
    image:     nginx
    env:
    - name: NGINX_GREETING
      valueFrom:
        configMapKeyRef:
          name: nginxconfig
          key: nginxgreeting

Save this YAML manifest in a file called nginxpod.yaml and then use kubectl to create it:

$ kubectl create -f nginxpod.yaml

You can then list the Pod’s container environment variables with the following command:

$ kubectl exec nginx -- printenv
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=nginx
NGINX_GREETING=hello from nginx
KUBERNETES_PORT_443_TCP=tcp://10.96.0.1:443
...

We’ve just shown how to pass in the configuration as an environment variable. However, you can also mount it into the pod as a file, using a volume.

Suppose you have the following config file, example.cfg:

debug: true
home: ~/abc

You can create a config map that holds the config file, as follows:

$ kubectl create configmap configfile --from-file=example.cfg
configmap/configfile created

Now you can use the config map just as you would any other volume. The following is the manifest file for a pod named oreilly; it uses the busybox image and just sleeps for 3,600 seconds. In the volumes section, there is a volume named oreilly which uses the config map configfile that we just created. This volume is then mounted at the path /oreilly inside the container. Hence, the file will be accessible within the pod:

apiVersion:        v1
kind:              Pod
metadata:
  name:            oreilly
spec:
  containers:
  - image:         busybox
    command:
      - sleep
      - "3600"
    volumeMounts:
    - mountPath:   /oreilly
      name:        oreilly
    name:          busybox
  volumes:
  - name:          oreilly
    configMap:
      name:        configfile

After creating the pod, you can verify that the example.cfg file is indeed inside it:

$ kubectl exec -ti oreilly -- ls -l /oreilly
total 0
lrwxrwxrwx   1 root   root   18 Mar 31 09:39 example.cfg -> ..data/example.cfg

$ kubectl exec -ti oreilly -- cat /oreilly/example.cfg
debug: true
home: ~/abc

For a complete example of how to create a config map from a file, see Recipe 11.7.

See Also

Configure a Pod to Use a ConfigMap

7.4 Using a Persistent Volume with Minikube

Problem

You don’t want to lose data on a disk your container uses—​that is, you want to make sure it survives a restart of the hosting pod.

Solution

Use a persistent volume (PV). In the case of Minikube, you can create a PV of type hostPath and mount it just like a normal volume into the container’s filesystem.

First, define the PV hostpathpv in a manifest called hostpath-pv.yaml:

apiVersion:         v1
kind:               PersistentVolume
metadata:
  name:             hostpathpv
  labels:
    type:           local
spec:
  storageClassName: manual
  capacity:
    storage:        1Gi
  accessModes:
  - ReadWriteOnce
  hostPath:
    path:           "/tmp/pvdata"

Before you can create the PV, however, you need to prepare the directory /tmp/pvdata on the node—​that is, the Minikube instance itself. You can get into the node where the Kubernetes cluster is running using minikube ssh:

$ minikube ssh

$ mkdir /tmp/pvdata && \
  echo 'I am content served from a delicious persistent volume' > / \
  tmp/pvdata/index.html

$ cat /tmp/pvdata/index.html
I am content served from a delicious persistent volume

$ exit

Now that you’ve prepared the directory on the node, you can create the PV from the manifest file hostpath-pv.yaml:

$ kubectl create -f hostpath-pv.yaml
persistentvolume/hostpathpv created

$ kubectl get pv
NAME        CAPACITY   ACCESSMODES   RECLAIMPOLICY   STATUS      ...   ...   ...
hostpathpv  1Gi        RWO           Retain          Available   ...   ...   ...

$ kubectl describe pv/hostpathpv
Name:            hostpathpv
Labels:          type=local
Annotations:     <none>
Finalizers:      [kubernetes.io/pv-protection]
StorageClass:    manual
Status:          Available
Claim:
Reclaim Policy:  Retain
Access Modes:    RWO
VolumeMode:      Filesystem
Capacity:        1Gi
Node Affinity:   <none>
Message:
Source:
    Type:          HostPath (bare host directory volume)
    Path:          /tmp/pvdata
    HostPathType:
Events:            <none>

Up to this point, you would carry out these steps in an admin role. You would define PVs and make them available to developers on the Kubernetes cluster.

Now you’re in a position to use the PV in a pod, from a developer’s perspective. This is done via a persistent volume claim (PVC), so called because, well, you literally claim a PV that fulfills certain characteristics, such as size or storage class.

Create a manifest file called pvc.yaml that defines a PVC, asking for 200 MB of space:

apiVersion:         v1
kind:               PersistentVolumeClaim
metadata:
  name:             mypvc
spec:
  storageClassName: manual
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage:      200Mi

Next, launch the PVC and verify its state:

$ kubectl create -f pvc.yaml
persistentvolumeclaim/mypvc created

$ kubectl get pv
NAME        CAPACITY  ACCESSMODES  ...  STATUS  CLAIM          STORAGECLASS
hostpathpv  1Gi       RWO          ...  Bound   default/mypvc  manual

Note that the status of the PV hostpathpv has changed from Available to Bound.

Finally, it’s time to consume the data from the PV in a container, this time via a deployment that mounts it in the filesystem. So, create a file called nginx-using-pv.yaml with the following content:

apiVersion:                    apps/v1
kind:                          Deployment
metadata:
  name:                        nginx-with-pv
spec:
  replicas:                    1
  selector:
    matchLabels:
      app:                     nginx
  template:
    metadata:
      labels:
        app:                   nginx
    spec:
      containers:
      - name:                  webserver
        image:                 nginx
        ports:
        - containerPort:       80
        volumeMounts:
        - mountPath:           "/usr/share/nginx/html"
          name:                webservercontent
      volumes:
      - name:                  webservercontent
        persistentVolumeClaim:
          claimName:           mypvc

And launch the deployment, like so:

$ kubectl create -f nginx-using-pv.yaml
deployment.apps/nginx-with-pv created

$ kubectl get pvc
NAME   STATUS  VOLUME      CAPACITY  ACCESSMODES  STORAGECLASS  AGE
mypvc  Bound   hostpathpv  1Gi       RWO          manual        12m

As you can see, the PV is in use via the PVC you created earlier.

To verify that the data actually has arrived, you could now create a service (see Recipe 4.1) along with an ingress object (see Recipe 4.5) and then access it like so:

$ curl -k -s https://192.168.99.100/web
I am content served from a delicious persistent volume

Well done! You’ve (as an admin) provisioned a persistent volume and (as a developer) claimed it via a persistent volume claim, and used it from a deployment in a pod by mounting it into the container filesystem.

Discussion

In the Solution, we used a persistent volume of type hostPath. In a production setting, you would not want to use this but rather ask your cluster administrator nicely to provision a networked volume backed by NFS or an Amazon Elastic Block Store (EBS) volume to make sure your data sticks around and survives single-node failures.

Note

Remember that PVs are cluster-wide resources; that is, they are not namespaced. However, PVCs are namespaced. You can claim PVs from specific namespaces using namespaced PVCs.

See Also

Kubernetes Persistent Volumes documentation

Configure a Pod to Use a PersistentVolume for Storage

7.5 Understanding Data Persistency on Minikube

Problem

You want to use Minikube to understand how you could deploy a stateful application in Kubernetes. Specifically, you would like to deploy a MySQL database.

Solution

Use a PersistentVolumeClaim object (see Recipe 7.4) in your pod definition and/or the template for your database.

First you need to make a request for a specific amount of storage. The following data.yaml manifest makes a request for 1 GB of storage.

apiVersion:    v1
kind:          PersistentVolumeClaim
metadata:
  name:        data
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi

On Minikube, create this PVC and immediately see how a persistent volume is created to match this claim:

$ kubectl create -f data.yaml
persistentvolumeclaim/data created

$ kubectl get pvc
NAME  STATUS  VOLUME                                    CAPACITY ...  ...  ...
data  Bound   pvc-da58c85c-e29a-11e7-ac0b-080027fcc0e7  1Gi      ...  ...  ...

$ kubectl get pv
NAME                                      CAPACITY  ...  ...  ...  ...  ...
pvc-da58c85c-e29a-11e7-ac0b-080027fcc0e7  1Gi       ...  ...  ...  ...  ...

You are now ready to use this claim in your pod. In the volumes section, define a volume by name with a PVC type and a reference to the PVC you just created. In the volumeMounts field, you’ll mount this volume at a specific path inside your container. For MySQL, you mount it at /var/lib/mysql:

apiVersion:        v1
kind:              Pod
metadata:
  name:            db
spec:
  containers:
  - image:         mysql:latest
    name:          db
    volumeMounts:
    - mountPath:   /var/lib/mysql
      name:        data
    env:
      - name:      MYSQL_ROOT_PASSWORD
        value:     root
  volumes:
  - name:          data
    persistentVolumeClaim:
      claimName:   data

Discussion

Minkube is configured out of the box with a default storage class that defines a default persistent volume provisioner. This means that when a persistent volume claim is created, Kubernetes will dynamically create a matching persistent volume to fill that claim.

This is what happened in the Solution. Once you created the data PVC, Kubernetes automatically created a PV to match that claim. If you look a bit deeper at the default storage class on Minikube, you will see the provisioner type:

$ kubectl get storageclass
NAME                 PROVISIONER
standard (default)   k8s.io/minikube-hostpath

$ kubectl get storageclass standard -o yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
...
provisioner: k8s.io/minikube-hostpath
reclaimPolicy: Delete

This specific storage class is using a storage provisioner that creates persistent volumes of type hostPath. You can see this by looking at the manifest of the PV that got created to match the claim you created previously:

$ kubectl get pv
NAME                                       CAPACITY   ... CLAIM           ...
pvc-da58c85c-e29a-11e7-ac0b-080027fcc0e7   1Gi        ... default/data  ...

$ kubectl get pv pvc-da58c85c-e29a-11e7-ac0b-080027fcc0e7 -o yaml
apiVersion: v1
kind: PersistentVolume
...
  hostPath:
    path: /tmp/hostpath-provisioner/default/data
    type: ""
...

To verify that the host volume created holds the database data, you can connect to Minikube and list the files in the directory:

$ minikube ssh

$ ls -l /tmp/hostpath-provisioner/default/data
total 99688
...
drwxr-x--- 2 999 docker     4096 Mar 31 11:11  mysql
-rw-r----- 1 999 docker 31457280 Mar 31 11:11  mysql.ibd
lrwxrwxrwx 1 999 docker       27 Mar 31 11:11  mysql.sock -> /var/run/mysqld/mysqld.sock
drwxr-x--- 2 999 docker     4096 Mar 31 11:11  performance_schema
-rw------- 1 999 docker     1680 Mar 31 11:11  private_key.pem
-rw-r--r-- 1 999 docker      452 Mar 31 11:11  public_key.pem
...

Indeed, you now have data persistence. If the pod dies (or you delete it), your data will still be available.

In general, storage classes allow the cluster administrator to define the various types of storage they might provide. For the developers, it abstracts the type of storage and lets them use PVC without having to worry about the storage provider itself.

See Also

Persistent Volumes documentation

Storage Classes documentation