This is a fourth post in a series, describing the problems my team has faced during implementation of Jenkins pipelines in Kubernetes.
- Jenkins Java Centric Pipelines in Kubernetes
- Building Maven Projects in Jenkins Docker workers
- Using Maven and Jenkins to perform modular Java builds
- Building Docker Images in Jenkins on Kubernetes (This article)
Why would we want to build Docker images in Kubernetes
Often Kubernetes is the platform of choice both for the production environments and the CI(/CD) pipelines that we use for our development processes.
Since Docker images are the basic building blocks, we must be able to build them from within our Kubernetes hosted pipeline.
Ways for building Docker images from within Kubernetes
There are multiple ways and tools that we can use for building Docker images from our Kubernetes based CI, and they were not created equal.
I’ve compiled below a list of various ways you can build Docker images in you CI pipeline with advantages and drawbacks of each method.
Mount the host machine Docker socket
When mounting the host machine Docker socket from within the Kubernetes pod, it allows the Docker client or other tool to start a build on the host machine Docker daemon.
| Pros | Cons |
|---|---|
| Easy to set up. | Security opening - allowing attacker root permissions on the Kubernetes / Docker host from a compromised instance. |
| Guaranteed to be compatible with the OCI standard. | Performance degradation when several builders operate on the same host. |
| Familiar to developers, since it uses the standard Dockerfile. | Possible collision with the Kubernetes engine. |
Run Docker daemon inside a Docker container (build node)
In this scenario, we install a Docker daemon inside a Docker container. While avoiding using the host machine Docker daemon, and avoiding related security risks, one must run it in --privileged mode adding additional security risks.
Further details are available in Jérôme Petazzoni article “Using Docker-in-Docker for your CI or testing environment? Think twice." and Daniel Weibel article “Docker in Docker?"
| Pros | Cons |
|---|---|
| Avoid the drawbacks of sharing the host Docker daemon. | Linux security policies collisions. |
| Possible file system collisions. | |
| Possible data corruption. | |
| Hard to maintain. |
Build Docker images without a Docker daemon
There are several projects that enable building Docker images without using a Docker daemon.
Google Jib
Google Jib project builds Docker and OCI images for Java applications. It is available as a Maven and Gradle plugins.
Jib allows unique Java related features, like separating various components of what was previously inside an uber-jar, to a separate layers, thus improving efficiency in consecutive updates. It provides a rich feature set, however it imposes some limitations by design, like it expects a ready docker image for any system packages.
| Pros | Cons |
|---|---|
| No Docker daemon is required | Some features require separate base image build. |
| Faster and more efficient consecutive builds |
Google Kaniko
Google Kaniko project is specifically designed to build Docker images from a Dockerfile inside a Kubernetes cluster. Can obtain the context from Git, online storage (S3 or GCS) or a local directory. Can store image caches in a persisted volume. Requires root.
| Pros | Cons |
|---|---|
| Uses standard Dockerfile. | Requires root permissions. |
| Can use context from an online storage. | |
| Can run in cloud native builders. |
RedHat fabric8.io
RedHat fabric8.io Maven plugin has been developed by RedHat as an opinionated Java centric build and deployment tool with Kubernetes and OpenShift as target environments in mind. Supports Kubernetes and OpenShift deployment yaml files. It is possible to generate Docker images via standard Docker build and by using Jib.
| Pros | Cons |
|---|---|
| Flexible - allows working with Docker daemon and without it. | Might be an overkill. |
| One shop stop for image creation and deployment to Kubernetes / Openshift. |
Buildah
Buildah project specializes in building OCI images. It uses its own set of commands that replicate all the commands found in Dockerfile. The ability to build images without using a Dockerfile, allow incorporate building images in scripts.
| Pros | Cons |
|---|---|
| Doesn’t need Docker daemon. | Image building instructions a messy and hard to read. |
| Allows seamlessly incorporating image commands in scripts. |
Google Bazel
Google Bazel is a multi-language scalable and extensible build tool. The component for building Docker images is called Docker rules.
| Pros | Cons |
|---|---|
| Doesn’t require Docker daemon. | Doesn’t provide complete language specific package and dependency system. |
| Highly scalable. | |
| Good for polyglot environment. |
Use a cloud native build solution
Using one of the cloud providers managed services for building Docker images, like the Google Cloud Build.
| Pros | Cons |
|---|---|
| Easy to set up. | Low level of control. |
| Highly scalable. | Tight coupling to a specific cloud provider. |