Build containers with AutoDevOps without requiring DinD and privileged runners
With the changes in %11.2, GitLab now supports Kaniko. We should swap out docker build for kaniko in Auto DevOps, so we reduce the need for a privileged runner.
Proposal
Let's try to see if buildah works and what would fail with it. An initial test shows that it can be used as a drop-in replacement for `docker.
Proposal 1
Due to the issues with Kaniko (below), I propose we allow Kaniko as an opt-in when user sets AUTO_DEVOPS_KANIKO.
Proposal 2
Use buildah or img to build container images.
Proposal 3
Use Podman with Buildpacks to build images.
Issues With Kaniko
There are some issues with using kaniko builds that need to be resolved before we can think about trying kaniko again:
- Slow filesystem snapshots https://gitlab.com/gitlab-org/gitlab-ce/issues/50313#note_157460108
- sudo permissions
sudo: /usr/bin/sudo must be owned by uid 0 and have the setuid bit set error building image: error building(https://gitlab.com/gitlab-org/gitlab-ce/issues/60064) - Kaniko does not support the way we pass secrets to docker build (#23706 (comment 229768496))
- Incompatibility with
AUTO_DEVOPS_BUILD_IMAGE_FORWARDED_CI_VARIABLES
Issues with Buildah
Buildah specializes in building OCI images. Buildah's commands replicate all of the commands that are found in a Dockerfile. This allows building images with and without Dockerfiles while not requiring any root privileges. Buildah’s ultimate goal is to provide a lower-level coreutils interface to build images. The flexibility of building images without Dockerfiles allows for the integration of other scripting languages into the build process. Buildah follows a simple fork-exec model and does not run as a daemon but it is based on a comprehensive API in golang, which can be vendored into other tools. (source)
TBD