Docs feedback: Surfacing Docker and Kubernetes Executor Performance Tuning Info

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

Problem to solve

Executors that use docker images have some significant performance impacts and bandwidth costs depending on how they are tuned. This includes both builds that use containers for building software and Docker in Docker (DIND) builds of application containers.

The tuning is discussed in these places:

• https://docs.gitlab.com/runner/executors/docker.html#how-pull-policies-work
• https://docs.gitlab.com/runner/executors/docker.html#the-builds-and-cache-storage
• https://docs.gitlab.com/runner/executors/docker.html#clearing-docker-cache

but it does not list which executors are in scope.

Most notably the Kubernetes executor also obeys this setting, along with Docker and Docker-Machine executors.

Kubernetes documentation only mentions the setting in passing and is missing the nice elaboration on the docker executor page: https://docs.gitlab.com/runner/executors/kubernetes.html#the-keywords

The consequences to the default pull policy being "always" is high for both regular builds and DIND and it affects build time as well as bandwidth for pulling images from docker hub for each stage that requires a given image.

Precaching commonly used images when an individual has control over runner build automation is also a plus.

More Background

If this documentation concern was broadened beyond containers to "runner farm performance" then caching and artifacts from here would also be very helpful information: https://docs.gitlab.com/ee/ci/caching/#cache-vs-artifacts. Especially for customers doing runner farms against GitLab.com where caching might need to be favored for ephemeral pipeline artifacts that don't need persistence after the build. In the context of a customer runner farm off of Gitlab.com, it would also be helpful to note the flexibility of distributed runner caches - and colocating them to the customer's runner farm: https://docs.gitlab.com/runner/configuration/autoscale.html#distributed-runners-caching

Proposal(s)

1. Create an explicit list of executors that the pull policy applies to (even if their details are in another document)

Alternates (with above bullet done for all alternates):

1. Include the updated pull policy documentation section under both the Kubernetes and Docker executor pages (via an included file? Harder for the average person to find where include file content is for future editing)
2. Break out this information in to either a "Docker Image Settings and Optimizations" or "Container Build Optimizations" page and reference it from both executor pages.

Who can address the issue

One or more members of the verify team should be involved in collaboration and documentation merge approvals.