Skip to content

Docker build cache and multi-stage builds

I think the way we (don't) use docker build cache in combination with the multi-stage dockerfile is incorrect.

build: ensure-fresh-nid-available Dockerfile
	docker build $(DOCKER_BUILD_CACHE_ARG) --target build -t $(IMAGE_PATH)$(PROJECT_NAME)/build:$(DOCKER_TAG) --build-arg NSO_IMAGE_PATH=$(NSO_IMAGE_PATH) --build-arg NSO_VERSION=$(NSO_VERSION) --build-arg PKG_FILE=$(IMAGE_PATH)$(PROJECT_NAME)/package:$(DOCKER_TAG) .
	docker build $(DOCKER_BUILD_CACHE_ARG) --target testnso -t $(IMAGE_PATH)$(PROJECT_NAME)/testnso:$(DOCKER_TAG) --build-arg NSO_IMAGE_PATH=$(NSO_IMAGE_PATH) --build-arg NSO_VERSION=$(NSO_VERSION) --build-arg PKG_FILE=$(IMAGE_PATH)$(PROJECT_NAME)/package:$(DOCKER_TAG) .
	docker build $(DOCKER_BUILD_CACHE_ARG) --target package -t $(IMAGE_PATH)$(PROJECT_NAME)/package:$(DOCKER_TAG) --build-arg NSO_IMAGE_PATH=$(NSO_IMAGE_PATH) --build-arg NSO_VERSION=$(NSO_VERSION) --build-arg PKG_FILE=$(IMAGE_PATH)$(PROJECT_NAME)/package:$(DOCKER_TAG) .

The skeletons first build the project/build image using the build stage. Using --no-cache is fine in this case because we want to make sure the commands installing things (pyvenv) pull the latest packages, even when our source has not changed. The skeletons then build the other two stages. The testnso and package stage both copy files from the build stage.

In CI on the main branch we set DOCKER_BUILD_CACHE_ARG to --no-cache. This is then passed to the docker build ... --target stage commands listed above. This results in docker running the build stage 3x! It is only necessary to use --no-cache on the build stage. In addition to the process taking longer than it should, it could also produce inconsistencies. Unlikely, but imagine the upstream versions change while the pipeline is running. We would install package-v1 in the testnso stage and package-v2 in the package stage.