Skip to content

Improve windows helper images build

Alexander Kutelev requested to merge kutelev/gitlab-runner:improve-windows-helper-images-build into master

What does this MR do?

Improves/optimizes a process of building Windows helper Docker images.

Why was this MR needed?

  1. bleeding edge servercore**** docker images jobs require only artifacts from a single job: helper images. But instead of retrieving artifacts just from this job, artifacts from 27 different are downloaded. It causes extra network traffic, prolongs a job execution. It becomes a real problem when helper image is tried to be built in a forked repository using a runner which is far away from GitLab infrastructure.
  2. At the moment docker build command is invoked without --no-cache argument, as result some commands are not really executed but cache is used instead, for example I see it at the following job: https://gitlab.com/gitlab-org/gitlab-runner/-/jobs/266359366 
Step 1/20 : FROM mcr.microsoft.com/windows/servercore:1803 as builder
 ---> 072c57eb11e2
Step 2/20 : SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
 ---> Using cache
 ---> 34fbfe932cec
Step 3/20 : ARG GIT_VERSION
 ---> Using cache
 ---> 19c03fe5ea5a
Step 4/20 : ARG GIT_VERSION_BUILD
 ---> Using cache
 ---> f9057e408100
Step 5/20 : ARG GIT_256_CHECKSUM
 ---> Using cache
 ---> 3f50a32fb6f4
Step 6/20 : ARG GIT_LFS_VERSION
 ---> Using cache
 ---> b95202044fd4
Step 7/20 : ARG GIT_LFS_256_CHECKSUM
 ---> Using cache
 ---> ac114eddcd99
Step 8/20 : RUN [Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor [Net.SecurityProtocolType]::Tls12;     Invoke-Webrequest "https://github.com/git-for-windows/git/releases/download/v${Env:GIT_VERSION}.windows.${Env:GIT_VERSION_BUILD}/MinGit-${Env:GIT_VERSION}-64-bit.zip" -OutFile git.zip -UseBasicParsing;     Invoke-Webrequest "https://github.com/git-lfs/git-lfs/releases/download/v${Env:GIT_LFS_VERSION}/git-lfs-windows-amd64-v${Env:GIT_LFS_VERSION}.zip" -OutFile git-lfs.zip -UseBasicParsing
 ---> Using cache
 ---> 94cf987edbac
Step 9/20 : COPY [".\\helpers\\checksum.ps1", ".\\"]
 ---> Using cache
 ---> e2b6bbbedac1

Taking result of commands like Invoke-Webrequest from the cache may lead to unpredictable consequences. URLs are constructed using environment variables, values of these variables can be changed and docker will not notice it and a cached result will be used instead instead of rerunning a command.

Does this MR meet the acceptance criteria?

  • Documentation created/updated
  • Added tests for this feature/bug
  • In case of conflicts with master - branch was rebased

What are the relevant issue numbers?

-

Merge request reports