Skip to content

Leverage docker buildx for the helper image and build for s390x

Lyubomir Raykov requested to merge helper-buildx into master

What does this MR do?

Leverages docker buildx to build helper images and adds s390x. Still keeps a separate tag for each platform, i.e. does not produce a multiplatform image just yet. This will come with #25869.

It includes the following changes:

  • builds helper binaries for s390x
  • extracts several functions from ci/release_docker_images to ci/docker_buildx_commands and uses them in ci/build_helper_docker.
  • stops using docker create + docker export and leverages --output type=tar instead
  • adds the newly produced s390x image to the released ones
  • unifies all linux images to a single Dockerfile using library/alpine instead of multiarch/alpine. Note: this only changes including COPY ./apk-repositories /etc/apk/repositories for x86_64, though I think this is fine.

Why was this MR needed?

#25795 (closed)

What's the best way to test this MR?

container-diff

One way to inspect the new images is to use container-diff. i.e. you download an image from this MR, e.g. https://gitlab.com/gitlab-org/gitlab-runner/-/jobs/602977035/artifacts/browse/out/helper-images/ and one built before it.

Then you do something like:

docker import new-helper-image.tar.xz runner-helper:new
docker import old-helper-image.tar.xz runner-helper:old
container-diff diff daemon://runner-helper:old daemon://runner-helper:new --type file

Z/OS VM

IBM provided a Z/OS Ubuntu VM for us which can be accessed using the credentials in our 1Password Vault under: "zOS login":

If you download one of images produced in the pipeline:

# copy the image over
scp -i <keyname>.pem prebuilt-s390x.tar.xz <user>@<address>:/home/ubuntu/

# ssh in the VM
ssh -i gitlabkey.pem <user>@<address>

# import the image and run it
sudo docker import ./prebuilt-s390x.tar.xz gitlab/gitlab-runner-helper:s390x-d2ea80a8
sudo docker run -it gitlab/gitlab-runner-helper:s390x-d2ea80a8 bash
gitlab-runner-helper help

Running a job

There's a zos runner binary copied in the VM with the same revision as the docker import above:

sudo ./gitlab-runner-linux-s390x run -c ~/.gitlab-runner/zos-lyubo.toml

Here's a job executed with it: https://gitlab.com/lraykov/ci-yaml-play/-/jobs/626833986

What are the relevant issue numbers?

Closes #25795 (closed) Closes #25487 (closed)

Edited by Lyubomir Raykov

Merge request reports