Skip to content

Allow specifying image platform

Muhammed Ali requested to merge gitlab-community/gitlab:image-platform into master

What does this MR do and why?

Allows users to specify the platform of the image used during pull. See: gitlab-runner!3916 (merged)

Related to:

Business Rationale:

  • ARM architecture on some clouds is MUCH more cost effective (and therefore sustainable) - for instance, AWS Graviton 3 uses 60% less energy and costs 20% less than x86 equivalents (https://aws.amazon.com/ec2/graviton/). This could reduce the carbon footprint of gitlab.com and therefore support ESG as well as do the same for customers.
  • Since much of GitLab standard DevOps componentry (e.g. SAST) only supports x86 containers, enabling platform switching would enable all those containers without having to maintain them in both arches - improving people efficiency at both GitLab and customers who have the desire not to maintain containers in multiple hardware arches.
  • Customers would not have to run multiple runners to access both arches - reducing management overheads.
  • GitLab, ARM and a major cloud company are all going to be focusing on ARM runners for vECU development to support the Software Defined Vehicle (SDV) and GitLab's Automotive Working Group.

How to set up and validate locally

Requires a GitLab Runner, See: gitlab-runner!3916 (merged)

Setup

Steps to reproduce

  • Register runner as docker executor
  • Runner configuration:
concurrent = 1
check_interval = 0
shutdown_timeout = 0
[session_server]
  session_timeout = 1800

[[runners]]
  name = "test-runner"
  url = "https://3000-gitlabcommunity-gitlab-xxxx.ws-eu102.gitpod.io"
  id = 27
  token = "glrt-xxxx"
  token_obtained_at = 2023-07-27T20:59:41Z
  token_expires_at = 0001-01-01T00:00:00Z
  executor = "docker"
  [runners.cache]
    MaxUploadedArchiveSize = 0
  [runners.docker]
    tls_verify = false
    image = "ruby:2.7"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache"]
    shm_size = 0
  • start runner (DOCKER_HOST=... ./out/binaries/gitlab-runner run -c ~/.gitlab-runner/config.toml)
  • Create Project in GitLab, create a ci file

Contents of .gitlab-ci.yml

stages: 
  - build

build-job:
  stage: build
  image: 
      name: mysql:5.7
      docker:
          platform: linux/amd64
  script:
    - uname -m
    - uname -a

build-job-arm64:
  stage: build
  image: 
      name: mysql:8
      docker:
          platform: arm64
  script:
    - uname -m
    - uname -a

alpine-amd64: 
  stage: build
  image: 
      name: alpine
      docker:
          platform: amd64
  script:
    - uname -m
    - uname -a
alpine-arm64: 
  stage: build
  image: 
      name: alpine
      docker:
          platform: arm64
  script:
    - uname -m
    - uname -a

alpine-default: 
  stage: build
  image: 
      name: alpine
  script:
    - uname -m
    - uname -a

Job Logs

alpine-arm64 

[0KRunning with gitlab-runner 16.3.0~beta.16.gf786bae9 (f786bae9)[0;m
[0K  on test-runner DYKPsvmHx, system ID: s_15cdb5e965f4[0;m
section_start:1690495093:prepare_executor
[0K[0K[36;1mPreparing the "docker" executor[0;m[0;m
[0KUsing Docker executor with image alpine ...[0;m
[0KPulling docker image alpine ...[0;m
[0KUsing docker image sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1 for alpine with digest alpine@sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1 ...[0;m
section_end:1690495103:prepare_executor
[0Ksection_start:1690495103:prepare_script
[0K[0K[36;1mPreparing environment[0;m[0;m
Running on runner-dykpsvmhx-project-19-concurrent-0 via My-Macbook-Pro.local...
section_end:1690495103:prepare_script
[0Ksection_start:1690495103:get_sources
[0K[0K[36;1mGetting source from Git repository[0;m[0;m
[32;1mFetching changes with git depth set to 20...[0;m
Initialized empty Git repository in /builds/root/test/.git/
[32;1mCreated fresh repository.[0;m
[32;1mChecking out df24c567 as detached HEAD (ref is main)...[0;m

[32;1mSkipping Git submodules setup[0;m
section_end:1690495104:get_sources
[0Ksection_start:1690495104:step_script
[0K[0K[36;1mExecuting "step_script" stage of the job script[0;m[0;m
[0KUsing docker image sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1 for alpine with digest alpine@sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1 ...[0;m
[32;1m$ uname -m[0;m
aarch64
[32;1m$ uname -a[0;m
Linux runner-dykpsvmhx-project-19-concurrent-0 5.15.49-linuxkit-pr #1 SMP PREEMPT Thu May 25 07:27:39 UTC 2023 aarch64 Linux
section_end:1690495104:step_script
[0Ksection_start:1690495104:cleanup_file_variables
[0K[0K[36;1mCleaning up project directory and file based variables[0;m[0;m
section_end:1690495104:cleanup_file_variables
[0K[32;1mJob succeeded[0;m

Image


REPOSITORY                                                          TAG               DIGEST                                                                    IMAGE ID       CREATED          SIZE
alpine                                                              latest            sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1   82d1e9d7ed48   5 minutes ago   11.7MB
alpine-amd64 

[0KRunning with gitlab-runner 16.3.0~beta.16.gf786bae9 (f786bae9)[0;m
[0K  on test-runner DYKPsvmHx, system ID: s_15cdb5e965f4[0;m
section_start:1690499380:prepare_executor
[0K[0K[36;1mPreparing the "docker" executor[0;m[0;m
[0KUsing Docker executor with image alpine ...[0;m
[0KPulling docker image alpine ...[0;m
[0;33mWARNING: Failed to pull image with policy "always": Error response from daemon: content digest sha256:e3bd82196e98898cae9fe7fbfd6e2436530485974dc4fb3b7ddb69134eda2407: not found (manager.go:246:0s)[0;m
[31;1mERROR: Preparation failed: failed to pull image "alpine" with specified policies [always]: Error response from daemon: content digest sha256:e3bd82196e98898cae9fe7fbfd6e2436530485974dc4fb3b7ddb69134eda2407: not found (manager.go:246:0s)[0;m
[32;1mWill be retried in 3s ...[0;m
[0KUsing Docker executor with image alpine ...[0;m
[0KPulling docker image alpine ...[0;m
[0;33mWARNING: Failed to pull image with policy "always": Error response from daemon: content digest sha256:e3bd82196e98898cae9fe7fbfd6e2436530485974dc4fb3b7ddb69134eda2407: not found (manager.go:246:0s)[0;m
[31;1mERROR: Preparation failed: failed to pull image "alpine" with specified policies [always]: Error response from daemon: content digest sha256:e3bd82196e98898cae9fe7fbfd6e2436530485974dc4fb3b7ddb69134eda2407: not found (manager.go:246:0s)[0;m
[32;1mWill be retried in 3s ...[0;m
[0KUsing Docker executor with image alpine ...[0;m
[0KPulling docker image alpine ...[0;m
[0;33mWARNING: Failed to pull image with policy "always": Error response from daemon: content digest sha256:e3bd82196e98898cae9fe7fbfd6e2436530485974dc4fb3b7ddb69134eda2407: not found (manager.go:246:0s)[0;m
[31;1mERROR: Preparation failed: failed to pull image "alpine" with specified policies [always]: Error response from daemon: content digest sha256:e3bd82196e98898cae9fe7fbfd6e2436530485974dc4fb3b7ddb69134eda2407: not found (manager.go:246:0s)[0;m
[32;1mWill be retried in 3s ...[0;m
section_end:1690499402:prepare_executor
[0K[31;1mERROR: Job failed (system failure): failed to pull image "alpine" with specified policies [always]: Error response from daemon: content digest sha256:e3bd82196e98898cae9fe7fbfd6e2436530485974dc4fb3b7ddb69134eda2407: not found (manager.go:246:0s)[0;m

Image


REPOSITORY                                                          TAG               DIGEST                                                                    IMAGE ID       CREATED          SIZE
alpine                                                              latest            sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1   82d1e9d7ed48   10 minutes ago   11.4MB
alpine-default 

[0KRunning with gitlab-runner 16.3.0~beta.16.gf786bae9 (f786bae9)[0;m
[0K  on test-runner DYKPsvmHx, system ID: s_15cdb5e965f4[0;m
section_start:1690495640:prepare_executor
[0K[0K[36;1mPreparing the "docker" executor[0;m[0;m
[0KUsing Docker executor with image alpine ...[0;m
[0KPulling docker image alpine ...[0;m
[0KUsing docker image sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1 for alpine with digest alpine@sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1 ...[0;m
section_end:1690495647:prepare_executor
[0Ksection_start:1690495647:prepare_script
[0K[0K[36;1mPreparing environment[0;m[0;m
Running on runner-dykpsvmhx-project-19-concurrent-0 via My-Macbook-Pro.local...
section_end:1690495647:prepare_script
[0Ksection_start:1690495647:get_sources
[0K[0K[36;1mGetting source from Git repository[0;m[0;m
[32;1mFetching changes with git depth set to 20...[0;m
Reinitialized existing Git repository in /builds/root/test/.git/
[32;1mChecking out df24c567 as detached HEAD (ref is main)...[0;m

[32;1mSkipping Git submodules setup[0;m
section_end:1690495648:get_sources
[0Ksection_start:1690495648:step_script
[0K[0K[36;1mExecuting "step_script" stage of the job script[0;m[0;m
[0KUsing docker image sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1 for alpine with digest alpine@sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1 ...[0;m
[32;1m$ uname -m[0;m
aarch64
[32;1m$ uname -a[0;m
Linux runner-dykpsvmhx-project-19-concurrent-0 5.15.49-linuxkit-pr #1 SMP PREEMPT Thu May 25 07:27:39 UTC 2023 aarch64 Linux
section_end:1690495649:step_script
[0Ksection_start:1690495649:cleanup_file_variables
[0K[0K[36;1mCleaning up project directory and file based variables[0;m[0;m
section_end:1690495649:cleanup_file_variables
[0K[32;1mJob succeeded[0;m

Image


alpine                                                              latest            sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1   82d1e9d7ed48   5 minutes ago   11.7MB
build-job-arm64 

[0KRunning with gitlab-runner 16.3.0~beta.16.gf786bae9 (f786bae9)[0;m
[0K  on test-runner DYKPsvmHx, system ID: s_15cdb5e965f4[0;m
section_start:1690496040:prepare_executor
[0K[0K[36;1mPreparing the "docker" executor[0;m[0;m
[0KUsing Docker executor with image mysql:8 ...[0;m
[0KPulling docker image mysql:8 ...[0;m
[0KUsing docker image sha256:51c4dc55d3abf4517a5a652794d1f0adb2f2ed1d1bedc847d6132d91cdb2ebbf for mysql:8 with digest mysql@sha256:51c4dc55d3abf4517a5a652794d1f0adb2f2ed1d1bedc847d6132d91cdb2ebbf ...[0;m
section_end:1690496098:prepare_executor
[0Ksection_start:1690496098:prepare_script
[0K[0K[36;1mPreparing environment[0;m[0;m
Running on runner-dykpsvmhx-project-19-concurrent-0 via My-Macbook-Pro.local...
section_end:1690496099:prepare_script
[0Ksection_start:1690496099:get_sources
[0K[0K[36;1mGetting source from Git repository[0;m[0;m
[32;1mFetching changes with git depth set to 20...[0;m
Reinitialized existing Git repository in /builds/root/test/.git/
[32;1mChecking out 4b416a13 as detached HEAD (ref is main)...[0;m

[32;1mSkipping Git submodules setup[0;m
section_end:1690496100:get_sources
[0Ksection_start:1690496100:step_script
[0K[0K[36;1mExecuting "step_script" stage of the job script[0;m[0;m
[0KUsing docker image sha256:51c4dc55d3abf4517a5a652794d1f0adb2f2ed1d1bedc847d6132d91cdb2ebbf for mysql:8 with digest mysql@sha256:51c4dc55d3abf4517a5a652794d1f0adb2f2ed1d1bedc847d6132d91cdb2ebbf ...[0;m
[32;1m$ uname -m[0;m
aarch64
[32;1m$ uname -a[0;m
Linux runner-dykpsvmhx-project-19-concurrent-0 5.15.49-linuxkit-pr #1 SMP PREEMPT Thu May 25 07:27:39 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux
section_end:1690496100:step_script
[0Ksection_start:1690496100:cleanup_file_variables
[0K[0K[36;1mCleaning up project directory and file based variables[0;m[0;m
section_end:1690496100:cleanup_file_variables
[0K[32;1mJob succeeded[0;m

Image


REPOSITORY                                                          TAG               DIGEST                                                                    IMAGE ID       CREATED          SIZE
mysql                                                               8                 sha256:51c4dc55d3abf4517a5a652794d1f0adb2f2ed1d1bedc847d6132d91cdb2ebbf   51c4dc55d3ab   21 seconds ago       812MB
build-job 

[0KRunning with gitlab-runner 16.3.0~beta.16.gf786bae9 (f786bae9)[0;m
[0K  on test-runner DYKPsvmHx, system ID: s_15cdb5e965f4[0;m
section_start:1690498707:prepare_executor
[0K[0K[36;1mPreparing the "docker" executor[0;m[0;m
[0KUsing Docker executor with image mysql:5.7 ...[0;m
[0KPulling docker image mysql:5.7 ...[0;m
[0KUsing docker image sha256:2eabad08824e3120dbec9096c276e3956e1922636c06fbb383ae9ea9c499bf43 for mysql:5.7 with digest mysql@sha256:2eabad08824e3120dbec9096c276e3956e1922636c06fbb383ae9ea9c499bf43 ...[0;m
section_end:1690498859:prepare_executor
[0Ksection_start:1690498859:prepare_script
[0K[0K[36;1mPreparing environment[0;m[0;m
Running on runner-dykpsvmhx-project-19-concurrent-0 via Muhammeds-MacBook-Pro.local...
section_end:1690498860:prepare_script
[0Ksection_start:1690498860:get_sources
[0K[0K[36;1mGetting source from Git repository[0;m[0;m
[32;1mFetching changes with git depth set to 20...[0;m
Reinitialized existing Git repository in /builds/root/test/.git/
[32;1mChecking out 15969289 as detached HEAD (ref is main)...[0;m

[32;1mSkipping Git submodules setup[0;m
section_end:1690498861:get_sources
[0Ksection_start:1690498861:step_script
[0K[0K[36;1mExecuting "step_script" stage of the job script[0;m[0;m
[0KUsing docker image sha256:2eabad08824e3120dbec9096c276e3956e1922636c06fbb383ae9ea9c499bf43 for mysql:5.7 with digest mysql@sha256:2eabad08824e3120dbec9096c276e3956e1922636c06fbb383ae9ea9c499bf43 ...[0;m
[32;1m$ uname -m[0;m
x86_64
[32;1m$ uname -a[0;m
Linux runner-dykpsvmhx-project-19-concurrent-0 5.15.49-linuxkit-pr #1 SMP PREEMPT Thu May 25 07:27:39 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
section_end:1690498861:step_script
[0Ksection_start:1690498861:cleanup_file_variables
[0K[0K[36;1mCleaning up project directory and file based variables[0;m[0;m
section_end:1690498861:cleanup_file_variables
[0K[32;1mJob succeeded[0;m

Image


REPOSITORY                                                          TAG               DIGEST                                                                    IMAGE ID       CREATED          SIZE
mysql                                                               5.7               sha256:2eabad08824e3120dbec9096c276e3956e1922636c06fbb383ae9ea9c499bf43   2eabad08824e   47 seconds ago   807MB

Conclusion

As you can see from the build-job logs, the output of the architecture is x86_64 which is expected. So it does work.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by DarwinJS

Merge request reports