Skip to content

Resolve "Shorten `build-qa-image` by optimizing Docker build"

Albert Salim requested to merge 42647-shorten-build-qa-image-by-optimizing-docker-build into master

What does this MR do?

  • Pull qa docker image built from master
  • Rebuild a new qa docker image using pulled master qa image as cache

This would enable docker to build image using cached layers.

When nothing changed in the QA parts of the docker image, it reduced the build time from ~4 minutes to ~10 seconds

Before: https://gitlab.com/gitlab-org/gitlab/-/jobs/377209644

160 Step 2/24 : LABEL maintainer "Grzegorz Bizon <grzegorz@gitlab.com>"
161  ---> Running in ba0e889f663e
162 Removing intermediate container ba0e889f663e
163  ---> 165d96719d75
164 Step 3/24 : ENV DEBIAN_FRONTEND noninteractive
165  ---> Running in 89458829895c

1079 Successfully built 769f8767d8ad
1080 Successfully tagged registry.gitlab.com/gitlab-org/gitlab/gitlab/gitlab-ee-qa:master
1081 real	3m57.847s
1082 user	0m10.042s
1083 sys	0m0.506s

After: https://gitlab.com/gitlab-org/gitlab/-/jobs/377293323

274 Step 2/24 : LABEL maintainer "Grzegorz Bizon <grzegorz@gitlab.com>"
275  ---> Using cache
276  ---> b7c834a35e04
277 Step 3/24 : ENV DEBIAN_FRONTEND noninteractive
278  ---> Using cache
279  ---> 7e07886179b7
280 Step 4/24 : RUN echo "deb http://ftp.debian.org/debian stretch-backports main" >> /etc/apt/sources.list
281  ---> Using cache
282  ---> 25f4c5efec5d

343 Successfully built fa4bf71b5351
344 Successfully tagged registry.gitlab.com/gitlab-org/gitlab/gitlab/gitlab-ee-qa:42647-shorten-build-qa-image-by-optimizing-docker-build
345 real	0m9.675s
346 user	0m11.607s
347 sys	0m0.400s

The example above is the best case scenario when nothing changed in any of the following:

WORKDIR /home/gitlab/qa
COPY ./qa/Gemfile* /home/gitlab/qa/
COPY ./config/initializers/0_inject_enterprise_edition_module.rb /home/gitlab/config/initializers/
# Copy VERSION to ensure the COPY succeeds to copy at least one file since ee/app/models/license.rb isn't present in FOSS
# The [b] part makes ./ee/app/models/license.r[b] a pattern that is allowed to return no files (which is the case in FOSS)
COPY VERSION ./ee/app/models/license.r[b] /home/gitlab/ee/app/models/
COPY ./lib/gitlab.rb /home/gitlab/lib/
COPY ./lib/gitlab/utils.rb /home/gitlab/lib/gitlab/
COPY ./INSTALLATION_TYPE ./VERSION /home/gitlab/
RUN cd /home/gitlab/qa/ && bundle install --jobs=$(nproc) --retry=3 --quiet
COPY ./qa /home/gitlab/qa

Screenshots

Does this MR meet the acceptance criteria?

Conformity

Closes #42647 (closed)

Edited by Albert Salim

Merge request reports