ci: Build workhorse binaries and upload/download as generic package
-
Review changes -
-
Download -
Patches
-
Plain diff
Merged
Rémy Coutable requested to merge 351035-research-streamlining-or-eliminating-setup-test-env into master
What does this MR do and why?
This MR does two things:
- Adds a new
build-components
job that runs automatically for all GitLab.comgitlab-org/gitlab
scheduled pipelines, for anymaster
commit that touches theworkhorse/
folder, and is manual for GitLab.com'sgitlab-org
's MRs.- This job tries to download a generic package that contains GitLab Workhorse binaries needed in the GitLab's test suite (under
tmp/tests/gitlab-workhorse
). - If the package URL returns a 404:
- It runs
scripts/setup-test-env
, so that the GitLab Workhorse binaries are built. - It then creates an archive which contains the binaries and upload it as a generic package to https://gitlab.com/gitlab-org/gitlab/-/packages/.
- It runs
- This job tries to download a generic package that contains GitLab Workhorse binaries needed in the GitLab's test suite (under
- Change the
setup-test-env
job to:- First download the GitLab Workhorse generic package build and uploaded by
build-components
. - If the package is retrieved successfully, its content is placed in the right folder (i.e.
tmp/tests/gitlab-workhorse
), preventing the building of the binaries whenscripts/setup-test-env
is run later on. - If the package URL returns a 404, the behavior doesn't change compared to the current one: the GitLab Workhorse binaries are built as part of
scripts/setup-test-env
- First download the GitLab Workhorse generic package build and uploaded by
Notes:
- The diff to achieve this is quite small, and we can see that
TestEnv
didn't have to be touched at all, so this is really an optimization that sits on top of what we have today (i.e. caching of GitLab Workhorse binaries). - The version of the package is the
workhorse
tree SHA (i.e.git rev-parse HEAD:workhorse
). - Since the
workhorse
shouldn't change too often, most of the MRs should be able to take advantage of the pre-built package, which would be built every two hours. - As a further iteration, the package could be downloaded by
TestEnv
directly, so that local GDK instances, and GitPod instances (not sure about that) would benefit from it as well (though, different architectures should be taken in account, see gitlab-development-kit#1391).
Improvements
Duration of scripts/setup-test-env
With package | Duration |
---|---|
No | 92s, 84s, 82s |
Yes | 42s, 44s, 46s |
It looks like we could already save 42 seconds on average for the setup-test-env
job.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR. -
Documentation is up-to-date: #353078 (closed) => !81289 (merged)
Related to #351035 (closed).
Edited by Rémy Coutable
Merge request reports
Compare and
Show latest version
- version 36dd872329
- version 3557047163
- version 3459b13cb5
- version 33bdbf5fed
- version 32298c97e4
- version 31fbcd2d5d
- version 30c986de6b
- version 2990f1f60a
- version 28ca3a9c3e
- version 273a4a54c8
- version 264fda761a
- version 252526d6ca
- version 246954abca
- version 234ff832c2
- version 229f35aca7
- version 21b99f8129
- version 206cc5814a
- version 199926a51b
- version 18e2298bcf
- version 175284de9e
- version 1633a88e0b
- version 150b47b355
- version 1454faf483
- version 13191ff796
- version 128a25bb8c
- version 11dfdb3e47
- version 10e895a4f0
- version 93c6d24ac
- version 806779441
- version 7e2d076ad
- version 6213710f2
- version 5046cfe61
- version 43e5ffc79
- version 3364c6c7c
- version 2f7eae216
- version 172a48fae
- master (base)
- latest versionb48936ac1 commit,
- version 36dd8723291 commit,
- version 35570471631 commit,
- version 3459b13cb51 commit,
- version 33bdbf5fed1 commit,
- version 32298c97e41 commit,
- version 31fbcd2d5d1 commit,
- version 30c986de6b2 commits,
- version 2990f1f60a2 commits,
- version 28ca3a9c3e2 commits,
- version 273a4a54c82 commits,
- version 264fda761a2 commits,
- version 252526d6ca2 commits,
- version 246954abca2 commits,
- version 234ff832c22 commits,
- version 229f35aca72 commits,
- version 21b99f81292 commits,
- version 206cc5814a2 commits,
- version 199926a51b2 commits,
- version 18e2298bcf2 commits,
- version 175284de9e2 commits,
- version 1633a88e0b2 commits,
- version 150b47b3552 commits,
- version 1454faf4832 commits,
- version 13191ff7962 commits,
- version 128a25bb8c2 commits,
- version 11dfdb3e472 commits,
- version 10e895a4f01 commit,
- version 93c6d24ac1 commit,
- version 8067794411 commit,
- version 7e2d076ad1 commit,
- version 6213710f21 commit,
- version 5046cfe611 commit,
- version 43e5ffc791 commit,
- version 3364c6c7c1 commit,
- version 2f7eae2161 commit,
- version 172a48fae1 commit,
Compare changes
- Side-by-side
- Inline
Files
283Loading