Add CI variable GLCI_MEDIUM_RUNNER_REQUIRED
What does this MR do and why?
Add CI variable GLCI_MEDIUM_RUNNER_REQUIRED
- Larger runners are required to reliably run newer chrome versions.
- See Use Chrome 138 in system tests (specs/features) (!204309 - merged) • Chad Woolley, Jay McCure • 18.6
References
- Issue to track runner upgrade information and approvals: https://gitlab.com/gitlab-org/gitlab/-/issues/573287+s
- Use Chrome 138 in system tests (specs/features) (!204309 - merged) • Chad Woolley, Jay McCure • 18.6
- https://gitlab.com/groups/gitlab-org/-/epics/19140+s
Testing
General testing approach to ensure this actually works on any given environment:
- Set the
GLCI_MEDIUM_RUNNER_REQUIRED
:- visibility:
visible
- flags: both
protect variable
andexpand variable reference
off - description:
A tag pointing to runners which have at least 4 cores and 16G RAM. See https://gitlab.com/gitlab-org/gitlab/-/merge_requests/207000 for context
- key:
GLCI_MEDIUM_RUNNER_REQUIRED
- Value: Whatever is appropriate for the env. E.g. currently
saas-linux-medium-amd64
on canonicalgitlab-org/gitlab
environment in order to use medium SaaS runners.
- visibility:
- Add pipelinetier-3
- Add pipeline:run-all-rspec => https://gitlab.com/gitlab-org/gitlab/-/pipelines/2073422310
- Cherry-pick the commits from this MR to test MRs on other environments (listed below in rollout plan) to ensure they behave as expected (either using the runner defined in the var, or not running the system jobs at all)
- See the Test environments section below for details.
From a code perspective, there's only two general scenarios we need to test:
- ENV var is defined.
- ENV var is not defined.
We have already tested both of these scenarios successfully, in multiple environments. See the columns for GLCI_MEDIUM_RUNNER_REQUIRED
and Comment
in the Test environments table below.
Rollout plan
-
Define CI variable GLCI_MEDIUM_RUNNER_REQUIRED
as necessary on all environments where system tests need to run. Test environments table below for details. -
Confirm that we are OK with any environments that previously ran system tests but will no longer run them after this MR (we think that this may only impact personal forks, which may not have ever run them, still need to confirm. see Test environments table below) -
Docs - will be done in follow-up: Add documentation for CI variable GLCI_MEDIUM_R... (#576121) • Chad Woolley -
Document next steps and pending actions on runners/env vars: DONE - see pre-merge comment: !207000 (comment 2811679931)
Test environments
These are the environments we have tested on:
Instance | Repo | MR | User | Pipeline |
|
Status | Comment |
---|---|---|---|---|---|---|---|
gitlab.com | Canonical | GitLab team member |
set to |
|
System tests run | ||
gitlab.com | Community contributor |
https://gitlab.com/gitlab-community/gitlab-org/gitlab/-/pipelines/2089292343 |
not set |
|
No system tests run | ||
gitlab.com | Community contributor |
https://gitlab.com/gitlab-community/gitlab-org/gitlab/-/pipelines/2089662503 |
set to |
|
System tests run | ||
gitlab.com | GitLab team member |
set to |
|
System tests run | |||
gitlab.com |
Security fork - gitlab (ee) |
https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/5397 |
GitLab team member |
https://gitlab.com/gitlab-org/security/gitlab/-/pipelines/2089721294 |
not set |
|
No system tests run |
gitlab.com |
Security fork - gitlab (ee) |
https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/5397 |
GitLab team member |
https://gitlab.com/gitlab-org/security/gitlab/-/pipelines/2090720684 |
set to |
|
System tests run |
gitlab.com |
Security fork - gitlab-foss |
https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/5397 MRs trigger a child pipeline |
GitLab team member |
https://gitlab.com/gitlab-org/security/gitlab-foss/-/pipelines/2091712515 |
not set (inherited from parent pipeline) TODO: set env var for foss project for branch pipelines |
|
system tests run on saas-linux-medium-amd64, inherited from parent pipeline |
https://dev.gitlab.org/gitlab/gitlab-ee/-/merge_requests/1560 |
n/a |
|
|
System tests run (pipeline failed, but seems unrelated test failure) | |||
gitlab/gitlabhq (foss) |
MRs not enabled | n/a | n/a |
|
|
||
Personal fork | Community contributor |
|
Maybe not possible, forking gitlab puts you over the storage limit. Would have to try with a paid user maybe? In any case it should still will fallback gracefully to not running system tests until they have the env var pointed to an adequate runner. |
Screenshots or screen recordings
N/A, see links to individual MRs and pipelines in Test environments table above.
How to set up and validate locally
You can cherry-pick this MR's commit to all other necessary environments and ensure it works (either runs or doesn't run system tests, without failing).
But we have already tested this on multiple environments, see links to individual MRs and pipelines in Test environments table above.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.