Use Chrome 138 in system tests (specs/features)
What does this MR do and why?
- Use Chrome 138 in feature/specs (aka
system
tests) - Fix associated test failures with the above
- Quarantine some feature specs failing due to changes in Chrome 138
-
UseRunners for system tests were made configurable via env variable in Add CI variable GLCI_MEDIUM_RUNNER_REQUIRED (!207000 - merged) and are configured to usesaas-linux-medium-amd64
(4 cores) runners for feature specs rather than the the default shared runners (2 cores)gitlab-org-medium
as per https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/issues/27759+
As per the above point, the below two sections are no longer relevant for this MR:
### Using a higher spec runner for feature specs
We need to use a more capable runner as the current runners cannot handle the amount of virtual memory required to run the latest version of Chrome (thread from attempting to update to Chrome 133 where system tests
would frequently fail). There is also a concern about cpu bottlenecks on the default shared runners.
There is only one machine type used on the private shard, so there is no option to use a medium
machine on the private shard: https://gitlab.slack.com/archives/C9X79MNJ3/p1758241355684459?thread_ts=1757549045.798639&cid=C9X79MNJ3 (Internal)
In this MR we are proposing to change the runners used by all system rspec
jobs from the default shared runners (2 cores) to saas-linux-medium-amd64
(4 cores) hosted runners.
To illustrate the flakiness of the feature specs on our current runners:
- Example pipeline with all changes from this MR but without the change to the runners: https://gitlab.com/gitlab-org/gitlab/-/pipelines/2048568857
- Previous attempt to update Chrome which had lots of flaky pipelines: !202749 (closed)
### Cost
Adding to internal note
References
Resolves https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/issues/3895+
Notes
- In
spec/features/projects/settings/repository_settings_spec.rb
an assertion was commented out, without this assertion the test fails with chrome 138, so I have uncommented the assertion. According to #383717 (comment 1197780367) it sounds like it should have been uncommented anyway.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.