ci: define test packages as variables
What does this MR do?
The original MR started as a way to re run failed tests using gotestsum's --rerun-fails flag. However, after discussing with the team we decided remove the flag and keep the refactoring done.
The repurposed MR uses the variables yaml keyword to define the test packages to be used by the GO_TEST variable.
It uses anchors to be able override different values where needed.
rerun failures
gotestsum has a --rerun-fails=n flag that can retry tests that fail. It ensures each test passes at least once up to n tries. This should save us some time in CI that would help with some flakes identified in Improve the fragility and speed of tests in the... (&9194)
Notes
- Tests that panic like in https://gitlab.com/gitlab-org/container-registry/-/jobs/3477222975 are not retried:
-
ERROR rerun aborted because previous run had a suspected panic and some test may not have run 235 exit status 3
-
- There are still some tests failing, even after retrying. For example https://gitlab.com/gitlab-org/container-registry/-/jobs/3619851945. This makes me think that once that particular test fails, any retries will still fail. So perhaps there is a problem with the DB setup and the timings
🤔
Edited by Jaime Martinez