Testing Gitaly master against GitLab master
Gitaly and Praefect are changing quite rapidly, and right now it doesn't seem that we have an easy way to verify that changes in these services don't break GitLab Rails. Right now it seems that the main way we catch errors is when an auto-deploy branch fails a test, which seems too late in the process because this blocks deployments. Ideally, Gitaly merges to master
don't pass unless GitLab Rails tests also pass, but launching all 250+ GitLab Rails builds for Gitaly changes seems excessive and slow.
Adopted solutions
In #222497 (comment 366454290) we discussed the first iteration for this.
- release-tools will check for the latest green master on gitaly and prepare a merge request on targeting GitLab master branch like !36542 (merged)
- This merge request will be set in MWPS by release-tools.
- The auto-deploy branch will no longer receive direct version bumps.
- release-tools will never attempt to update an ongoing merge request
In the first iteration if the pipeline fails it will be up to the team to figure out why it failed and update the merge request.
If the merge request is closed, the next run will generate a new one.
This was implemented by release-tools!1062 (merged)
Initial proposals
Couple of ways we might consider improve this:
-
Change GitLab
master
to useGITALY_SERVER_VERSION
master
. Pro: This means that any issues in Gitaly tests will be seen by more people. Con: Developers may be impacted by unrelated test failures. -
Use multi-project pipelines (https://docs.gitlab.com/ee/ci/multi_project_pipelines.html) and trigger a new GitLab Rails build every time a Gitaly merge to
master
occurs. Notify the Gitaly channel when a test failure happens. Pro: Faster feedback when something goes wrong. Cons: Noise--many build failures may be flagged unrelated to Gitaly. -
Run a subset of GitLab Rails tests as part of Gitaly tests (e.g. everything in spec/models). This would at least catch test setup issues and basic issues, but it wouldn't find everything.
-
Use a manual CI step for 2
@rymai @zj-gitlab @johncai @pokstad1 Any other ideas or thoughts here?