Automatically merge CE master into EE master
What does this MR do?
This sets up GitLab CI to automatically push CE master changes into EE
master, or revert them if the changes cause merge conflicts. The CI
configuration contains a single job to do this: merge:master
. This job
is executed for every push to master, and periodically using a CI
schedule.
The periodic job is necessary because incremental jobs may not be able to revert commits if newly added commits depend on these commits. By re-running the job periodically (including all changes since a large enough time frame), we can ensure that such commits are also reverted (if they still conflict at that time).
The job runs in its own "merge" stage, after the build and prepare stages, but before running the tests. This ensures that randomly failing tests won't prevent code from being merged into EE. Running the stage after the "prepare" stage reduces the chances of the job reverting CE changes just because it ran before a corresponding EE MR was merged into EE master.
The container used by this CI job is built daily in https://gitlab.com/gitlab-org/merge-train/.
Example CI output
Running with gitlab-runner 11.5.0-rc1 (e900028d)
on docker-auto-scale 0277ea0f
Using Docker executor with image registry.gitlab.com/gitlab-org/merge-train ...
Pulling docker image registry.gitlab.com/gitlab-org/merge-train ...
Using docker image sha256:994868a307623150907ad66a0c491ca6dd866f1d9d73421b17be5adf816df2a9 for registry.gitlab.com/gitlab-org/merge-train ...
Running on runner-0277ea0f-project-9683125-concurrent-0 via runner-0277ea0f-srm-1543511992-95e93c87...
Cloning repository for master with git depth set to 20...
Cloning into '/builds/yorickpeterse/gitlab-ce'...
Checking out f1f70328 as master...
Skipping Git submodules setup
Checking cache for merge-1...
FATAL: file does not exist
Failed to extract cache
$ scripts/merge-train
Agent pid 12
Identity added: /root/.ssh/merge_train (automatic merge)
Cloning yorickpeterse/gitlab-ee
Warning: Permanently added the ECDSA host key for IP address '35.231.145.151' to the list of known hosts.
I, [2018-11-29T17:22:43.635210 #19] INFO -- : Fetching master from remote merge-train-source
I, [2018-11-29T17:22:44.320720 #19] INFO -- : 1 commits need to be merged
I, [2018-11-29T17:22:44.485251 #19] INFO -- : f1f7032826336d527e89386fb5a4081151b3199a was merged successfully
I, [2018-11-29T17:22:44.485666 #19] INFO -- : Pushing changes to master
I, [2018-11-29T17:22:45.983972 #19] INFO -- : Pushed changes to master
Creating cache merge-1...
gitlab-ee: found 22784 matching files
Uploading cache.zip to https://storage.googleapis.com/gitlab-com-runners-cache/project/9683125/merge-1
Created cache
Job succeeded
Without the cache in place, the job will take at least 2 minutes and 50 seconds (ish). With a cache in place it takes at least one minute less to run.
What are the relevant issue numbers?
https://gitlab.com/gitlab-org/release/framework/issues/49
TODO
-
Set up EE MR for these changes: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/8657 -
Store the SSH public and private key as a CI variable -
Put the SSH public and private key in 1Password -
Add the public key to GitLab EE -
Create an API token attached to @gitlab-bot, and store it as a CI variable in CE -
Stop the periodic "CE upstream" job, so it doesn't conflict in any way -
Update any documentation mentioning the "CE upstream" job so that it covers the new setup -
Announce on the company/backend call -
Add to the engineering week in review document
Does this MR meet the acceptance criteria?
-
Documentation created/updated -
Conforms to the code review guidelines -
Conforms to the merge request performance guidelines -
Conforms to the style guides -
Security reports checked/validated by reviewer