Allow a pipeline to specify automatically-generated changes to be applied when an MR is merged
Problem to solve
Within GitLab, we have a few places where automatically generated code/files are committed to a repository. The most obvious example is https://gitlab.com/gitlab-org/gitaly-proto . These automatically-generated changes are not composable. Currently, the changes are generated by having the contributor run a script when the branch is created, and committing the result.
If we have two MRs we want to merge into gitaly-proto at about the same time, the automatically-generated changes conflict with each other. Due to the inner workings of protobuf, even when they don't conflict, a simple rebase isn't enough - the regeneration command must be re-run in full. This currently means that if a gitaly-proto reviewer has N merge requests that are ready for merge, they can only merge one of them, and must then pass the remainder back to the contributors for rebase+regenerate.
Target audience
Reviewers, maintainers, and contributors to any project that commits automatically-generated files that change over time.
Further details
A similar case in the gitlab-ce repository is the locale/gitlab.pot
file. We ask contributors to run rake gettext:regenerate
in their MRs, but this is often forgotten, or done incorrectly. It also can't (currently) be done solely through the web IDE. Wouldn't it be great if GitLab would automatically do this for us?
Having contributors do this kind of work is error-prone, and makes it harder to contribute at all. In the gitaly-proto case, for instance, not only does the contributor need to run the command, but they also need to have a specific version of Golang installed.
Maintainers could take up the burden and merge the changes themselves, but this is just as error-prone, and it raises the cost of being a maintainer significantly.
Proposal
Add a new job type or report to merge requests. When the pipeline runs, a specific job is executed that runs the command, detects any changes to the git repository, and passes the diff back to the merge request. In conjunction with the merge train (choo choo!), we can use this job to remove the burden from the contributor, show the maintainer the automatically-generated changes that the merge implies, and to automatically generate and apply an up-to-date version of the changes when the MR is merged.
This proposal is a little vague, as I don't have a great understanding about exactly how all the moving parts would fit together, and the dependency on the merge train makes this a medium-term, rather than short-term, idea - and by then, the implementation details may have changed significantly!
What does success look like, and how can we measure that?
Several of us can create MRs that modify .proto
files in the gitaly-proto repository at the same time. These MRs can be given to a gitaly-proto maintainer who will be able to review them, see the automatically-generated changes they imply, and then merge all of them without needing to do any extra work themselves, or asking N-1 contributors to update their MRs each time 1 is merged.
Links / references
/cc @jramsay @erushton @DouweM @jacobvosmaer-gitlab @zj @grzesiek