[CI] merge requests and dependencies - another look ("release candidates")

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

~"feature proposal"

Description

I have a situation in a project, where I have a following structure:

myproject/main.git
myproject/shared-module-1.git
myproject/shared-module-2.git

so what happens when I want to submit I feature? I create a branch in one of the shared modules, then I am able to use .gitlab-ci.yml to spawn up unit tests. Great. The problem is that those modules are also used in the main project. I have seen some solution posted here which make use of either hooks and / or .gitlab-ci.yml, however I don't think this is a valid choice, and let me argument for that.

The CI in the main project would have to pull the dependencies from sub-repositories, based on a particular branch I'm doing. Let me be more verbose than that. Let's suppose that I have to implement a 'feature-1'. Here's a simplified scenario for that:

  • I see that I have to modify shared-module-1. Easy enough, I create a branch - called 'feature-1'.
  • I write some code, and I push to shared-module-1 repo, CI runs the tests. Great
  • I now want to check whether this module would cause any breakage in the main project. Let's say that I also create a branch in the main.git to match the feature
  • Great, how do I pull the changes? Well, seems easy enough, right? I modify code inside my main.git so that the dependencies are pulled from the 'feature-1' branch of 'shared-module-1.git'.
  • Oh, I have now realized that the changes spawn upon the shared-module-2 as well. Well, easy enough I suppose - I just have to repeat the above steps for the other dependent repo (Here comes the important part :D)
  • Everything seems great, until ... I want to close the merge request and delete the branch. What happens then? Then my tests would obviously fail, because the branches wouldn't be available. Therefore I would have to create a dummy commit to replace the branches back to master (or whatever the stable branch is). This does not scale

(note: I believe I could achieve some sort of functionality with .gitlab-ci.yml and create some additional job with environment variables, like so: )

environment:
  shared-module-1: pull-from-this-branch
  shared-module-2: pull-from-that-branch

then, inside of the test script, I would use this environment variables to dynamically set branches for modules. This does not scale as well

Proposal

My proposition is to keep this workflow inside gitlab universum, not inside .gitlab-ci.yml. The change would be quite subtle:

1/ in main.git project (or whatever project for what it matters) I would have a button / form titled 'create release candidate'. When clicked, this would allow me to select merge requests from other projects, from a very simple html form like so:

name: 'prepare code for feature 1'
dependencies:
   - my-shared-module / merge request #12
   - my-shared-module-2 / merge request #859

there are couple of benefits:

  • gitlab already knows which branch is referred to in the merge request, therefore it could prepare dynamic environment variables based on the commits
  • when I close one of the dependent merge requests (but not all of them) I would have less environment variables. I.e. if I close the merge from shared-module-1, then I know that I don't have to override the branch for shared-module-1 in main project (because it already got merged)
  • gitlab-ci runner knows which project is running the build on, therefore it could use the API / whatever to access opened release candidates and do builds on them
  • the process can be repeatable, i.e. when I make a push into the shared module, gitlab can identify that it is one of the dependencies and trigger a build
  • (here comes the best part). When all of the dependent branches get merged, gitlab can automatically close the release candidate.
  • the test script can be modified very easily, it would just need to recognize whether it is run within this special mode and use environment variables provided by gitlab to override branches of the dependencies

In practice, this would mean that when the code from dependent projects would hit master / stable branch, I wouldn't have to change a single line of code in .gitlab-ci.yml because all magic would happen inside gitlab universum. In other words, i wouldn't have to change the dependencies just for the sake of running the tests and change them back to 'master' once the release is prepared.

If you have any questions, please don't hesitate to ask. I may be not quite clear with explaining, but I am quite positive that I haven't (yet) seen a solution which would be similar to mine.

kindest regards, toudi

Edited by 🤖 GitLab Bot 🤖