Duo Workflow is a feature in GitLab where the user can ask a Duo to complete some task. Duo Workflow can read and write files and do some other actions in the context of a workflow. At present the feature is in alpha testing stages inside of VS Code. You can read more about the Duo Workflow architecture at https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/duo_workflow/ . As part of !176742 (merged) we have added (behind a feature flag) the ability to trigger a workflow to run in a CI Pipeline. To accomplish this we create a new branch every time a worfklow is triggered and then trigger a dynamically constructed pipeline to run against that branch.
This previous implementation has a problem because just creating a branch in GitLab will trigger a pipeline already. The pipeline triggered will run against the .gitlab-ci.yml in the project. This means that at present we are triggering 2 pipelines. For Duo Workflow we do not want to trigger this normal .gitlab-ci.yml pipeline against these branches.
This MR ensures that we skip this .gitlab-ci.yml pipeline being created when we create the branch.
In order to understand this change you need to understand how these pipelines are triggered in the first place. The flow is:
GitLab Rails makes a gRPC call to gitaly to create a branch
While processing this request gitaly makes an internal API call back to GitLab Rails to trigger "PostReceive" hooks which are triggered every time a new branch is created (even when a user pushes via git CLI)
GitLab Rails processes the API request and triggers a PostReceiveWorker
The PostReceiveWorker sees that a new branch was created and eventually through a few more class/method calls ends up creating a pipeline
So in order to prevent the pipeline creation we needed some way to pass a boolean value through to gitaly and back to GitLab Rails to tell it not to create a pipeline for this specific branch.
This was accomplished using gitaly_context which is a JSON encoded field passed with the gRPC calls. This is already wired up to pass arbitrary context through this exact call stack. This MR introduced a new JSON boolean field skip-ci which can optionally be passed in when creating a branch. This change required editing a lot of classes because the call stack for add_branch is quite deep to get all the way to constructing the gitaly gRPC request. Additionally on the Rails receiving side it also required editing quite a few classes because the call stack before we create a pipeline is also quite deep.
For the most part all of the changes in this MR are 1 of 2 things:
Passing an extra boolean argument ci_skip through a bunch of method definitions
Passing the gitaly_context through a bunch of classes/methods/sidekiq workers that previously did not accept it
References
Please include cross links to any resources that are relevant to this MR.
This will give reviewers and future readers helpful context to give an efficient review of the changes introduced.
MR acceptance checklist
Please evaluate this MR against the MR acceptance checklist.
It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Screenshots or screen recordings
Before
After
How to set up and validate locally
Numbered steps to set up and validate the change are strongly suggested.
