Skip pipeline creation for workloads

What does this MR do and why?

Duo Workflow is a feature in GitLab where the user can ask a Duo to complete some task. Duo Workflow can read and write files and do some other actions in the context of a workflow. At present the feature is in alpha testing stages inside of VS Code. You can read more about the Duo Workflow architecture at https://handbook.gitlab.com/handbook/engineering/architecture/design-documents/duo_workflow/ . As part of !176742 (merged) we have added (behind a feature flag) the ability to trigger a workflow to run in a CI Pipeline. To accomplish this we create a new branch every time a worfklow is triggered and then trigger a dynamically constructed pipeline to run against that branch.

This previous implementation has a problem because just creating a branch in GitLab will trigger a pipeline already. The pipeline triggered will run against the .gitlab-ci.yml in the project. This means that at present we are triggering 2 pipelines. For Duo Workflow we do not want to trigger this normal .gitlab-ci.yml pipeline against these branches.

This MR ensures that we skip this .gitlab-ci.yml pipeline being created when we create the branch.

In order to understand this change you need to understand how these pipelines are triggered in the first place. The flow is:

1. GitLab Rails makes a gRPC call to gitaly to create a branch
2. While processing this request gitaly makes an internal API call back to GitLab Rails to trigger "PostReceive" hooks which are triggered every time a new branch is created (even when a user pushes via git CLI)
3. GitLab Rails processes the API request and triggers a PostReceiveWorker
4. The PostReceiveWorker sees that a new branch was created and eventually through a few more class/method calls ends up creating a pipeline

So in order to prevent the pipeline creation we needed some way to pass a boolean value through to gitaly and back to GitLab Rails to tell it not to create a pipeline for this specific branch.

This was accomplished using gitaly_context which is a JSON encoded field passed with the gRPC calls. This is already wired up to pass arbitrary context through this exact call stack. This MR introduced a new JSON boolean field skip-ci which can optionally be passed in when creating a branch. This change required editing a lot of classes because the call stack for add_branch is quite deep to get all the way to constructing the gitaly gRPC request. Additionally on the Rails receiving side it also required editing quite a few classes because the call stack before we create a pipeline is also quite deep.

For the most part all of the changes in this MR are 1 of 2 things:

1. Passing an extra boolean argument ci_skip through a bunch of method definitions
2. Passing the gitaly_context through a bunch of classes/methods/sidekiq workers that previously did not accept it

References

Please include cross links to any resources that are relevant to this MR. This will give reviewers and future readers helpful context to give an efficient review of the changes introduced.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

Before	After

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

Related to #517411

added Category:Duo Workflow devopsai-powered groupduo workflow sectiondata-science labels

assigned to @DylanGriffith

added pipelinetier-1 label

changed the description

removed pipelinetier-1 label

added backend label

	9 Warnings
	88ac21a7: Commits that change 30 or more lines across at least 3 files should describe these changes in the commit body. For more information, take a look at our Commit message guidelines.
	fe371d71: Commits that change 30 or more lines across at least 3 files should describe these changes in the commit body. For more information, take a look at our Commit message guidelines.
	c93e3906: The commit subject must contain at least 3 words. For more information, take a look at our Commit message guidelines.
	8005b640: The commit subject must contain at least 3 words. For more information, take a look at our Commit message guidelines.
	8005b640: The commit subject must start with a capital letter. For more information, take a look at our Commit message guidelines.
	8005b640: Commits that change 30 or more lines across at least 3 files should describe these changes in the commit body. For more information, take a look at our Commit message guidelines.
	83a04d8d: Commits that change 30 or more lines across at least 3 files should describe these changes in the commit body. For more information, take a look at our Commit message guidelines.
	featureaddition and featureenhancement merge requests normally have a documentation change. Consider adding a documentation update or confirming the documentation plan with the Technical Writer counterpart. For more information, see: The Handbook page on merge request types. The definition of done documentation.
	This merge request does not refer to an existing milestone.

	1 Message
	CHANGELOG missing: If this merge request needs a changelog entry, add the Changelog trailer to the commit message you want to add to the changelog. If this merge request doesn't need a CHANGELOG entry, feel free to ignore this message.

Reviewer roulette

Category	Reviewer	Maintainer
backend	@smaglangit (UTC+8, 3 hours behind author)	@pskorupa (UTC+1, 10 hours behind author)
devopsverify	Reviewer review is optional for devopsverify	@ayufan (UTC+1, 10 hours behind author)

Please refer to documentation page for guidance on how you can benefit from the Reviewer Roulette, or use the GitLab Review Workload Dashboard to find other available reviewers.

If needed, you can retry the danger-review job that generated this comment.

Generated by Danger

Proper labels assigned to this merge request. Please ignore me.

@DylanGriffith - please see the following guidance and update this merge request.

	1 Error
	Please add typebug typefeature, or typemaintenance label to this merge request.

added 1 commit

• 83a04d8d - Pass skip-ci: true in gitaly_context

Compare with previous version

added 1 commit

• 8005b640 - Draft: refactor

Compare with previous version

added 2 commits

• c93e3906 - Another spec
• fe371d71 - Rubocop fixes across it all

Compare with previous version

added 1 commit

• 8668fc5b - Fix repository_spec passing skip_ci: false

Compare with previous version

added 1 commit

• 0fec55f7 - More spec fixes

Compare with previous version

changed the description

added 1 commit

• 88ac21a7 - Dont update arguments to workers update push_options instead

Compare with previous version

changed the description

changed the description