Ideas for saving pipeline runs in MR's to save resources.

Working on some WIP MR's and was having multiple pipelines run when I was not ready to have pipelines start running, which seemed like a waste of resources to me.

This can also happen when more than one person are collaborating on an MR, and push up work on an MR to get some prelim feedback.

I believe this will save the company A LOT of resources.

How do I see this happening?

• We can look at the title of the MR, maybe have something like NOPIPELINE like we have for WIP: on MR's.
• We can also default it to not running the pipelines, and then someone needs to turn it on to have the pipeline run.
• Pre push hooks for frontend linting
• Create Merge Request/Branch in an issue will cause branch name to fail danger bot if issue title is long.

Relevant Issue Links

• Apply Suggestion commit is too long and danger fails.
• Accepting multiple suggestions at once in an MR.
• Pre push hook guards for frontend devs.
• Create MR via UI fails danger bot
• Reduce Costs of Gitlab Project
• Do not run MR pipelines when master is failing
• Leverage interruptible in GitLab's own pipelines

added Category:Source Code Management label

/cc @nick.thomas @jramsay for thoughts on the issue!

Setting devopscreate groupsource code based on Category:Source Code Management.

added devopscreate groupsource code labels

This seems more like a Verify cost savings optimization.

@DarrenEastman what do you think?

It's an interesting question - but doesn't it change the way we thing about CI in GitLab?

added devopsverify grouprunner labels and removed Category:Source Code Management devopscreate groupsource code labels

mentioned in issue #35979 (closed)

Setting Category:Runner based on grouprunner.

added [Deprecated] Category:Runner label

@jramsay @DarrenEastman just following up on this

Thanks

@jyavorska Hi Jason - what are your thoughts about this proposal? I feel as if this seems like a broader discussion.

Interesting idea. This is more of a ~"group::continuous integration" feature, so to start with I'll move it over there.

Awesome thanks for following up on this @jyavorska

I mean maybe we could start with quantifying how much a pipeline costs the company in terms of resources.

My above recommendation was just a brainstorm and POC. A potential easier way would be defaulting to not run the pipeline of Feature Branches and having a button that triggers the pipeline might be a little more simple.

Something that came up in %12.7 that might be related is if master is failing should new pipelines be kicked off until master has been resolved? (Just another use case we could potentially explore)

@jyavorska this has been brought up before in a slightly diff context #27667

Id like to POC this with @fatihacet approach(Create Merge Request Button) and maybe turning off for WIP MR's.

WDYT?

Do we have metrics on the cost each pipeline run costs us?

WIP doesn't really imply you don't want pipelines, it's more you aren't ready for it to merge (even if it's passing.)

Maybe a nice way to do this @sstern is just say if SKIPCI is in all caps in the title of a MR then no pipeline is run for that MR? It matches our naming here: https://docs.gitlab.com/ee/ci/yaml/#skipping-jobs

Thanks for the feedback. Maybe we can start outlining the times where you wouldnt want to run the CI.

Ie, clicking the create merge request button?

But i think thats a good start. My larger point is how to cut down on unnecessary pipelines running. This would save the company and the users money and pipeline resource.

Another example is i just pushed a commit and need to ammend my commit, when i force push now i have 2 pipelines running, but instead of EVERY TIME I PUSH, id like it to default to no pipeline being run.

WDYT?

@sstern naming your MR with SKIPCI in it would achieve the same thing, and still allow for situations where someone does want a pipeline run on MR creation (like if you create a branch with changes in it at the same time you make the MR).

One situation @sstern that results in a lot of pipelines is when an MR has 10 or so suggestions, and you click on "apply suggestion" on each one and it makes 10 pipelines. It would save a lot of resources if there was an "apply all suggestions" button and it did them all as one commit.

and you click on "apply suggestion" on each one and it makes 10 pipelines.

Thats a great observation

@jyavorska seems like a good opportunity to figure out how to batch apply suggestions via a queue that is built during the review process -- where all suggestions in the queue would be applied at the end of the review. If we went with an "apply all" button, we would need a way to discard suggestions beforehand -- which i don't think currently exists when reviewing an MR (or i just haven't seen it).

This would also probably alleviate the pain points around the "apply suggestion" button not reflecting the current state often times after it is pressed. I've found that I need to refresh an MR for it to tell me if it was successfully applied or not.

FWIW we should be able to implement this pretty easily after !26280 (merged) is merged.

e.g.

workflow:
  rules:
    ...
    - if: '$CI_MERGE_REQUEST_TITLE =~ "^SKIPCI"'
      when: never

added Category:Continuous Integration grouppipeline execution labels and removed [Deprecated] Category:Runner grouprunner labels

mentioned in merge request !26280 (merged)

changed title from Allow Disabling of Pipelines in MR's to save resources to Ideas for saving pipeline runs in MR's to save resources

changed title from Ideas for saving pipeline runs in MR's to save resources to Ideas for saving pipeline runs in MR's to save resources from a frontend perspective.

changed the description

created merge request !26941 (closed) to address this issue