Previously rules keyword used to determine if a job should be include or exclude from pipelines, in this release we've enhanced rules capabilities so users could decide if certain conditions are met to override variables in jobs allowing higher flexibility when configuring your pipelines.
Problem to solve
Rules define when certain jobs can be run, but they could also provide overrides for configuration. This would be a handy way to set up rules for when certain policies or variables should be set. One example use case would be making cache-policy settable by “rules” to say for example that you only want to update the cache pull-push on master and not on branches. Another could be setting interruptible using various rules.
In this issue, we only add variables to the rules. Other configs can be added in further iterations.
Add variables to rules to specify variables. To use the example from the problem to solve section, this would look like:
# This will be implemented in #294232.workflow:rules:-if:$CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"variables:MY_VAR:"hello"my_job:...rules:-if:$MY_OTHER_VAR == "true"variables:MY_VAR2:"hello"# would only be set for this jobMY_VAR:"hello2"# overwrite workflow-level override, for this job
An override will by definition always take precedence over anything set under variables in a job or in the pipeline; job-level overrides take precedence over workflow level overrides. We should use the same evaluation policy for rules that conflict as we do for when, which is that (only) first matching rule wins.
Permissions and Security
N/A
Documentation
Need to make sure we document that, just like the rest of rules, the first matching rule wins and only the overrides for that rule will be applied. If other rules also match, they will not be also applied (after the first match).
Availability & Testing
What does success look like, and how can we measure that?
N/A in terms of metrics. This would however make pipelines easier to configure in complex use cases.
What is the type of buyer?
Individual contributor who writes pipelines
Links / references
This page may contain information related to upcoming products, features and functionality.
It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes.
Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.
I think this is a useful feature and would be great if it aligns to the precedence nature. The job.rules should always have capability to overwrite with workflow.rules which gives a complex pipeline having specific requirements to work as required.
Only workaround for now I would suggest is limit the use of workflow.rules unless the pipeline is designed to be simple and straight, but instead each job can be specified with this own set of rules.
@jyavorska If #213734 (closed) is considered a duplicate of this, which means the proposal is also applicable to job:rules, I'm not really sure if the options should be under overrides. It doesn't add any value. There's job:rules:allow_failure and it's great. It should simply override whatever job:allow_failure is set to if a given rule triggers.
What's more, even if overrides was to stay, I don't understand why it's proposed to be an array. Non-array approach seems way more natural:
Thanks for the ping @Nowaker. I'm curious if @ayufan had a reason for the array format - I don't recall at the moment. Also, cc'ing @kencjohnston who is covering for @thaoyeager (PM for this product area) this week.
I would tend to agree that overrides isn't adding value here except potentially prevent keyword collisions in the future.
I'm not sure where the array did came from. Both approaches are good for me.
I'm not sure about workflow:rules part, but maybe this is acceptable as well. Current logic makes it harder to merge with workflow:rules overrides, but this could be adopted as well.
In testing @dhershkovitch and I just confirmed that it's first-come-first-serve, so in the above example, only VAR1 overrides would be set (so MY_VAR==hello, and MY_VAR2 is unset. I think that's clear it will just need to be clearly documented.
@furkanayhan Friendly reminder to see if you can help us refine / weight this before %13.6 just to flag any concerns before we start on this work next milestone.
Just my two cents:
It would be great if besides overrides: there would be as well additionally: that would work for eg variables. Overrides would replace all variables that were set outside rules:, additionally would add to them, replacing only those that have the same key.
The same mechanism probably would be good for other properties than variables, but those are most obvious example.
@furkanayhan Update: after a sync meeting with @furkanayhan I was convinced that there is no need to add a new keyword since we already support overrides for overriding when / allow_failure, let's use the proposed suggestion
Just to be sure: you guys are talking about variables, but this task applies to other properties as well, right?
Because I definitely have use-cases for overriding needs,dependencies,artifacts and can imagine uses for tags,stage, perhaps *script. Changing script seems like it deserves separate job, but possibilities would be really nice with this. And changing before/after_script conditionally seems not bad.
My point is it would be better to list keywords that can't be overriden (extends?, rules) that those which can (all the rest hopefully).
For workflow:rules (to be opened after the first one)
As far as I understand from the release post MR, the first one is enough for it. However, the first one is still in review and even if we manage to merge it today, it will be on the production tomorrow. Then I need to roll-out the feature flag. The release post can be merged once the feature flag is enabled by default. It is unlikely for now but let's see...