Support merging rules arrays with !reference
### Release notes
The YAML function called `!reference` which was introduced early this year allowed you to reuse the same configuration in multiple jobs, however merging rules was not supported. In this release, we've added support for merging rules array `!reference`, which lets you target the rules configuration you want to reuse as apart of your CI/CD pipeline, even if it's in another file.
### Summary
Issue https://gitlab.com/gitlab-org/gitlab/-/issues/266173 introduced the `!reference` notation to re-use job sections. This works well for the `script`-related tags and some other cases as noted in the documentation, but in the case of `rules` doesn't work (and maybe some other places), or at least doesn't work as expected per the documentation.
### What is the current *bug* behavior?
When trying to `!refence` a `rules` array a syntax error is reported. For example:
```yaml
.no_schedule:
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
when: never
- when: on_success
job1:
image: alpine:latest
rules:
- !reference [.no_schedule, rules]
script:
- echo "run except schedules and tags"
```
fails linting with the error `This GitLab CI configuration is invalid: jobs:job1 rules should be an array of hashes.`
It appears, assuming the merged YAML view is correct (a great feature BTW), that this is because with the current implementation `!reference` doesn't flatten arrays merged into an array element, it inserts the referenced array creating a nested array. For example, this script:
```yaml
.my_script:
script:
- echo "foo"
- echo "bar"
after_script:
- echo $CI_COMMIT_BRANCH
job4:
image: alpine:latest
before_script:
- !reference [.my_script, after_script]
script:
- !reference [.my_script, script]
- echo "after foo and bar"
- !reference [.my_script, after_script]
```
results in the following merged job with various nested arrays in `before_script` and `script`
```yaml
:job4:
:image: alpine:latest
:before_script:
- - echo $CI_COMMIT_BRANCH
:script:
- - echo "foo"
- echo "bar"
- echo "after foo and bar"
- - echo $CI_COMMIT_BRANCH
```
The various `script` properties can accept nested arrays of strings (per the linter), so this executes as expected, but `rules` does not as noted in the error above.
The only ways to to use `!reference` with `rules` appears to be either reference only a rule list with syntax like that shown below, which does not allow merging of rules (at least I couldn't find a syntax that worked)
```yaml
.no_schedule:
rules:
- if: $CI_PIPELINE_SOURCE == "schedule"
when: never
- when: on_success
job2:
image: alpine:latest
rules: !reference [.no_schedule, rules]
script:
- echo "run except schedules"
```
or create individual rule hashes (not arrays) and reference those hashes as array elements
```yaml
.only_no_schedule:
rules:
if: $CI_PIPELINE_SOURCE == "schedule"
when: never
job3:
image: alpine:latest
rules:
- if: $CI_COMMIT_TAG
when: never
- !reference [.only_no_schedule, rules]
- when: on_success
script:
- echo "run except schedules and tags"
```
In the latter case it appears any existing hash key that can be resolved by `!reference` can also be used, for example:
```yaml
.rule_references:
no_schedule:
if: $CI_PIPELINE_SOURCE == "schedule"
when: never
no_tag:
if: $CI_COMMIT_TAG
when: never
job5:
image: alpine:latest
rules:
- !reference [.rule_references, no_schedule]
- !reference [.rule_references, no_tag]
- when: on_success
script:
- echo "run except schedules and tags"
```
### What is the expected *correct* behavior?
The capability should exist to merge `rules` arrays with `!reference`.
issue