I wonder if anyone else is running into this problem and how they have resolved the issue. I'm currently using the include statement in my .gitlab-ci.yml. Everything works as expected granted that I include a second yaml file into the main .gitlab-ci.yml with one include. However once I look to add a include to the second yml as well I'm unable to detect any jobs on the second yml file and I question whether this is working at all.
Currently I have a need for the level of abstraction which would be helpful to include a serious of yaml files "russian doll style" which prevent me to constantly updated multiple gitlab-ci.yml in multiple repos.
Description
The ultimate goal of this would be to have my main gitlab-ci.yml have the following
I just ran into the same issue. It seems to me that recursive (nested) includes doesn't work. Perhaps this is a better title? I was looking for it and stumbled in here.
Making a guess that this is a bug, but if the intention is not to support recursive includes, the documentation should probably state that it won't work. I found an early merge requests saying a (current, at that time) limitation was not supporting it. I'm happy to elaborate on my use case should motivation be required to support it.
I can definitely change the topic. I understand how this could possibly throw people off. But the use case for me is very simple unable to best yml files with include where the job is on the inner most yml file.
It looks like it only merges the content from the included file and doesn't actually respect "include" statements in included files. ping @mayra-cabrera!
@mayra-cabrera In the yaml lint could you also include the proper error messages to reflect this not being supported or if there is a time table to support in future releases could you please include.
@sleath said: could you also include the proper error messages
I believe this is already covered elsewhere; saw a MR/Issue the other day (apologies for not searching it for you).
@mayra-cabrera said: The documentation should definitely mention that.
Thanks for that.
My short argument for why I think this should be classified as a bug (and as a result, perhaps documented as such): you define a keyword in a language (yaml) and users expect keywords to act consistently through their environment; much like you for instance can do object inheritance or even just define an array. By adding exceptions you make this slippery.
I understand that the implications of recursive merges are technically more challenging; be it recursive loops (include _self for instance), nesting limits and so on. It still seems like the "correct" thing to do; have a keyword in a language act consistently.
users expect keywords to act consistently through their environment,
I think that there should be at least a restraint around the amount of levels you can go. At the very least the ability should be 2 levels but definitely defined in docs. I can see how the includes could be expected behavior and I vote for this to be resolved in a future release.
In the yaml lint could you also include the proper error messages to reflect this not being supported or if there is a time table to support in future releases could you please include.
@sleath what do you think if we divide and conquer? As:
1 - We prepare this issue as a feature proposal for including recursivity in the include keyword
2 - We open another issue to also include the recursivity behavior in the Ci YAML Linter. As a follow up for the first point.
@mayra-cabrera said: what about giving control to the user?
My take would be to avoid exposing recurse control. I'm sure there's more than one disingenuous way of abusing this. How about just setting a hard limit of, say, 4 includes and expose a setting via gitlab.rb to increase this (to a sane degree).
Also: after arguing this as being a bug I'm starting to feel like it's indeed feature enhancement. It's a tricky problem and you took the sane approach.
@jlenny I think that recursive includes could be useful in some cases. Implementation shouldn't be too difficult, but there are some edge cases like circular includes, too deep nesting etc, but nothing that would make it impossible to implement that.
Why interested: developing CI/CD pipeline templates which can be used for project creation templates; also allows for control/consistency of stage names and jobs
I guess when we say "Nested", we mean you can include one level deep into the root .gitlab-ci.yml, but not more than that which would be full recursive support.
So root .gitlab-ci.yml files can have include. The files being included can not.
@gonzoyumo is it fair to say that what was proposed in the description isn't completely implemented? Do we have any plans to support includes outside the main/root .gitlab-ci.yml file @jlenny?