Make it possible to use any language to generate `.gitlab-ci.yml` and pipelines
Description
.gitlab-ci.yml
can quickly become very complex. .gitlab-ci.yml in GitLab CE is quite complex already, and we are struggling with maintaining it sometimes.
Proposal
Make it possible to allow anyone to use any language they would like (Python, Ruby) to create CI/CD configuration.
How to do that?
Jenkins made it possible to use seed jobs, and this is fairly established concept we could use too.
We could make it possible to extend existing pipeline with configuration from .gitlab-ci.yml
generated by a seed build. It does not matter what technology / language / approach seed build uses, what matters is a valid .gitlab-ci.yml
sent back in the artifacts archive.
Imagine following scenario:
.gitlab-ci.yml
:
1. Define a seed build in image: ruby:alpine
seed:
script:
- gem install gitlab-ci-config
- gitlab-ci-config generate my-config.rb > artifacts/.gitlab-ci.yml
artifacts: # ...
gitlab-ci-config
is an example, it can by a piece of Ruby code that is not a gem, it can be a Python code, or anything else. We can build one or two tools by ourselves to make it easier to adopt this feature, but it is a secondary aspect.
2. Push code to GitLab
After pushing code with .gitlab-ci.yml
like this to GitLab, GitLab Runner would pick the job, to generate .gitlab-ci.yml
in a secure way.
It would push artifacts with .gitlab-ci.yml
back.
In case of an invalid .gitlab-ci.yml
generated, change seed
build status to failed.
3. Augment existing pipeline to extend it with subsequent stages
GitLab should be able to easily see that seed
build has artifacts with a new .gitlab-ci.yml
. We already do something like this with codequality, sast and other jobs.
It would then extend existing pipeline with additional stages / builds, and the pipeline would look like:
seed -> build -> test -> deploy
What do you think @ayufan @bikebilly @markpundsack?
/cc @godfat