GitLab Runner plugins
This is continuation of: https://gitlab.com/gitlab-org/gitlab-ce/issues/14149
We will extend the .gitlab-ci.yml
to support the custom plugins.
This will simplify the .gitlab-ci.yml
parsing and push all plugin related logic to GitLab Runner,
thus effectively to person implementing the plugin.
What is important that current shell scripts will also be plugins. There will be no difference between shell plugin or any other plugins.
The plugin will be able to implement different workflows:
- They could run scripts,
- They could be configured from .gitlab-ci.yml,
- They could implement custom workflow, simplify usage of toolkits,
- They could send notifications,
- They could use CI API in special authorization mode to post data back to GitLab: annotations (https://gitlab.com/gitlab-org/gitlab-ce/issues/13409#note_4186592), custom artifacts?
- They will have all informations about all previously executed jobs.
Why plugins?
- Make it easy to extend, allowing everyone to contribute it's own plugin,
- Plugin should work on all platforms (in some cases it will require to install platform specific dependencies),
- How the plugins will be deployed is still to be defined,
- I got inspired by http://readme.drone.io/devs/plugins/ (awesome idea),
YML syntax
The .gitlab-ci.yml syntax will stay as it is:
image: ruby:2.1
before_script:
- echo execute for shell
variables:
BUILD_VARIABLE: value
test:
stage: test
plugin: shell
script:
- echo execute in context of shell script
notify
stage: notifications
plugin: slack
on_success: "All tests passed"
when: on_success
How we will parse the YML?
- We read .gitlab-ci.yml as Hash,
- We treat every key as job if it's Hash and contains either:
plugin
orscript
, - We have some support some workflow related keywords internally in GitLab:
stages
,stage
,when
,only
,except
and remove it from job specification, - We create the job specification from all other hash keys,
- Thus we don't parse the
image
,services
,cache
,artifacts
this will be responsibility of plugin, - The configuration will be created from merging options defined in global context and in job context,
- The new configuration option will be automatically forwarded to Runner and then to Plugin. It will be responsibility of Plugin to check and print information about invalid or missing configuration options,
- We create a Build,
- Runner ask for a new build.
How the Build response from the GitLab to Runner looks like?
For test job:
{
"id": 48584,
...
"plugin": "shell",
"options": {
"image": "ruby:2.1",
"before_script": [
"echo execute for shell"
],
"on_success": [
"echo execute in context of shell script"
],
},
"timeout": 3600,
"variables": [
{
"key": "BUILD_VARIABLE",
"value": "value",
"public": true
}
],
"depends_on_builds": [
{
"id": 48584,
"ref": "0.1.1",
"tag": true,
...
}
]
}
For notify job:
{
"id": 48584,
...
"plugin": "slack",
"options": {
"image": "ruby:2.1",
"before_script": [
"echo execute for shell"
],
"on_success": "All tests passed"
},
"timeout": 3600,
"variables": [
{
"key": "BUILD_VARIABLE",
"value": "value",
"public": true
}
],
"depends_on_builds": [
{
"id": 48584,
"ref": "0.1.1",
"tag": true,
...
}
]
}
Credentials
We have the Secure Variables. We will extend that with Credentials where we add option to easily add credentials: for accessing the SSH host (SSH private key), for connecting to Docker Engine (or Cluster), for connecting to Kubernetes, or anything else that will make sense to easily configure. The credentials will be sent to runner as secure variables and will be handled by plugin.
Why plugins are awesome?
We could then implement plugin that will allow to run steps in sequence or in parallel (during executing the single Job):
test:
plugin: sequence
steps:
- plugin: script
script: echo "prepare configuration of docker environment"
- plugin: parallel
parallel:
- plugin: docker-build
path: docker/file/one
push: my.registry.com/my/image
- plugin: docker-build
path: docker/file/second
push: my.registry.com/my/second-image
The above definition will allow to:
- Run commands,
- Run two docker-builds in parallel,
This is just example, but show the flexibility of the system.
Plugin implementation
This will be done in GitLab Runner. The GitLab Runner will implement a set of internal plugins (like: shell). The exact details of how the plugins is something to define. Most likely we will use external programs executed by GitLab Runner with all information about the build passed to them.
This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.