Proposal: plugin distribution as a built-in feature
Background
We previously decided, that for now, we would distribute fleeting plugins as a GitLab release and users will manually install them.
As part of that discussion (https://gitlab.com/gitlab-org/fleeting/fleeting-plugin-aws/-/issues/2#note_1229075484), other potential plans for the future were also discussed, and I'd like to expand/improve on some of the ideas there.
Goals
-
All plugins should be easy to install to all environments that Runner can currently be deployed to, regardless of whether they're official or third-party.
-
I'd prefer to avoid bundling fleeting plugins within the release of Runner.
The bundling of additional software is something we're already guilty of and it dramatically increases the size of the package to be downloaded and the security footprint. Customers end up with a package that contains more irrelevant software than relevant for their specific setups. We have an issue to eventually fix some of the offending items.
-
It should be very easy to upgrade/downgrade a fleeting plugin. Fleeting plugins have a built-in mechanism for negotiating protocol changes, so it's likely that the best plugin version to use at any point in time is always the most recently tagged version. We should avoid Runner strictly requiring a certain plugin version.
Proposal
Runner is installed in a few ways:
- Package (deb, rpm etc.)
- Single static binary download for specific system architecture
- Container (docker, helm chart/kubernetes/operator)
In terms of packaging plugins, this is really 2 problems:
- Non-container: How do we install a plugin for packages/static binary downloads.
- Container: How do we install a plugin for containers. We don't want to download plugins each time a container is run.
For the non-container scenario, I like how Packer plugins work. The packer config references the plugin and version constraint and then running packer plugin init
and packer plugin install
will ensure that the plugin is available on the system.
For the container scenario, the non-container scenario is also supported, but would require either extending the container image, executing commands inside the container, or mounting the volume where the plugins have been installed to. In some setups, this is absolutely fine.
An alternative for the container scenario would be to allow plugin sidecars, this is very Helm-chart and operator friendly, and rather than use a plugin install
command, the plugin could be configured as a sidecar. This has the benefit that the sidecar won't be downloaded and installed each time, as a Kubernetes node, for example, would cache the image.
To make this work, and not require Docker/Kubernetes for the non-container scenario, and to also avoid plugin authors having to create both a non-container and container variant, I think we can make light-weight specialised container images that are also packages for the non-container scenario.
Most plugins are written in Go and can be statically compiled, where this isn't true or where a plugin needs additional dependencies, these already have to be distributed in a way that doesn't rely on a full Linux system like Docker would provide. If this is the only setup we support, then instead of packaging into a zip file, we instead make an OCI image, we'll have a distribution format that can be both installed locally (with a tool that can talk to OCI registries) as well as a sidecar with Docker/Kubernetes.
To summarise:
- All plugins must be pushed to an OCI registry. I don't think this should be a problem these days as GitHub, GitLab and different cloud providers offer this. This is only to support the "official" way of distributing a plugin, you could of course still install a plugin locally, and that could come from anywhere.
- We would take our ordinary plugin package and make it OCI image compatible. We're not making proper docker images that contain an OS: we're simply making them compatible to be run as side-cars.
- Fleeting would include a command line utility to package and push a plugin. We're not reinventing buildx/buildah here: It would simply package the current directory, and add metadata.
- Fleeting's CLI would also include the ability to install/uninstall etc, but these commands would be written so they can easily be imported into other programs that use fleeting, such as GitLab-Runner (would become something like:
gitlab-runner fleeting-plugin install
).