Skip to content

Automate keeping up to date with packagecloud release

Axel von Bertoldi requested to merge avonbertoldi/29502/packagecloud-improvements into main

This is a second attempt of !3756 (merged)

What does this MR do?

This MR automates selection of the OS distribution releases for which we publish gitlab-runner packages. With this change, the distro releases for which we publish runner packages (deb ad rpm) will no longer be hard coded in the RPM_PLATFORMS and DEB_PLATFORMS Makefile variables (and require manual updating to keep up with packagecloud). Instead, the distro releases will be determined by scripts/packagecloud-releases.go and be based on whatever releases our packagecloud instance currently supports.

The script does this by querying our packagecloud instance for all the distro/versions it supports, and whittling that list down by:

  • the package type (deb or rpm) specified in the command invocation
  • the branch (stable or bleeding) specified in the command invocation
  • the distros we claim to support as encoded in the variable supportedDistrosByPackageAndBranch
  • releases not mentioned in the variable skipReleases
  • releases up to and including the oldestRelease (i.e. releases older than oldestRelease will be excluded).

Why was this MR needed?

Currently we have keep up to date with packagecloud releases manually. This means:

  • Manually checking when our packagecloud instance (at https://packages.gitlab.com) has been updated, which brings support for new OS distribution versions.
  • Manually update the distro versions to which we publish packages. i.e. update RPM_PLATFORMS and DEB_PLATFORMS here and here

So far we've only been doing this reactively when users file issues requesting we add support for the latest distro versions. Automating this is better for us (less work) and better for users (don't have to wait for support for new distro releases).

What's the best way to test this MR?

> PACKAGECLOUD_TOKEN="..." PACKAGE_CLOUD_URL="https://packages.gitlab.com" go run scripts/packagecloud-releases.go <deb|rpm> <stable|bleeding>

Currently the output looks like this:

Click to expand 
> PACKAGECLOUD_TOKEN="..." PACKAGE_CLOUD_URL="https://packages.gitlab.com" go run scripts/packagecloud-releases.go rpm stable
el/9 el/8 el/7 fedora/36 fedora/35 fedora/34 fedora/33 fedora/32 ol/8 ol/7 ol/6 amazon/2022 amazon/2
> PACKAGECLOUD_TOKEN="..." PACKAGE_CLOUD_URL="https://packages.gitlab.com" go run scripts/packagecloud-releases.go rpm bleeding
el/9 el/8 el/7 fedora/36 fedora/35 fedora/34 fedora/33 fedora/32 amazon/2022 amazon/2
> PACKAGECLOUD_TOKEN="..." PACKAGE_CLOUD_URL="https://packages.gitlab.com" go run scripts/packagecloud-releases.go deb stable   
ubuntu/jammy ubuntu/impish ubuntu/focal ubuntu/bionic ubuntu/xenial debian/trixie debian/bookworm debian/bullseye debian/buster debian/stretch linuxmint/una linuxmint/uma linuxmint/ulyssa linuxmint/ulyana linuxmint/sonya linuxmint/serena linuxmint/sarah raspbian/bullseye raspbian/buster raspbian/stretch raspbian/jessie
> PACKAGECLOUD_TOKEN="..." PACKAGE_CLOUD_URL="https://packages.gitlab.com" go run scripts/packagecloud-releases.go deb bleeding 
ubuntu/jammy ubuntu/impish ubuntu/focal ubuntu/bionic ubuntu/xenial debian/trixie debian/bookworm debian/bullseye debian/buster debian/stretch

What are the relevant issue numbers?

Notes

  • Best reviewed commit-at-a-time
Edited by Axel von Bertoldi

Merge request reports