Security release 16.2.1 corrective actions and improvements
In the 16.2.1 we had wrong GPG keys in the Runner security repo which caused:
- The packages in Package Cloud to be signed with the wrong key, breaking users installs
- The UBI images build to fail because it couldn't verify the files in S3 since they were also created with the wrong GPG key.
This was caused by the environment variable in the Security repo not being updated properly during the last keys rotation. Understandably so, since there are a lot of keys to be updated. The security repo in general has quite a few differences to the canonical repo in terms of env variables.
Integrate syncing and diffing of env variables ... (#36332)
Releasing broken Package Cloud packages results in the need to yank the packages currently. This is problematic because:
- We have no procedure to yank packages. Someone might know how to do it, someone might not. Generating all the packages that need to be yanked together with all the distros is also not obvious. I, for example had to figure this out myself. Maybe for someone familiar with the Package Cloud API that would be an easy feat but in any case that's a tall order for most people.
- All the packages by default have their build number set to
1
e.g.16.2.1-1
- Package managers might cache these packages and even yanking and re-uploading them might cause problems for users
We should have a way to increment the build number on Package Cloud packages in such cases:
Add ability to increment build number in Packag... (#36333 - closed)
For context, we had a similar issue an year ago but I think the one above has a different goal: https://gitlab.com/gitlab-org/gitlab-runner/-/issues/29191+
Releases are SLOW. We have one pipeline. It's huge. It does a bunch of things which make the release pipeline slow and fragile.
We should separate and optimize our release pipeline to make iterating on releases easier and faster.
Separate and optimize release tag pipelines (#36334)
Runner Helper images should be used with the ve... (#36335 - closed)
Security repo's pipelines pushing resources can... (#29056)
Runner release process should check the existen... (#36510)
Runner release process should automatically che... (#36511)