Use major release specific repos in our package server
Summary
On packages.gitlab.com, have seperate repos for each major release. This should help speed up indexing, as well as allow us to have filelist metadata files that are a more reasonable size.
(If we had the file turned on for gitlab-ee I estimate,based on our fips filelist, that the filelist file would be several hundred MB.)
Proposal
To fix the size issue with the repository filelist metadata, I propose we come up with a seperate repository for each major release. For example:
gitlab/gitlab-ee-16
gitlab/gitlab-ce-16
gitlab/gitlab-fips-16
This likely does not solve all the issues we have with the filelist metadata file in our current package server. But I believe this would be a necessary change in order to have the filelist be a reasonable size regardless of other things we explore for the generation and indexing.
This repository change also has a few added benefits:
- Forces users to be purposeful about their major upgrade, less likely to accidentally upgrade without looking at the release notes
- Increases likelihood that users are running our install script at least once a year, which may give us easier opportunity to make public key changes smoother for users
- In the future, easier for us to move old repos to colder storage, or different package servers, to improve cost and speed of the modern repos.
- It's also a lot easier for others to mirror our repos if they don't need to mirror all our older packages.
References
Note that we have currently left the filelist metadata turned on for the gitlab-fips
repository. If we are able to keep it on (without index performance impacts), we will get a better idea of what we can expect our yearly filelist metadata filesize to be for a major release line.
Update Mar 2023
The fips repo filelist file is 42MB compressed, over 500MB uncompressed. While this is far more reasonable than our main repos, which would be 10 times worse, it's still incredibly large for a filelist.