Container Registry - Test GC performance improvements for S3 in dev.gitlab.org
We have been working on performance improvements for the Container Registry garbage collection algorithm.
The first stage is to improve performance for S3 (gitlab-org/gitlab#31071 (closed)). This is scheduled for 12.8, and we have split the work in two:
- Mark phase improvements: gitlab-org/container-registry!23 (merged)
- Sweep phase improvements: gitlab-org/container-registry!24 (merged)
We want to test these improvements against a real and sizeable S3 repository before releasing a new version of the Container Registry.
As the registry in dev.gitlab.org is running against an S3 bucket, we were wondering if it would be possible to deploy a custom version of the registry there, including the performance improvements.
The intention would be to run the garbage collector, making sure everything is OK and possibly comparing the execution time between the current version and the new one.
Considering the above:
- Is it possible/feasible to test the performance improvements there?
- If so, how should we proceed?
- How large is the dev.gitlab.org S3 repository?
We should probably open separate issues for the mark and sweep phase tests, as they can be tested independently, but we can discuss the overall strategy here first as they’re related.
/cc @hswimelar @stanhu