Ops Showcase Presentation: Moving to the Next-Generation Conainer Registry with Minimal Downtime

✨ Summary of the issue

In Milestone 8.8, GitLab launched the MVC of the Container Registry. This feature integrated the Docker Distribution registry into GitLab so that any GitLab user could have a space to publish and share container images.

However, there was an inherent limitation with Docker Distribution, as all metadata associated with a given image/tag was stored in the object storage backend. This made using that metadata to build API features (like storage usage visibility, sorting, and filtering) unfeasible. The most recent Container Registry update added a new PostgreSQL backend, which is used to store the metadata. Additionally, this new version also includes an automatic online garbage collector to remove untagged images and recover storage space.

We needed a way for self-managed users to import object storage metadata into the new database, so that users with existing instances could start participating in the beta for the metadata database feature. We required a way to preserve data consistency, minimize downtime, and keep the process easy enough that any self-managed admin could complete an import.

To do this, we designed a single procedure that could either be ran as a single step command or a multistep command. The multistep process is more complex, but allows the user to greatly reduce the overall downtime while importing metadata. The linked video shows the multistep command, because the single step command is boring in comparison.

🔑 Relevant Details

❓ Questions

NOTE: Using the format noted below, each question should be threaded. Answers can be placed as subpoints under each threaded question, tagging the original team member who asked the question

{Insert Question} - {tag presenter username}