Skip to content

Onboard Simon Tomlinson as the database stable counterpart

Context

As per !743 (comment 667519383), @stomlinson is now the database stable counterpart for the Container Registry. This is documented at https://about.gitlab.com/handbook/engineering/development/enablement/database/stable.html.

Welcome!

Background

Historical Context

There was not an official database stable counterpart for the Container Registry until now. Still, we have been working closely with the Database team (more precisely with @abrandl and @iroussos) since the beginning.

Here are a few links for the most important milestones during the development of the Container Registry database, in chronological order:

  1. Define container registry database schema to help drive online garbage collection - This was when the investigation started. This particular comment has an explanation of two of the options we considered along with an example of how the metadata is currently stored in a nested filesystem structure and how that translates to a database representation, which might be helpful.

  2. Container registry on PostgreSQL - A discussion we had on whether the database would be part of the Rails instance/cluster or a separate one. More details in #93 (closed) as well. We end up going with a dedicated database (exclusively owned by the registry) in a separate dedicated cluster for GitLab.com. For self-managed, it will be possible to have it as a separate logical database in the same instance (for simplified maintenance) once PostgreSQL 12 becomes the minimum required version.

  3. Discussion/review of the database schema and migrations - This was when we started working with the Database team to review the database schema. There is a lot of context in there, with multiple relevant MRs. gitlab#234255 (closed) and gitlab-com/www-gitlab-com!60918 (merged) are particularly useful. As a result, we come up with a new partitioned schema.

  4. Document database queries required for HTTP API - After settling on a new schema, we documented, profiled, and fine-tuned all queries required for the registry REST API.

  5. Online garbage collection spec - This is where we have come up with a specification for online garbage collection. The document includes all triggers/functions and an explanation of how they work.

  6. Define ideal number of hash partitions - Last but not least, we have investigated what would be the ideal number of partitions, based on rate and size requirements. In the end, we settled with 64 partitions.

Documentation