Proposal: Freeze the development of Container Registry features beyond 12.10, in favour of the online garbage collection effort
Context
At the time of writing, the highest priority task for the Container Registry is to implement online garbage collection (&2313 (closed)). To achieve this, first, we'll have to move the image metadata from the filesystem into a database (gitlab#207147 (closed)).
The metadata database will also enable us to drastically improve the registry API throughput (major concern) and extend it with additional functionality (e.g. pagination, filtering, advanced querying, etc.).
The above is something that we've been struggling with due to the constraints imposed by the filesystem backend. As an example, to list the tags of a given repository, we currently have to traverse the filesystem recursively and parse/inspect a large number of files. With a metadata database, this will become a single SELECT
query.
Problem
Lately, we have seen the number of Container Registry feature requests increase. The development team has been trying to address these in the best way possible. However:
-
A good portion of these feature requests ends up being unfeasible due to the limitations imposed by the registry filesystem backend (i.e., by the fact that we don't have a database that we can efficiently query/manipulate to obtain the desired information). gitlab#38047 and #13 (closed) are good examples.
-
When it's not completely clear if features are feasible or not, a considerable amount of time is required for investigating and experimenting with possible solutions. When a feature ends up being unfeasible (either due to missing data or performance concerns), the time spent in this process ends up being "lost". We learn important things from the experiments, but the time could have been used towards our main priorities;
-
Even if a new feature is feasible, it'll likely require changes that will need to be refactored later once we have the metadata database. So these will end up costing us more time in the long run.
Proposal
We should freeze/postpone the development of features for the Container Registry beyond those planned for the 12.10 release (gitlab#209298 (closed)). This will allow the development team to focus on implementing the metadata database and online garbage collection.
Unless it's an important bug fix or a straightforward performance optimization, we should not commit to make additional changes. Once we have the metadata database, we'll be able to become more efficient and implement new features without the need for ephemeral and time consuming workarounds.