Allow proxying requests to another registry for new repositories
Context
As part of &3014, and to support the proposal in #191 (closed), we need to provide a way to allow existing registry instances (not backed by the metadata database) to proxy requests for new repositories (those not found in the existing registry storage backend) to new registry instances backed by the metadata database.
Proposal
sequenceDiagram
autonumber
participant C as Client
participant R1 as Proxy Registry (public)
participant B1 as Proxy Registry<br>Storage Backend
participant R2 as Target Registry (private)
participant B2 as Target Registry<br>Storage Backend
C->>R1: HTTP /v2/<name>/...
R1->>B1: The repository<br>with <name> exists?
B1->>R1: Yes/No
alt Yes
%%rect rgb(229, 255, 204)
R1->>B1: Fulfil request
R1->>C: Response
%%end
else No
rect rgb(255, 255, 204)
R1-xR2: Proxy request
R2->>B2: Fulfil request
R2->>R1: Response
R1-xC: Proxy response
end
end
The idea is to inspect every incoming request and, based on the repository path embedded in every request URL, determine if the target repository is known to the existing registry instance or not.
To determine if the repository is known, the registry must issue a stat request against the storage backend, determining if the repository already exists in its bucket. In case it exists, the instance should proceed and fulfill the client request, otherwise, it should proxy the request to another registry instance, backed by the metadata database. The new registry instance should receive the request, fulfill it, and return the response to the client.
To make this behavior optional, we should add a new proxy
section in the configuration file, under a new migration
section, which groups all the new (temporary) functionalities required to support the migration effort:
migration:
proxy:
enabled: true
url: https://registry2.gitlab.com
By placing this functionality behind a feature flag (toggled by the migration.proxy.enabled
setting), and considering that the database functionality is also behind its feature flag, we can use the same registry version for both the "old" (not backed by a metadata database) and new registry instances. The redirect feature flag should be toggled for the "old" instances only, while the database feature flag should be toggled for the new instances only.
It's important to note that for this to work in production, both old and new registries must be configured with the same shared HTTP secret (http.secret
setting). Furthermore, the new registry should be configured to advertise itself with the old registry hostname (http.host
setting), otherwise, the Location
header in POST
/PUT
responses will contain its (private) hostname instead of the old registry public hostname.