Update import request parameters
🎒 Context
There is an ongoing migration in the Container Registry to move data to a database. We are now in the phase where we can start moving existing container images.
This phase is going to be driven by Rails. Basically, this means that Rails will inspect the existing container repositories, pick the next eligible one and call the container registry API to start the migration process.
The migration operation is a two step process. This means that rails will get notified when a step is completed.
In Add support for the gitlab container registry API (!78452 - merged), we added support for the new endpoints that the Container Registry implemented in a dedicated API: the gitlab api.
Among those endpoints, there is the one that actually start the migration step (step one or two).
That request parameters of that endpoint has been updated recently.
We now need to update the rails client to use those new endpoints. That's issue #353355 (closed).
Last piece of context, this whole migration is happening on gitlab.com only and behind feature flags to control different aspects of the migration (among other things, the speed).
🤔 What does this MR do and why?
- In
lib/container_registry/gitlab_api_client.rb
, update the#start_import
method so that the request parameters are updated to reflect the changes of container-registry!896 (merged). - Update the related spec.
🖼 Screenshots or screen recordings
See next section
🔧 How to set up and validate locally
-
Check out the container registry project and build the binaries with
$ make
-
Update
<gdk_root>/registry/config.yml
with:database: enabled: true host: <path to gdk root>/postgresql port: 5432 user: dbname: registry password: gc: disabled: true # just to reduce the noise
-
Connect to psql with
$ gdk psql
-
Create the
registry
database withCREATE DATABASE registry;
-
Execute the registry migrations within the container registry project with :
$ ./bin/registry database migrate up <path to gdk root>/registry/config.yml
-
Start the container registry from its project with :
$ ./bin/registry serve <gdk_root>/registry/config.yml
-
Ensure that you have no errors in the logs
Understand that we're going to create a container repository on the rails side but not on the Container Registry side. Using the updated endpoints here will make the Container Registry API return an error. That's ok, we can still inspect the request parameters on the Container Registry logs.
First, create a dummy container repository and get hold of the client (in a rails console):
image = FactoryBot.create(:container_repository, project: Project.last)
client = image.gitlab_api_client
Now, let's start step 1 of the migration:
-
client.pre_import_repository(image.path) => :not_found
- Check the Container Registry logs and we see:
{"content_type":"text/plain; charset=utf-8","correlation_id":"01FWZYYQXTPBNE2H1H13Y14MEJ","duration_ms":0,"host":"gdk.test:5000","level":"info","method":"PUT","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"172.16.123.1:51036","remote_ip":"172.16.123.1","status":404,"system":"http","time":"2022-02-28T11:45:26.586+01:00","ttfb_ms":0,"uri":"/gitlab/v1/import/root/commit_teamplte/test_image_1/?import_type=pre","user_agent":"GitLab/14.9.0-pre","written_bytes":19}
- Notice the
import_type=pre
in the logs.✅
Now let's try to start step 2 of the migration
-
client.import_repository(image.path) => :not_found
- Check the Container Registry logs and we see:
{"content_type":"text/plain; charset=utf-8","correlation_id":"01FWZYV212QCE3Z2DNXWPAKQ25","duration_ms":0,"host":"gdk.test:5000","level":"info","method":"PUT","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"172.16.123.1:51033","remote_ip":"172.16.123.1","status":404,"system":"http","time":"2022-02-28T11:43:25.858+01:00","ttfb_ms":0,"uri":"/gitlab/v1/import/root/commit_teamplte/test_image_1/?import_type=final","user_agent":"GitLab/14.9.0-pre","written_bytes":19}
- Notice the
import_type=final
in the logs.✅
🔮 Additional testing
There are several end to end tests that are planned to happen in staging. No doubt that any issue with the changes of this MR will get caught fairly quickly.
🚥 MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.