Create QA tests for the Container Registry
Context
This is a followup from a conversation between the Package and Delivery teams, recorded in https://www.youtube.com/watch?v=LmZI_Pf2lk0.
Related to gitlab-com/gl-infra&316.
Problem
Currently, there are no QA tests for the Container Registry. We have an extensive unit and integration test suite in the registry codebase, which runs as part of the GitLab CI pipeline that precedes every release. However, there are none triggered from outside the registry.
To boost confidence on the registry stability (especially with the upcoming metadata database), it's essential to create QA tests that can be integrated into the deployment pipeline.
This is a prerequisite to adding support for running database migrations as part of the current deployment pipeline, allowing us to deploy the new registry backed by the metadata database to the various environments.
Solution
Create QA tests for the registry. Its HTTP API is already well tested and covered with the current integration tests, but there are no tests for external interaction with the Docker client CLI, namely for the docker push
and docker pull
commands. These tests can be later expanded to include further coverage.
This is the list of steps for this effort (TBD):
-
Create QA tests for the Docker client commands, using the current version of the registry, with no metadata database;
-
Work with the Delivery team to create a pipeline for these tests and have them running as part of the current deployment process;
-
Expand the QA tests, adding support for the metadata database, including testing the database migrations and the Docker client commands.
Once done, we can work with the Delivery team to integrate support for running database migrations as part of the current deployment process.