Skip to content

Update importing repository guard in the delete tags service

David Fernandez requested to merge 360652-update-delete-tags-service-guard into master

🛡 Context

We're currently implementing a data migration on the Container Registry. This migration is going to be driven by the rails backend. For all the nitty-gritty details, see &7316 (comment 897867569).

Migrating an image repository is done in two steps: pre_import and import. Due to how steps are processed in the container registry, it is highly important that during the import step there are no write operations to the image repository. To achieve that, we lock the image repository in "read only" mode during the import step.

One of the most used write operation is destroying a tag in the image repository. This can be done by two ways:

  1. Automated, by cleanup policies.
  2. Manual, done by users using the UI.

Both ways will end up using the delete tags service. As such, in !79660 (merged), we added a guard to the service: if the image repository is in the importing step, the service execution is rejected with an error.

During the rollout of the migration on gitlab.com, we observed that we could have tags destruction during the pre_import step. This would then make the import step fail on the Container Registry side as the number of tags doesn't match. This would abort the migration and will be retried at a later time.

We can't block user operations during the pre_import step (that's (2.)) but we could block cleanup policies from removing tags (that's (1.)) so that the migration will not fail. The cleanup policy execution will fail but that's totally fine. Cleanup policies are executed on a recurrent schedule. If the current execution fails, it's ok, the next execution will resume deleting tags.

That's issue #360652 (closed).

🔬 What does this MR do and why?

  • Update the delete tags service importing repository guard to not allow cleanup policies use the service in the pre_import and pre_import_done migration state.
  • Update the related spec.

We don't have any changelog because the whole migration is gated behind several feature flag. It is currently only enabled for gitlab.com.

🖥 Screenshots or screen recordings

n / a

How to set up and validate locally

  1. Setup GDK with the registry support.
  2. Push several tags to an image, let's say a, b and c:
    $ docker build -t gdk.test:5000/<project_path>/bananas:a .
    $ docker build -t gdk.test:5000/<project_path>/bananas:a .
    $ docker build -t gdk.test:5000/<project_path>/bananas:a .
    
    $ docker push gdk.test:5000/<project_path>/bananas:a
    $ docker push gdk.test:5000/<project_path>/bananas:b
    $ docker push gdk.test:5000/<project_path>/bananas:c
  3. Update the container image to the pre_import migration state:
    ContainerRepository.last.update!(migration_state: 'pre_importing', migration_pre_import_started_at: Time.zone.now)
  4. Try to delete a tag from a cleanup policy:
    Projects::ContainerRepository::DeleteTagsService.new(Project.last, User.first, { tags: %w[a], container_expiration_policy: true }).execute(ContainerRepository.last)
    # => {:message=>"repository importing", :status=>:error}
    • Execution aborted
  5. Try to delete a tag not from a cleanup policy:
    Projects::ContainerRepository::DeleteTagsService.new(Project.last, User.first, { tags: %w[a] }).execute(ContainerRepository.last)
    # => {:deleted=>["a"], :status=>:success}
    • Execution successful

🚦 MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by David Fernandez

Merge request reports