Skip to content

Update aborted registry import logic

Steve Abrams requested to merge 351337-improve-aborted-enqueuer into master

🏛 Context

We are preparing for Phase 2 of the Container Registry migration which involves importing all existing container repositories to the new platform (Phase 1 involved routing all new container repositories to the new platform). See &7316 (closed) for full details of how the import will work.

The EnqueuerWorker queries the container_repositories table, finding a repository to import and makes a request to the registry to start the process. Along the way, some imports are aborted. This is expected. The EnqueuerWorker also looks to see if there are any aborted imports retries the import or pre-import. Currently, we check the :migration_pre_import_done_at timestamp on the container_repository to decide if we should retry the import or pre-import. The problem is, rails really doesn't know the true state of the container repository import since that is all happening in the registry, so this is just a best guess.

What we would like to do is query the registry for the true state, and then make a better decision on which action to take based on that response. That is exactly what this MR does.

🔎 What does this MR do and why?

  1. Adds an additional request endpoint to ContainerRegistry::GitlabApiClient to fetch the import status for a container repository.

  2. Updates the logic when retrying an aborted import to base the action on the response from (1.). If the request results in a error, we fallback on the old logic, checking the container_repository.migration_pre_import_done_at timestamp.

Screenshots or screen recordings

N/A

How to set up and validate locally

Unfortunately, we cannot test the new API request as this migration process is all in development so many of the supporting pieces are being developed in parallel. All of this work is behind a set of feature flags so the worker that uses the state_machine and #retry_aborted_migration are not yet in use. The goal is to have both the rails and registry updates implemented in 14.8 so we can begin testing and adjusting the import workflow in 14.9. See &5523 (closed) for more info about the timeline.

✔ MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #351337 (closed)

Edited by Steve Abrams

Merge request reports