Cleanup policies: refresh the container registry token before deleting tags.

Summary

As stated in #288812 (closed), here is the cleanup process and all its network requests:

  1. Get the tags list (GET /v2/#{name}/tags/list)
    • The whole list is retrieved in a single call. No pagination. The list is simply an array of names.
  2. For each tag, get its manifest (GET /v2/#{name}/manifests/#{reference}")
    • The policy specifies how many tags to retain. To compute that, the backend needs to have the created timestamp and order the list by it. The create timestamp is not returned by (1.), so a request per tag must be done to get this field.
  3. For each tag to delete, delete it (DELETE /v2/#{name}/tags/reference/#{reference})

(1.) + (2.) could take so much time that the token used to make those requests to the container registry can expire.

Example: https://log.gprd.gitlab.net/goto/dee248e81101a9abbee3d0f9f0701373 -> All the DELETE requests end up in a 401 Unauthorized 😿

This token has an availability limited in time. So if (1.) + (2.) take too much time, we might very well go beyond this availability period.

Improvements

Before step (3.), reset the container registry token. Do it for both conditions:

  • When the gitlab container registry is used
  • When a third party container registry is used

Risks

Low

Involved components

app/services/projects/container_repository/delete_tags_service.rb

Edited by Tim Rizzi