Cleanup policies: refresh the container registry token before deleting tags.
Summary
As stated in #288812 (closed), here is the cleanup process and all its network requests:
- Get the tags list (GET
/v2/#{name}/tags/list
)- The whole list is retrieved in a single call. No pagination. The list is simply an array of names.
- For each tag, get its manifest (GET
/v2/#{name}/manifests/#{reference}"
)- The policy specifies how many tags to retain. To compute that, the backend needs to have the
created
timestamp and order the list by it. Thecreate
timestamp is not returned by (1.), so a request per tag must be done to get this field.
- The policy specifies how many tags to retain. To compute that, the backend needs to have the
- For each tag to delete, delete it (DELETE
/v2/#{name}/tags/reference/#{reference}
)
(1.) + (2.) could take so much time that the token used to make those requests to the container registry can expire.
Example: https://log.gprd.gitlab.net/goto/dee248e81101a9abbee3d0f9f0701373 -> All the DELETE requests end up in a 401 Unauthorized
This token has an availability limited in time. So if (1.) + (2.) take too much time, we might very well go beyond this availability period.
Improvements
Before step (3.), reset the container registry token. Do it for both conditions:
- When the gitlab container registry is used
- When a third party container registry is used
Risks
Low
Involved components
app/services/projects/container_repository/delete_tags_service.rb
Edited by Tim Rizzi