endpoint_id: GET /api/:version/groups/:id/registry/repositories
Context
You can use the container registry API to return a list of container image repositories and their associated tags for a given groupID.
Problem to solve
Returning a list of tags for every image repository is an expensive query and can result in slow response times that are beyond GitLab's tolerated apdex threshold of 1 second.
This endpoint continues to display very poor performance - the p75 is over 5 seconds, with p99 sometimes spiking up to over 20 seconds.
Update the API to only return a list of image repositories. The tags can be pulled for a specific image repository rather than generating every tag for every group.
This means that the GET /groups/:id/registry/repositories endpoint will no longer support the tags and tags_count parameters. To obtain the tag count/list, users can use the GET /registry/repositories/:id endpoint, which will continue to support the tags and tags_count options as it does today.
Since this may introduce a breaking change, we'll need to consider this change for milestone 15.0
Reports shows this issue as a soon to ~"missed-SLO", but all teams involved agreed #329637 (comment 697533859) to have this resolved in %15.0 because is a breaking change.
Hey @sabrams , I noticed this comment after opening the MR.
I reused the Entities::ContainerRegistry::Repository entity and updated the project_container_repositories and group_container_repositories endpoints to use tags: false explicitly for better clarity. WDYT ?
Please feel free to suggest a different approach, this is one of my first MRs on the Ruby/Rails side so any tip/feedback is always welcomed.
Hi @feistel, thanks for picking this up! tags: false and tags_count: false is perfect. I completely overlooked that the conditionals were already in place. Feel free to ping me when the MR is ready for review or if you have any other questions!
Hey @sabrams, I've updated the MR and rebased on the main branch now that is ready for review but I noticed that it was targeting two different endpoints.
Thanks @feistel for pointing this out, I missed this when I first wrote up my explanation above. My understanding is that while the group endpoint has a much higher chance of low performance, the project endpoint could also suffer from performance problems.
@trizzi@10io do you have any thoughts on this? I'm inclined to deprecate for both the project and group, but if we have only been communicating about the group, maybe it's too late to introduce the project as another breaking change.
@sabrams I think we may have missed our chance with the project endpoint. We need to announce deprecations at least 2 milestones ahead of removal. We can announce a deprecation now, but we'd have to wait until 16.0 for the removal.
@feistelhttps://gitlab.com/api/v4/projects/:id/registry/repositories?tags=1
this endpoint returns only 100 tags per container
it would be ok if last tags returns, but the API returns tags in random order
as a temporaty solution i created another containers and turned on automatically clean up, but... it will be a problem soon again