Expose container repository sizes

☄ Context

Container Registry data is partially present on the Rails backend. The other part is in the Container Registry itself and we can access that part using APIs.

We have two APIs at our disposal:

1. The standardized API.
   • That's the API that is present in all OCI registries.
2. The API customized for GitLab needs.
   • This API only exists in the GitLab fork of the Container Registry.

Users can request container image details through Rest and GraphQL APIs. Those APIs also power the frontend. Now, fields for which we need to ping the Container Registry are gated behind parameters. The reasoning is that by default, we serve a short form of the container image data and that short form is containing all the parts from the Rails backend. If users need additional attributes, they need to ask for them and the backend will ping the Container Registry APIs to get them.

This MR adds one additional attribute, the (deduplicated) size of a container image and expose it in APIs. That's exactly issue #347349 (closed). This attribute is available on an endpoint (repository details) that exists in the custom API (2.).

That endpoint exists only on gitlab.com and for more recent container images for now.

Last thing to understand, since reading this attribute triggers an API call to the Container Registry, size will only be available on APIs that request the details of a single Container Image. Endpoints that list Container Images will not be updated. That would introduce a n+1 situation with the API calls.

🔬 What does this MR do and why?

• Update the gitlab API client.
  • Add support for multiple tokens. Up to now, we used a JWT token with type import. The problem is that we need a JWT token of type repository for the endpoint we need.
  • Add support for the get repository details endpoint
• Update the model to add the size attribute
• Update the Rest API to expose the size attribute, only if a size=true parameter is sent.
• Update the GraphQL API to expose the size attribute.
• Update the related specs
• Update the related documentation

📸 Screenshots or screen recordings

1️⃣ Rest API

Without the size parameter:

With the size parameter:

2️⃣ GraphQL

👽 How to set up and validate locally

🛠 Setup the Container Registry

The custom API is currently under heavy work and depends on the existence of a database for the Container Registry. These details are currently not handled by GDK, so we need to set up the Container Registry server manually.

1. Set up the container registry in GDK.
2. Start everything $ gdk start
3. Stop the Container Registry from gdk: $ gdk stop
4. Check out the Container Registry project and build the binaries with $ make
5. Update <gdk_root>/registry/config.yml with:

   database:
     enabled: true
     host: <path to gdk root>/postgresql
     port: 5432
     user:
     dbname: registry
     password:
   gc:
     disabled: true # just to reduce the noise

6. Connect to psql with $ gdk psql
7. Create the registry database with CREATE DATABASE registry;
8. Execute the registry migrations within the container registry project with : $ ./bin/registry database migrate up <path to gdk root>/registry/config.yml
9. Start the container registry from its project with : $ ./bin/registry serve <gdk_root>/registry/config.yml
10. Ensure that you have no errors in the logs

🤹 Play with the size field

1. Build an image and push it to any project.
2. Open the container image in the UI and note the container image id or check ContainerRepository.last.id in a rails console.

Test the rest API:

1. Go to /api/v4/registry/repositories/<container_repository_id>?size=true
   • Try again removing the size parameter

Test the GraphQL API:

1. Go to /-/graphql-explorer and use this query:

   query {
     containerRepository(id: "gid://gitlab/ContainerRepository/<container_repository_id>") {
       id
       size
     }
   }

   • Try again without reading the size field

🚥 MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.