GAR Integration: Custom client class

Integrate the Artifact Registry official client

It would be better to centralize the access to the official ruby client. This way, it's very easy to check for permissions.

The client comes in two versions (two different libraries/gems): the main client and the versioned client. The main client will use the versioned client behind the scenes. So what's the difference? Well, according to the documentation, the main client provides a stable API and interfaces to use.

From our quick code inspection, the main client provides only a way to build the versioned client. As such, we don't think that there is a huge difference between using the main client from the versioned client.

We suggest having a custom client class located in GoogleCloudPlatform::ArtifactRegistry::Client. That class will need to require a User and a GoogleCloudPlatform::ArtifactRegistry to be built.

The client will then need to expose three functions:

#repository.
- Return at least fields:
  - name
  - field
#docker_images
- expose the pagination and ordering options.
- return at least fields:
  - name
  - uri
  - image_size_bytes
  - upload_time
#docker_image
- return at least fields:
  - same as docker_images
  - tags
  - media_type
  - build_time
  - update_time

For return objects, either use the one provided by the official client or map them into simple hashes.

To setup the official client, we will also need to properly set:

the timeout.
the retry_policy.

For these, we can simply either use the default values if they are ok or use fixed values.

This client should be gated behind the Google Artifact Registry feature flag.

💥 Errors

Create custom classes for errors that we can have on the GAR client:

general error.
network error or can't connect to the repository error.
the repository is of the wrong format error.
can't find docker image error.

🛃 Permissions

Before calling the official client, this class will need to check the user permissions. The given User should have read_gcp_artifact_registry_repository on the Project related with the Integrations::GoogleCloudPlatform::ArtifactRegistry (see below).

We will need a new permission on the Project policy:

read_gcp_artifact_registry_repository granted to at least reporter users.

⚙ Technical details

The authentication is going to be handled by glgo.

Basically, we need to do the following steps:

Build a JWT with the proper claims (wlif) and issuer. Handled by this class.
Submit that JWT to glgo /token endpoint.
Get the "glgo" JWT.
Exchange that JWT for an access token (short lifespan).
Use that access token in the client to access the GCP API.

This sounds complex but fortunately, the official ruby client can handle all of this for us (except step (1.)).

Here is an example snippet:

project = Project.find(<project_id>)
user = User.find_by_username(<username>)
jwt = ::Integrations::GoogleCloudPlatform::Jwt.new(project: project, user: user, claims: { audience: 'https://glgo.staging.runway.gitlab.net', wlif: '//iam.googleapis.com/projects/604150606412/locations/global/workloadIdentityPools/10io-testing/providers/gstg' })

jwt_encoded = jwt.encoded

credentials = {
  'type' => 'external_account',
  'audience' => '//iam.googleapis.com/projects/<google project id>/locations/global/workloadIdentityPools/<workload identity pool identifier>/providers/<provider identifier>',
  'token_url' => 'https://sts.googleapis.com/v1/token',
  'subject_token_type' => 'urn:ietf:params:oauth:token-type:jwt',
  'credential_source' => {
    'url' => 'https://glgo.staging.runway.gitlab.net/token',
    'headers' => { 'Authorization' => "bearer #{jwt_encoded}" },
    'format' => { 'type' => 'json', 'subject_token_field_name' => 'token' }
  }
}

require "google/cloud/artifact_registry/v1"

client = ::Google::Cloud::ArtifactRegistry::V1::ArtifactRegistry::Client.new do |config|
  config.credentials = ::Google::Cloud::ArtifactRegistry::V1::ArtifactRegistry::Credentials.new(
    Google::Auth::ExternalAccount::Credentials.make_creds(json_key_io: StringIO.new(JSON.dump(credentials)), scope: 'https://www.googleapis.com/auth/cloud-platform')
  )
end

request = ::Google::Cloud::ArtifactRegistry::V1::ListDockerImagesRequest.new(parent: 'projects/<project_id>/locations/<location>/repositories/<repo name>')
resp = client.list_docker_images(request)
puts resp.response.docker_images.map(&:name)

The above required these specific gems:

gem 'googleauth', '~> 1.6.0'
gem 'google-cloud-artifact_registry-v1', '~> 0.9.1'

We can see that we can provide a custom credentials structure to will handle everything.
Unfortunately, we need to JSONize the structure first to then use make_creds. 🤦 We have an issue for this aspect: https://github.com/googleapis/google-auth-library-ruby/issues/466.

⚠ Points to consider

Caution with the code organization. Step (1.) and creating the credentials hash should be a common/central function for all clients.
The integration being an Saas only feature, we should locate this class and its helpers on the EE side.

Plan

Edited Jan 16, 2024 by David Fernandez