Handle null tokens obtained from CloudConnector::Tokens

Problem

We currently return tokens from Tokens.get as is, i.e. without verifying they are present. However, there are cases where tokens can be null for SM/Dedicated customers, such as when their Duo add-on (and token) expires, at which point TokenLoader will find no tokens in postgres:

module CloudConnector
  module Tokens
    class TokenLoader
      # for SelfManaged/Dedicated instances we are using an instance token synced from CustomersDot
      def token
        ::CloudConnector::ServiceAccessToken.active.last&.token
      end
    end
  end
end

https://gitlab.com/gitlab-org/gitlab/-/blob/9ba76a6a24f5bb8edbcfdba73d193bee9a8e22e2/ee/lib/cloud_connector/tokens/token_loader.rb

We currently handle nulls in some parts of the code base, but often we just write a null token to the Authorization header. This results in a large amount of 401 noise in AI gateway logs (>21k events in the past week), because we send an empty Authorization: Bearer header, which results in JWT signature errors: Not enough segments (the library will try to parse an empty string): https://log.gprd.gitlab.net/app/r/s/1ZY1o

See also https://gitlab.com/gitlab-com/runbooks/-/blob/master/docs/duo/triage.md#cause-jwt-access-token-empty-or-malformed

Proposal

I suggest we return a TokenResponse from Tokens::get instead, so that we can clearly indicate Success or Failure to callers:

class Tokens::Success
  attr_reader :token
  ...
end

class Tokens::Error
  attr_reader :message
end

Callers are then forced to inspect the return value and act accordingly. For example, they can render 401 or raise an error.

Edited by Matthias Käppler