Implement secrets manager client JWT auth

Resolves #510131 (closed)

Background

Currently, GitLab's secrets management system communicates with OpenBao through a proxy server. While this works, it adds complexity to our infrastructure and creates an additional point of failure. We want to simplify this by having GitLab authenticate directly with OpenBao using JSON Web Tokens (JWT).

What's Changing

This change introduces JWT-based authentication for GitLab's communication with OpenBao, eliminating the need for the proxy server. A key advantage of JWT-based authentication over AppRole is that JWTs can carry rich contextual information that enhances our audit capabilities.

GitLab already has JWT generation classes (Jwt and JwtV2), but these are specifically designed for CI/CD build authentication. They include CI-specific claims and behaviors that aren't relevant for secrets management. Instead of adapting these classes, we've created a new SecretsManagerJwt class that extends from JwtBase. This gives us a clean implementation focused solely on generating tokens for OpenBao authentication, while still leveraging GitLab's core JWT infrastructure.

Our JWT implementation includes rich contextual information in its claims that helps us understand and audit every interaction with OpenBao:

• User identity for tracking who performed each operation
• Project context for understanding the scope of operations
• Standard JWT claims for security validation and token expiration

This level of detail in our authentication tokens means that every operation in OpenBao can be traced back to the specific user and project that initiated it, providing much better audit trails than would be possible with AppRole authentication. When investigating security events or understanding system usage, this granular information becomes invaluable.

We've also reorganized the code to properly handle JWT authentication:

• Moved OpenBao client interactions to services and finders where user context is available
• Updated SecretsManagerClient to authenticate using these custom JWTs

Why Services

Moving client interactions to services isn't just about code organization - it's driven by the need for proper JWT authentication. These layers naturally have access to user context (i.e. current_user) needed for generating JWTs, making them the right place to handle OpenBao communication. Injecting this user context into the model layer would be architecturally awkward, as models should focus on representing domain objects and their validation rules rather than knowing about the current user making a request.

For example, when we create a secret, the service already knows about current_user through GitLab's service pattern. This makes it a natural place to generate JWTs and handle authenticated communication with OpenBao. The model, on the other hand, can stay focused on what makes a secret valid - its name, description, environment rules, etc. - without needing to understand anything about the user performing the operation.

How to test locally

I used the openbao server from GDK and didn't setup the proxy anymore.

Then I set up the needed JWT and policy configurations through the bao CLI:

First, I enabled the JWT authentication method in OpenBao:

bao auth enable -path=gitlab_rails_jwt jwt

Then I configured the JWT authentication with my GDK GitLab OIDC discovery URL and the expected issuer:

bao write auth/gitlab_rails_jwt/config \
    oidc_discovery_url="http://gdk.test:3000" \
    bound_issuer="http://gdk.test:3000"

I created a policy that grants comprehensive permissions for managing secrets:

bao policy write secrets_manager - <<EOF
path "*" {
  capabilities = ["create", "read", "update", "delete", "list", "sudo"]
}
EOF

Finally, I created a JWT role app that validates tokens and assigns the appropriate policy. This role specifies which JWT claims to expect and what permissions to grant upon successful authentication:

bao write auth/gitlab_rails_jwt/role/app \
    role_type=jwt \
    bound_audiences=openbao \
    user_claim=user_id \
    token_policies=secrets_manager

After this, I was able to authenticate with JWT within Rails without the help of the proxy.

Some sample queries and mutations I tested this on:

mutation {
  projectSecretsManagerInitialize(input: {projectPath:"root/test-openbao"}) {
    projectSecretsManager {
      status
      ciSecretsMountPath
    }
    errors
  }
}

mutation {
  projectSecretCreate(input: {
    projectPath:"root/test-openbao",
    name: "testnoproxy_secret",
    description:"a new secret to test",
    environment:"staging",
    branch:"development",
    value:"testnoproxyvalue"
  }) {
    projectSecret {
      name
    }
    errors
  }
}

query {
  projectSecrets(projectPath: "root/test-openbao"){
    nodes {
      name
      description
      environment
      branch
    }
  }
}

mutation {
  projectSecretDelete(input: {
    projectPath:"root/test-openbao",
    name: "testnoproxy_secret"
  }) {
    projectSecret {
      name
    }
    errors
  }
}