Can we remove CustomersDot from AI development?
Problem
A common pain point in AI development is the need to use and/or set up CustomersDot when testing AI features end-to-end in a self-managed setup. Since self-managed "mode" is the default (simulating .com is an opt-in configuration step by setting GITLAB_SIMULATE_SAAS=1
), this adds an additional component to the two other required systems, the AI gateway and GitLab Rails.
Proposal
Investigate approaches that would allow us to bypass CDot even when simulating self-managed use. There are three primary problems that need to be solved:
-
Provisioning an access token for the AI gateway. Typically, this is synced in for self-managed instances from CDot using a daily sync job. We think we may have a solution for this already with the recent addition of the
CLOUD_CONNECTOR_SELF_SIGN_TOKEN
env var, which was added in !154593 (merged) to support Custom Model development (which supports air-gapped customers who cannot sync with CDot either.) Setting this env var in gitlab-rails will make the instance a token authority and self-issue access tokens instead of loading it from the database.- NOTE: the
use_self_signed_token?
check is currently also behind a FF for Custom Model experimentation, which means there is some coupling to this work stream here. We would need to decouple this further to use this as a general development path. - Alternatively, we could not require authN with the AI gateway by setting
AIGW_AUTH__BYPASS_EXTERNAL=True
. However, this will require changes to the Rails monolith since we currently always expect a token to exist prior to sending a request. We would need to make these code paths "nil-token-safe". It would also not help with use cases where we do want to test authN/authZ changes.
- NOTE: the
- Provisioning add-on purchase data. This, too, is typically synced in from CDot after purchasing an add-on. Without syncing in this data, we would need to create it manually somehow (e.g. with a Rake task or some other mechanism.)
-
Removing the dependency on Cloud Licensing. This is likely strongly coupled to item 2. We currently assume and enforce the presence of an
Online Cloud License
in self-managed instances that again is synced with CDot. This is in contrast to using aLegacy License
file, which is the primary type of license most GitLab Contributors use today and is sufficient for .com mode AI development.
We should investigate what we could do allow developers to run GitLab in self-managed mode without CDot by looking at these 3 problems together.
Edited by Matthias Käppler