Connect a service to Vertex AI Search
Objective
In order to support Documentation tool of Duo Chat on SM instances, we need to provision resources to Vertex AI Search and BigQuery services in a GCP project and let ai-gateway
Cloud Run services connect to them.
Here is the architectural docs:
flowchart LR
subgraph GitLab managed
subgraph AIGateway
VertexAIClient["VertexAIClient"]
end
subgraph Vertex AI Search["Vertex AI Search"]
subgraph SearchApp1["App"]
direction LR
App1DataStore(["BigQuery"])
end
subgraph SearchApp2["App"]
direction LR
App2DataStore(["Cloud Storage / Website URLs"])
end
end
end
subgraph SM or SaaS GitLab
DuoFeatureA["Duo feature A"]
DuoFeatureB["Duo feature B"]
end
DuoFeatureA -- Semantic search --- VertexAIClient
DuoFeatureB -- Semantic search --- VertexAIClient
VertexAIClient -- Search from Gitlab Docs --- SearchApp1
VertexAIClient -- Search from other data store --- SearchApp2
Timeline
- ~ 2024-05-03: Provision Vertex AI Search in the designated GCP project.
- ~ 2024-05-06: Roll out the feature flag in gitlab.com to make sure that there is no performance degredation.
- ~ 2024-05-09: Enable the feature flag in GitLab-Rails by default to release the feature in GitLab 17.0.
Proposal
- In a non-Runway GCP project:
- Service maintainers build a Docker image to Setup in Vertex AI Search.
- Service maintainers deploy the Docker image to Cloud Run Jobs and periodically run it or Pipeline schedule (e.g. every day).
- In the Runway GCP project (i.e.
gitlab-runway-staging/production
):- Allow the service account used in AI Gateway service to access Vertex AI Search service in the non-Runway GCP project (IAM roles). This is similar to https://runway-docs-4jdf82.runway.gitlab.net/guides/cloud-sql/ and how
unreview-poc-390200e5
is used bygitlab-runway-production
today. - In AI Gateway
.env
file, we will specify the following variables:AIGW_VERTEX_SEARCH__PROJECT=non-runway-project
AIGW_VERTEX_SEARCH__LOCATION=us-central1
AIGW_VERTEX_SEARCH__ENDPOINT=us-central1-aiplatform.googleapis.com
- Allow the service account used in AI Gateway service to access Vertex AI Search service in the non-Runway GCP project (IAM roles). This is similar to https://runway-docs-4jdf82.runway.gitlab.net/guides/cloud-sql/ and how
Hint from @fforster
The Cloud Run service accounts, which are managed by Runway, have the
aiplatform.user
role, granting it 277 permissions, only one of which is used (aiplatform.endpoints.predict). This IAM membership was not created manually. It is managed via thegl-infra/config-mgmt
repository, using Terraform. The configuration is at: https://ops.gitlab.net/gitlab-com/gl-infra/config-mgmt/-/blob/main/environments/ai-assisted-legacy-prd/service_accounts.tf?ref_type=heads#L10Caveat: Terraform's management of IAM memberships is not authoritative. In other words, there are IAM memberships in
unreview-poc-390200e5