Skip to content

Use AIGW docker image for rspec tests

Shinya Maeda requested to merge shinya.maeda-main-patch-9468 into main

What does this merge request do and why?

In order to run AI Gateway service container in GitLab-Rails CI, we need to make a few changes in AI Gateway itself:

  • Allow to set GOOGLE_APPLICATION_CREDENTIALS via string variable. Currently, AI Gateway requires a Vertex AI credential to be present at boot time (more specifically when container_application.init_resources() runs). Since we can't pass the credential file to the service container unless it's successfully booted, we have a chicken-egg problem here. If we can set GOOGLE_APPLICATION_CREDENTIALS as a string variable, we can workaround the problem.
  • Expose the default port 5052. This port is accessed by Rails instance in CI. This is not harmful to production environment as they can still specify the open port e.g. 8082:8082. And it's overridable e.g. -p 5052:5052 will override it.

Related to Support AI Gateway in Rails CI (gitlab-org/gitlab!148508 - merged)

How to set up and validate locally

  1. Run a docker instance with string cred and confirm the instance is successfully booted up.
     JSON_KEY=$(cat <<EOF
     {
       "type": "service_account",
       ...
     }
     EOF
     )
    
     docker run -e AIGW_VERTEX_TEXT_MODEL__JSON_KEY="${JSON_KEY}" registry.gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/model-gateway:e929af09
  2. Run a docker instance without string cred and confirm the instance failed.
    docker run registry.gitlab.com/gitlab-org/modelops/applied-ml/code-suggestions/ai-assist/model-gateway:e929af09

Merge request checklist

  • Tests added for new functionality. If not, please raise an issue to follow up.
  • Documentation added/updated, if needed.
Edited by Shinya Maeda

Merge request reports