Skip to content

Add New AiGateway Scenarios

Valerie Burton requested to merge add-new-ai-gateway-scenarios into master

What does this MR do and why?

Relates to https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/issues/2427

This MR adds a new AiGatewayBase that other scenarios can inherit from to allow flexibility in various configurations, such as defining if a cloud license should be used and if seats should be assigned.

The following new scenarios have also been added, which inherit from this AiGatewayBase:

  1. AiGateway - same as current scenario today, GitLab instance with cloud license + AI add-on with seat assignment
  2. AiGatewayNoSeatAssigned - GitLab instance with cloud license + AI add-on, no seat assignment
  3. AiGatewayNoLicense - GitLab instance with no cloud license, no seat assignment
  4. AiGatewayNoAddOn - GitLab instance with different cloud license that has no AI add-on (will be a different code from the current code set in QA_EE_ACTIVATION_CODE today), no seat assignment

This MR also updates any applicable code suggestions references that can now be renamed to Duo Pro.

These scenarios will allow us to develop new orchestrated E2E tests for the test cases listed in here

How to set up and validate locally

  1. Check out this MR's branch locally
  2. Export the following ENV vars. The QA_EE_ACTIVATION_CODE value can be found in 1Password
export QA_EE_ACTIVATION_CODE=<1Password>
export GITLAB_LICENSE_MODE=test
  1. Run this command for any one of the following new scenarios:
exe/gitlab-qa Test::Integration::AiGatewayNoLicense EE --no-tests

This will spin up the docker containers, but not run any tests.

  1. Run the following cURL command to do a manual test for the code suggestions endpoint:
curl --location 'http://<gitlab container URL + port>/api/v4/code_suggestions/completions' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <gitlab admin access token>' \
--data '{
    "prompt_version": 1,
    "language_identifier": "ruby",
    "current_file": {
        "file_name": "/test.rb",
        "content_above_cursor": "def set_name(whitespace_name)\n    this.name = whitespace_name.",
        "content_below_cursor": "\nend"
    }
}'
  1. The response should return 401 Unauthorized for the AiGatewayNoSeatAssigned, AiGatewayNoLicense and AiGatewayNoAddOn scenarios (for AiGatewayNoAddOn, only if you set QA_EE_ACTIVATION_CODE to a separate cloud license activation code with no add-on provisioned to it). A response with a "fake code suggestion from PaLM Text" should be returned for the AiGateway scenario.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Valerie Burton

Merge request reports