E2E tests: add SM code suggestion scenarios
What does this MR do and why?
Adds additional self-managed code suggestion E2E api tests. These tests will run in MRs.
- code generation
- code generation with streaming
Resolves https://gitlab.com/gitlab-org/quality/quality-engineering/team-tasks/-/issues/2301
Other fixes:
- fixes completion test to use a longer code prompt, as the previous prompt was getting code generation, so currently we are not invoking completion
- log the output of the response (useful for debugging incidents)
-
language_identifier
has been moved in the request to match where it actually is - better assertion to show the expected status code
- some refactoring
It turns out that our completion
test was actually using generation
due to the SMALL_FILE_TRIGGER
, i.e our prompt was too small so it would revert to generation.
To test the self-managed version of the test locally:
CHROME_DISABLE_DEV_SHM=true QA_IMAGE="registry.gitlab.com/gitlab-org/gitlab/gitlab-ee-qa:jmc-cs-sm-tests" DOCKER_DEFAULT_PLATFORM=linux/amd64 GITLAB_LICENSE_MODE=test QA_EE_ACTIVATION_CODE= GITLAB_QA_ACCESS_TOKEN= bundle exec gitlab-qa Test::Integration::AiGateway EE
To verify on staging:
GITLAB_QA_USER_AGENT=x GITLAB_USERNAME=gitlab-qa GITLAB_PASSWORD=x GITLAB_QA_ACCESS_TOKEN=x QA_GITLAB_URL=https://staging.gitlab.com bundle exec rspec qa/specs/features/ee/api/3_create/code_suggestions_spec.rb --tag external_ai_provider
Edited by Jay McCure