Skip to content

E2E test: add code completion and code streaming e2e API tests

Jay McCure requested to merge jmc-api-cs-cc into master

What does this MR do and why?

Partially resolves #434110 (closed) and gitlab-org/quality/quality-engineering/team-tasks#2258 (closed)

This MR adds four new test cases and reafactors the test case for re-use:

  1. SaaS - code completion
  2. Self-managed - code completion (quarantined)
  3. SaaS - code generation streaming
  4. Self-managed - code generation streaming (quarantined)

The two existing tests were code generation tests, so they have been explicitly identified as such

For code completion we use a different prompt which should always return a code completion response. For streaming we pass in a streaming: true parameter and check that the header indicates the response is a stream.

I have tested this locally against staging and production. I have created test cases and linked them also.

How to set up and validate locally

This test only runs against live environments:

QA_LOG_LEVEL=debug GITLAB_QA_USER_AGENT=<1P> GITLAB_USERNAME=gitlab-qa GITLAB_PASSWORD=<1P> GITLAB_QA_ACCESS_TOKEN=<1P> QA_GITLAB_URL=https://staging.gitlab.com bundle exec rspec qa/specs/features/ee/api/14_model_ops/code_suggestions_spec.rb

Alternatively, can modify the host and token here: https://gitlab.com/gitlab-org/gitlab/-/blob/4b6d9936ac566a4ae7a8003b8130c823c5a90f0c/qa/qa/specs/features/ee/api/14_model_ops/code_suggestions_spec.rb#L177-180

Edited by Jay McCure

Merge request reports