Skip to content

Add Upgrade AI Scenario

John McDonnell requested to merge jmd/add-ai-upgrade-scenario into master

What does this MR do and why?

Adds ability to perform upgrade tests with a specific focus on AI related functionality.

This should allow for us to run upgrades between major/minor versions replicating the upgrade process that our self managed customers may follow, and ensure that any AI related functionality remains operational.

Implements Add gitlab-qa scenario to test upgrades of AI c... (#730)

Example Usage

Sample Results:

  • Verification of GitLab Duo availability for self managed upgrades immediately after upgrade complete
  • Connection to a locally hosted AI Gateway, using mocked responses.
  • NB, for this run manual debugging was used to verify environmental state as WIP, but automated tests with relevant tags will be used in completed version
# setup configuration to point at various interconnected components 
# export AI_GATEWAY_URL=https://cloud.staging.gitlab.com/ai
export AI_GATEWAY_URL=http://gdk.test:5052
export CUSTOMER_PORTAL_URL=https://customers.staging.gitlab.com

$ bundle exec ./exe/gitlab-qa Test::Omnibus::UpdateFromPreviousAi gitlab/gitlab-ee:17.1.0-ee.0 17.1.0 minor ee -- --tag duo_chat --tag ai_gateway
Deploy          GitLab Enterprise Edition v17.0.8-ee    --> GitLab Duo Available
Upgrade         GitLab Enterprise Edition v17.1.0-ee    --> Missing Duo, seems broken even after sync  
# this would have raised test failure and investigation by the relevant dev/test teams  

$ bundle exec ./exe/gitlab-qa Test::Omnibus::UpdateFromPreviousAi gitlab/gitlab-ee:17.4.0-ee.0 17.4.0 minor ee -- --tag duo_chat --tag ai_gateway
Deploy          GitLab Enterprise Edition v17.3.x-ee    --> GitLab Duo Available
Upgrade         GitLab Enterprise Edition v17.4.0-ee    --> GitLab Duo Available
# Successful scenario -- PASS

How to set up and validate locally

The scenario is based heavily of the pre-existing UpdateFromPrevious scenario currently used for upgrade testing so usage is consistent between scenarios.

e.g.

$ bundle exec ./exe/gitlab-qa Test::Omnibus::UpdateFromPreviousAi gitlab/gitlab-ee:17.1.0-ee.0 17.1.0 minor ee -- --tag duo_chat --tag ai_gateway  # (17.0.x -> 17.1.0) 17.0.8
$ bundle exec ./exe/gitlab-qa Test::Omnibus::UpdateFromPreviousAi gitlab/gitlab-ee:17.2.0-ee.0 17.2.0 minor ee -- --tag duo_chat --tag ai_gateway  # (17.1.x -> 17.2.0) 17.0.x
$ bundle exec ./exe/gitlab-qa Test::Omnibus::UpdateFromPreviousAi gitlab/gitlab-ee:17.3.0-ee.0 17.3.0 minor ee -- --tag duo_chat --tag ai_gateway  # (17.2.x -> 17.3.0) 17.0.x
$ bundle exec ./exe/gitlab-qa Test::Omnibus::UpdateFromPreviousAi gitlab/gitlab-ee:17.4.0-ee.0 17.4.0 minor ee -- --tag duo_chat --tag ai_gateway  # (17.3.x -> 17.4.0) 17.0.x

$ bundle exec ./exe/gitlab-qa Test::Omnibus::UpdateFromPreviousAi gitlab/gitlab-ee:17.4.0-ee.0 17.4.0 major ee -- --tag duo_chat --tag ai_gateway # Performing gitlab update: gitlab/gitlab-ee:16.11.10-ee.0 => gitlab/gitlab-ee:17.3.5-ee.0 => gitlab/gitlab-ee:17.4.0-ee.0 (17.4.0)

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by John McDonnell

Merge request reports