Add upgrade test scenario from patch to the latest stable

Nailia Iskhakovarequested to merge
upgrade-testing-next-version-coverage into master

What does this MR do and why?

This merge request introduces a new upgrade scenario UpdateToNext. It allows testing upgrades from a development version to the next available stable release to address the upgrade path gap unreleased patch N-1 -> latest stable N. This scenario will only run in backport MRs for stable branches to perform automated testing from current dev build (unreleased patch) to latest available next release. Needed for internal releases initiative https://gitlab.com/gitlab-com/gl-infra/software-delivery/framework/software-delivery-framework-issue-tracker/-/issues/28

The changes include:

  1. Adding documentation for the new upgrade scenario.
  2. Implementing the "UpdateToNext" class, which extends the existing "UpdateFromPrevious" functionality.
  3. Modifying the upgrade path logic to support finding the next available version.
  4. Updating the GitLab version info class to include a method for finding the next version.
  5. Adding new test cases to cover the new functionality.

How to set up and validate locally

Running actual upgrades with UpdateFromPrevious can be omitted as most important part is upgrade path. We need to verify that upgrade paths are built correctly before actual upgrade. Steps for testing:

  1. Add a breakpoint in lib/gitlab/qa/scenario/test/omnibus/update_from_previous.rb#L45 
    require 'pry'; binding.pry

update(rspec_args)
  2. Run commands for testing - 17.10.0 is not released yet, can't upgrade 17.9.0 -> 17.10 script will be no op: 
     bundle exec gitlab-qa Test::Omnibus::UpdateToNext registry.gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/gitlab-ee:9361d9b72da9aa366fde4f0f87a30fd747b5d57d-ruby3.2.5 17.9.0  -- --tag health_check

...
[Feb 20 2025 18:06:25 CET (Gitlab QA)] INFO  -- Skipping upgrade test as next version after 17.9.0 is not yet available
  3. Run commands for testing - 17.10.0 is not released yet, can't upgrade 17.10 -> 17.11, script will be no op: 
     bundle exec gitlab-qa Test::Omnibus::UpdateToNext registry.gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/gitlab-ee:9361d9b72da9aa366fde4f0f87a30fd747b5d57d-ruby3.2.5 17.10.0-pre  -- --tag health_check

...
[Feb 20 2025 18:08:46 CET (Gitlab QA)] INFO  -- Skipping upgrade test as version 17.10.0 is not yet released
  4. Run commands for testing - 17.8.2 (built from backport MR branch) -> 17.9.0: 
    bundle exec gitlab-qa Test::Omnibus::UpdateToNext registry.gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/gitlab-ee:9361d9b72da9aa366fde4f0f87a30fd747b5d57d-ruby3.2.5 17.8.2-pre  -- --tag health_check 

...
[Feb 20 2025 18:07:18 CET (Gitlab QA)] INFO  -- Performing gitlab update: (17.8.2-pre) registry.gitlab.com/gitlab-org/build/omnibus-gitlab-mirror/gitlab-ee:9361d9b72da9aa366fde4f0f87a30fd747b5d57d-ruby3.2.5 => gitlab/gitlab-ee:17.9.0-ee.0

Additional note: validated full run on a VM by running CHROME_DISABLE_DEV_SHM=true bundle exec gitlab-qa Test::Omnibus::UpdateToNext gitlab/gitlab-ee:17.6.5-ee.0 17.6.5-ee -- --tag health_check - worked as expected ✅

Click to expand 
[Feb 20 2025 12:37:55 UTC (Gitlab QA)] INFO  -- Fetching Docker tags page 1 from 'gitlab/gitlab-ee' registry
[Feb 20 2025 12:37:55 UTC (Gitlab QA)] INFO  -- Fetching Docker tags page 2 from 'gitlab/gitlab-ee' registry
[Feb 20 2025 12:37:56 UTC (Gitlab QA)] INFO  -- Fetching Docker tags page 3 from 'gitlab/gitlab-ee' registry
[Feb 20 2025 12:37:56 UTC (Gitlab QA)] INFO  -- Fetching Docker tags page 4 from 'gitlab/gitlab-ee' registry
[Feb 20 2025 12:37:56 UTC (Gitlab QA)] INFO  -- Fetching Docker tags page 5 from 'gitlab/gitlab-ee' registry
[Feb 20 2025 12:37:56 UTC (Gitlab QA)] INFO  -- Fetching Docker tags page 6 from 'gitlab/gitlab-ee' registry
[Feb 20 2025 12:37:57 UTC (Gitlab QA)] INFO  -- Fetching Docker tags page 7 from 'gitlab/gitlab-ee' registry
[Feb 20 2025 12:37:57 UTC (Gitlab QA)] INFO  -- Fetching Docker tags page 8 from 'gitlab/gitlab-ee' registry
[Feb 20 2025 12:37:57 UTC (Gitlab QA)] INFO  -- Fetching Docker tags page 9 from 'gitlab/gitlab-ee' registry
[Feb 20 2025 12:37:57 UTC (Gitlab QA)] INFO  -- Fetching Docker tags page 10 from 'gitlab/gitlab-ee' registry
[Feb 20 2025 12:37:58 UTC (Gitlab QA)] INFO  -- Fetching Docker tags page 11 from 'gitlab/gitlab-ee' registry
[Feb 20 2025 12:37:58 UTC (Gitlab QA)] INFO  -- Fetching Docker tags page 12 from 'gitlab/gitlab-ee' registry
[Feb 20 2025 12:37:58 UTC (Gitlab QA)] INFO  -- Fetching Docker tags page 13 from 'gitlab/gitlab-ee' registry
[Feb 20 2025 12:37:58 UTC (Gitlab QA)] INFO  -- Performing gitlab update: (17.6.5-ee) gitlab/gitlab-ee:17.6.5-ee.0 => gitlab/gitlab-ee:17.7.4-ee.0
[Feb 20 2025 12:37:58 UTC (Gitlab QA)] INFO  -- Running the development release: gitlab/gitlab-ee:17.6.5-ee.0
...
[Feb 20 2025 12:46:39 UTC (Gitlab QA)] INFO  -- Upgrading GitLab to target release: gitlab/gitlab-ee:17.7.4-ee.0
[Feb 20 2025 12:46:39 UTC (Gitlab QA)] INFO  -- Deploying release: gitlab/gitlab-ee:17.7.4-ee.0
....
Finished in 2 minutes 26.7 seconds (files took 1.63 seconds to load)
10 examples, 0 failures, 2 pending

Randomized with seed 11846

[Feb 20 2025 12:55:50 UTC (Gitlab QA)] INFO  -- Shell command: `docker exec gitlab-updatefromprevious-c71e8be4 bash -c "gitlab-psql -c 'select n_live_tup, n_dead_tup, relname from pg_stat_all_tables order by n_live_tup DESC, n_dead_tup DESC;'"`
...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
[Feb 20 2025 12:55:50 UTC (Gitlab QA)] INFO  -- Shell command: `docker exec gitlab-updatefromprevious-c71e8be4 bash -c "gitlab-psql -c 'select * from pg_stat_user_tables;'"`
.......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
[Feb 20 2025 12:55:50 UTC (Gitlab QA)] INFO  -- Shell command: `docker ps -f name=gitlab-updatefromprevious-c71e8be4`
..
[Feb 20 2025 12:55:50 UTC (Gitlab QA)] INFO  -- Shell command: `docker rm -f gitlab-updatefromprevious-c71e8be4`

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Nailia Iskhakova

Merge request reports