Skip to content

Add multi-version upgrade test

Dan Davison requested to merge dj-1919-add-bulk_data-seed into master

What does this MR do and why?

  1. Add bulk data seed file for GitLab Data Seeder to use.
    • This data populates the GitLab instance with random data (Projects, Merge Requests, Issues, Group Labels, Project Labels, Milestones, Epics, etc.)
  2. Add a logger to the Data Seeder
  3. Add db:migrate:multi-version-upgrade-1 and db:migrate:multi-version-upgrade-2 jobs

Note: UPGRADE_STOPs version <15.5 will not work. To seed the data, the GitLab Data Seeder is used in favor of the GitLab Rest API for a couple reasons:

  1. Speed
  2. No need to have to wait for the app to be fully up-and-running

The GitLab Data Seeder was implemented in 15.5, thus any versions previous to this will not work as the Seeder will not exist.


Multi-Version Upgrade Test Steps

db:migrate:multi-version-upgrade-1

  1. Pull the UPGRADE_STOP GitLab docker container from Docker Hub. (currently set to 16.3.6-ee)
  2. Configure the container using this snippet (specifically this shell script)
  3. Seed the UPGRADE_STOP container's database with randomized bulk data
  4. Dump the database and expose a gitlabhq_production.gz artifact of the dump
  5. Trigger db:migrate:multi-version-upgrade-2

db:migrate:multi-version-upgrade-2

  1. Prepare test environment
  2. Install postgresql client
  3. Decompress gitlabhq_production.gz artifact and import it into the gitlabhq_test database
  4. Reconfigure GitLab Database
  5. Pass/Fail (Pass if the Current MR is compatible with the UPGRADE_STOP version, Fail if not)

Examples

Passing Upgrade (16.3.6 => 16.8)

db:migrate:multi-version-upgrade-1 db:migrate:multi-version-upgrade-2
https://gitlab.com/gitlab-org/gitlab/-/jobs/5895342661 https://gitlab.com/gitlab-org/gitlab/-/jobs/5895342662

Failing Upgrade (16.1.5 => 16.8)

db:migrate:multi-version-upgrade-1 db:migrate:multi-version-upgrade-2
https://gitlab.com/gitlab-org/gitlab/-/jobs/5895521723 https://gitlab.com/gitlab-org/gitlab/-/jobs/5895521727

Failing Upgrade (15.6.8 => 16.8)

db:migrate:multi-version-upgrade-1 db:migrate:multi-version-upgrade-2
https://gitlab.com/gitlab-org/gitlab/-/jobs/5896371496 https://gitlab.com/gitlab-org/gitlab/-/jobs/5896371498

Screenshots or screen recordings

Screenshots are required for UI changes, and strongly recommended for all other merge requests.

2023-11-21_14-47-11

2024-01-10_10-46-19

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

$ GITLAB_LOG_LEVEL=debug bundle exec rake "ee:gitlab:seed:data_seeder[bulk_data.rb,1]"

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Dan Davison

Merge request reports