Gather current test suites and execution for release/security/backport pipelines

The issue tracks the current test setup for release and backport pipelines

Current test suites for backport MR pipelines

For EE and FOSS

  • Frontend full jest specs
  • RSpec background migration for PG14
  • RSpec fast_Spec_helper
  • Clickhouse Rollback

For EE Only

  • Frontend jest quarantine check
  • GraphQL verification
  • RSpec EE background migration for PG14
  • RSpec EE system, integration, fail-fast

End-to-end suites

  • e2e:test-on-omnibus-ee full suite in tier 3
  • qa:test-on-gdk gdk-instance suite in tier 3

Notes

  • For backports with frontend only changes, we still run jobs like background migration. This results in a lot of waste.
Expand for details

Tier 1 (https://gitlab.com/gitlab-org/gitlab/-/pipelines/1808036954)

test-frontend stage

EE
  • jest
  • jest vue3 check quarantined
  • jest vue3 mr
  • jest-build-cache
  • jest-build-cache-vue3-ensure-compilable-sfcs
  • jest-integration
  • jest-with-fixtures
  • jest-with-fixtures vue3 check quarantined
  • jest-with-fixtures vue3 mr
as-if-foss
  • jest
  • jest vue3 mr
  • jest-build-cache
  • jest-build-cache-vue3-ensure-compilable-sfcs
  • jest-integration
  • jest-with-fixtures
  • jest-with-fixtures vue3 mr

test stage

EE
  • gitlab:clickhouse:rollback:main
  • gitlab_git_test
  • graphql-verify
  • rspec background_migration pg14
  • rspec fail-fast
  • rspec fast_spec_helper
  • rspec integration pg14
  • rspec rspec-ee-pg14-rerun-previous-failed-tests
  • rspec rspec-pg14-rerun-previous-failed-tests
  • rspec system pg14
  • rspec unit clickhouse
  • rspec unit pg14
  • rspec-ee background_migration pg14
  • rspec-ee fail-fast
  • rspec-ee integration pg14
  • rspec-ee system custom-models pg14
  • rspec-ee system pg14
  • rspec-ee unit clickhouse
  • rspec-ee unit pg14
  • run-dev-fixtures-ee
  • e2e:test-on-omnibus-ee
as-if-foss
  • gitlab:clickhouse:rollback:main
  • rspec background_migration pg14
  • rspec fast_spec_helper
  • rspec integration pg14
  • rspec system pg14
  • rspec unit clickhouse
  • rspec unit pg14

Tier 2 (N/A?)

Tier 3 (https://gitlab.com/gitlab-org/gitlab/-/pipelines/1808098290)

test-frontend stage

EE
  • jest
  • jest vue3 check quarantined
  • jest vue3 mr
  • jest-build-cache
  • jest-build-cache-vue3-ensure-compilable-sfcs
  • jest-integration
  • jest-with-fixtures
  • jest-with-fixtures vue3 check quarantined
  • jest-with-fixtures vue3 mr
as-if-foss
  • jest
  • jest vue3 mr
  • jest-build-cache
  • jest-build-cache-vue3-ensure-compilable-sfcs
  • jest-integration
  • jest-with-fixtures
  • jest-with-fixtures vue3 mr

test stage

EE
  • db:check-schema
  • gitlab:clickhouse:rollback:main
  • gitlab_git_test
  • graphql-verify
  • rspec background_migration pg14
  • rspec fail-fast
  • rspec fast_spec_helper
  • rspec integration pg14
  • rspec rspec-ee-pg14-rerun-previous-failed-tests
  • rspec rspec-pg14-rerun-previous-failed-tests
  • rspec system pg14
  • rspec unit clickhouse
  • rspec unit pg14
  • rspec-ee background_migration pg14
  • rspec-ee fail-fast
  • rspec-ee integration pg14
  • rspec-ee system custom-models pg14
  • rspec-ee system pg14
  • rspec-ee unit clickhouse
  • rspec-ee unit pg14
  • run-dev-fixtures-ee
as-if-foss
  • gitlab:clickhouse:rollback:main
  • rspec background_migration pg14
  • rspec fast_spec_helper
  • rspec integration pg14
  • rspec system pg14
  • rspec unit clickhouse
  • rspec unit pg14

qa stage

  • qa:test-on-gdk (only gdk-instance)
  • e2e:test-on-omnibus-ee (full suite)

Current test suites for stable branches

(post-merge after backport MRs are merged)

  • Frontend jest, vue2 jest, and vue3 jest (🔍)
  • Git test
  • GraphQL verification
  • RSpec background migration for pg14
  • Rspec unit, system, integration for pg14
  • RSpec unit for clickhouse
  • RSpec EE background migration for pg14
  • RSpec EE unit, system, integration for pg14
  • RSpec EE unit for clickhouse
Expand for details

Stable Branches (e.g. 17-9-stable-ee)

(taken from https://gitlab.com/gitlab-org/gitlab/-/pipelines/1808208666)

frontend stage

  • jest
  • jest vue3 mr
  • jest-build-cache
  • jest-build-cache-vue3-ensure-compilable-sfcs
  • jest-integration
  • jest-with-fixtures
  • jest-with-fixtures vue3 mr
  • webpack-dev-server

test stage

  • gitlab:clickhouse:rollback:main
  • gitlab_git_test
  • graphql-verify
  • rspec background_migration pg14
  • rspec integration pg14
  • rspec system pg14
  • rspec unit clickhouse
  • rspec unit pg14
  • rspec-ee background_migration pg14
  • rspec-ee integration pg14
  • rspec-ee system custom-models pg14
  • rspec-ee system pg14
  • rspec-ee unit clickhouse
  • rspec-ee unit pg14
  • run-dev-fixtures-ee

Current test suites for release pipelines

  • End-to-end smoke tests after release

Current test suites for security backport MR pipelines

test-frontend stage

  • jest
  • jest vue3 mr
  • jest-build-cache
  • jest-build-cache-vue3-ensure-compilable-sfcs
  • jest-integration (tier 3 only)
  • jest-with-fixtures
  • jest-with-fixtures vue3 mr
  • webpack-dev-server

test stage

  • db:check-schema (tier 3 only)
  • db:docs-up-to-date
  • gitlab:clickhouse:rollback:main
  • gitlab_git_test
  • graphql-verify
  • memory-on-boot
  • rspec background_migration pg16
  • rspec fail-fast
  • rspec fast_spec_helper
  • rspec integration pg16
  • rspec migration pg16
  • rspec system pg16
  • rspec unit clickhouse
  • rspec unit pg16
  • rspec-ee background_migration pg16
  • rspec-ee fail-fast
  • rspec-ee integration pg16
  • rspec-ee migration pg16
  • rspec-ee unit clickhouse
  • rspec-ee unit pg16
  • run-dev-fixtures-ee

qa stage

  • e2e:test-on-omnibus-ee (investigate removal for tier 1)
  • e2e:test-on-gdk (tier 3 only)

Current test suites for security stable branch pipelines

test-frontend stage

  • jest
  • jest vue3 mr
  • jest-integration
  • jest-with-fixtures
  • jest-with-fixtures vue3 mr
  • webpack-dev-server

test stage

  • db:check-schema
  • db:docs-up-to-date
  • gitlab:clickhouse:rollback:main
  • gitlab_git_test
  • graphql-verify
  • memory-on-boot
  • rspec background_migration pg16
  • rspec fail-fast
  • rspec integration pg16
  • rspec system pg16
  • rspec unit clickhouse
  • rspec unit pg16
  • rspec-ee background_migration pg16
  • rspec-ee fail-fast
  • rspec-ee integration pg16
  • rspec-ee system pg16
  • rspec-ee unit clickhouse
  • rspec-ee unit pg16
  • run-dev-fixtures-ee

qa stage

  • e2e:test-on-gdk
  • e2e:test-on-omnibus-ee
Edited by Dan Davison