Skip to content

Integrate orchestrated e2e tests in to gitlab project pipeline

Andrejs Cunskis requested to merge acunskis-native-package-and-qa into master

What does this MR do and why?

Describe in detail what your merge request does and why.

Implements package-and-qa as a child pipeline: https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/1334+

Pros:

  • Moves test execution to where the code is
  • Easier to see the test results, integrates test results in to merge request widgets once #363302 is done
  • Cancelling redundant child pipeline should work correctly once cascading cancel is implemented
  • Should allow to simplify rules and pipeline setup in gitlab-qa project since it has to handle pipelines for several upstream triggers
  • Static yml definition makes it easier to read which jobs are included in the pipeline and simplifies maintenance
  • Improves feature toggle state change testing, omnibus image is only built once
  • Adds update-minor and update-major test jobs

Cons:

  • gitlab-qa now servers as a single point for running all e2e orchestrated tests, now this wouldn't be the case
  • Making reusable templates and extracting to pipeline-common might be difficult and might increase complexity and maintenance

Some notable changes compared to gitlab-qa project pipeline implementation:

  • Replaces somewhat ambiguous term package-and-qa with package-and-test and jobs are prefixed with e2e to make it clear what it does
  • Removes very verbose duplication of quarantine type jobs for every single job type in favour of one single parametrized manual quarantine job
  • More reuse of templates in pipeline-common project
  • Selective test execution is done via pipeline rules rather than dynamically generating whole pipeline yml
  • Moves all helpers in to the qa folder for better concern separation
  • Introduces new type of upgrade jobs which test upgrade from previous minor and major gitlab versions
  • Properly locks gitlab-qa version via Gemfile.lock

Tasks:

  • Add ci runner for e2e test execution
  • Add all of the missing job types
  • Set up all necessary environment variables
  • Set up issue creation/linking scripts
  • Set up knapsack report regeneration

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Test Runs

Issues

Closes: package-and-qa job does not auto run based on M... (#366871 - closed)
Closes: https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/1334+
Closes: https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/1415+
Closes: Create a new orchestrated scenario to upgrade f... (gitlab-qa#672 - closed)
Closes: https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/1417+

Edited by Andrejs Cunskis

Merge request reports