Optimize omnibus QA MR pipelines by removing unwanted test and running only smoke tests

Purpose of this issue : Make the MR pipelines run faster so that the devs can merge MRs more quickly, by analysing all tests running in omnibus-gitlabproject and add/remove/update tests which are relevant to omnibus.

Currently the QA pipeline in omnibus-gitlab project pipelines runs

• Test specific to Omnibus installation
• Tests irrelevant to Omnibus - These might be app specific test. For e.g. cloud-activation test which checks if the license is not provide -> Provides license and validates it -> Deletes license and validates successful deletion.
• Entire test suite - depending on what configuration the specific test is verifying e.g.
  • relative-url test as seen in the pipeline https://gitlab.com/gitlab-org/omnibus-gitlab/-/pipelines/882322717 test runs Test::Instance::ALL scenarios as seen in relative url config
  • decomposition-single-db or instance runs Test::Instance::Image scenarios as seen in [test instance image config](https://gitlab.com/gitlab-org/gitlab-qa/-/blob/master/lib/gitlab/qa/scenario/test/instance/image.rb#L25

As a part of this issue, analysis was done and each tests that run in the omnibus-gitlab project MR were gone through.

The focus during analysis was to remove Tests irrelevant to Omnibus and run smoke tests instead of Entire Test suite

I have the following after my analysis and discussion with Nailia

FYI - Clicking on the links below will take you to the comment which explains a bit about what the test does.

We should keep

• decomposition-single-db
• gitaly-cluster
• gitlab-pages
• group-saml
• instance
• praefect
• instance-saml
• ldap-no-server
• ldap-no-tls
• ldap-tls
• mattermost
• metrics
• mtls
• object-storage
• object-storage-aws
• object-storage-gcp
• packages
• registry
• registry-with-cdn
• relative-url
• repository-storage
• smtp
• update-minor
• update-major

Tests to remove

The following tests do not test omnibus installation and hence can be removed from the pipeline

• decomposition-multiple-db ~~Do we remove this functionality as its not ready for use in production as per ~~~~https://docs.gitlab.com/ee/administration/postgresql/multiple_databases.html~~~~?~~ - _Keep this as per gitlab-org/quality/quality-engineering/team-tasks#1761 (comment 1441538393)_
• cloud-activation as per gitlab-org/quality/quality-engineering/team-tasks#1761 (comment 1441534686)

Tests to have manual trigger

The following tests would only be run in special scenarios and hence can be moved to a manual trigger option in the pipeline

• update-ee-to-ce - Since this is not a common scenario
• oauth - Was manual before

Unsure if this touches omnibus config

• elasticsearch this functionality isn't included in the omnibus package and needs to be installed seperately so do we need to keep this test https://docs.gitlab.com/ee/integration/advanced_search/elasticsearch.html#install-elasticsearch
• importers this test checks the import functionality from GitHub and uses mock requests to GitHub using smoker to replicate importing of project
• integration this test mainly sets up webhooks in projects and groups and checks if its working. Also uses mock requests for testing
• jira this test sets up Jira integration and tests out the integration?
• large-setup this test checks if the user is able to download Patch and Plain diff as files from MR by clicking on Code button
• service-ping this test checks if service ping gathers details for sales metrics

Additional tests that we can add

• Other supporter Object Storage
  • Currently we only test object storage for AWS, GCP and MinIO. Do we need tests for remaining object storage providers as mentioned https://docs.gitlab.com/ee/administration/object_storage.html#supported-object-storage-providers
• Test for upgrading from ce->ee. WE currently have update-ee-to-ce

Additional Improvements to the pipeline

• Initial Test Setup Time - Some tests have initial setup which it does and running such tests in parallel does that setup multiple time. We can consider to have less runners for those tests which are heavy on initial setup so that the multiple iterations of initial test setup time can be reduced
• Queuing time - Queuing time is also a factor. If there is any way to reduce the queuing time(May be by increasing gitlab runner) we can consider that. I personally don't know much about this, but will take the direction from the team on this.