Eliminate UI during API only test execution

Summary

We should decouple UI workflows from API only testing.

The UI is currently being leveraged to perform some tasks before testing starts even when that testing is meant to be API only.

When starting tests where the environment variable EE_LICENSE is set, then the UI is launched in order to ensure that the contents of that variable are inserted into the license field by this class https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/ee/strategy.rb.

Also when the environment variable GITLAB_QA_ACCESS_TOKEN is not set properly then the UI launches and creates an API token for use via this method https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/runtime/api/client.rb#L65

However, when running API only tests, these steps are mixing in UI work-flows into an API only test. This does two things:

  • slows down the API only test
  • prevents us from running an API only test.

We want to be able to run an API only test for speed reasons, and also to test that the code paths between the API and UI return the same result. If we can't exercise some part of our product by any method except by the UI then we cannot ensure this. Further we are preventing automation via the API which is beneficial to us in the Quality department for reasons stated above (faster tests) and beneficial to the wider community as API automation is more consistently reliable than UI automation.

Improvements

Require a GITLAB_QA_ACCESS_TOKEN via https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/runtime/env.rb#L268 before running API only tests. And if the token is present skip the UI launch entirely

Require an EE license by checking for one via the API or by adding the environment variable EE_LICENSE to the instance of GitLab via the API and circumvent the UI.

Risks

Involved components

qa/qa/ee/strategy.rb qa/qa/runtime/api/client.rb:65 create_personal_access_token method

Optional: Intended side effects

UI will not launch (even when running CHROME_HEADLESS=false) when running tests in the qa/qa/specs/features/api or qa/qa/specs/features/ee/api directories.

Optional: Missing test coverage