Skip to content

[E2E] Add ENV to allow setting ZUORA_TIMEOUT on a per env basis

John McDonnell requested to merge jmd/increase-zuora-timeout into master

What does this MR do and why?

We've occasionally found cases where the non-prod Zuora account can be slower which has an impact on our E2E tests causing failures which make excessive noise in the pipelines.
In order to avoid such noise, let's allow for an ENV that can be quickly set in the target test environment pipeline with the accepted caveat that we know it will make the job slower, as I believe that in these type of situations a slower test is preferable to a failure.

The benefit of the ENV is that it can be set/unset quickly in the target environment, rather than needing a code change to roll out either a quarantine/longer timeout if these specs that are dependant on Zuora have external problems.

Sample failure: #425033 (comment 1558660799)

How to set up and validate locally

# With an ENV setting
$ QA_GITLAB_URL="https://staging.gitlab.com" GITLAB_QA_USER_AGENT="TOKEN" GITLAB_QA_ADMIN_ACCESS_TOKEN="TOKEN" QA_ZUORA_TIMEOUT=600 bundle exec rspec ./qa/specs/features/ee/browser_ui/11_fulfillment/purchase/upgrade_group_spec.rb:33

# Reverting to default value
$ QA_GITLAB_URL="https://staging.gitlab.com" GITLAB_QA_USER_AGENT="TOKEN" GITLAB_QA_ADMIN_ACCESS_TOKEN="TOKEN" bundle exec rspec ./qa/specs/features/ee/browser_ui/11_fulfillment/purchase/upgrade_group_spec.rb:33

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by John McDonnell

Merge request reports