Skip to content

E2E: Add Retry for Purchase Flow Errors

Valerie Burton requested to merge e2e-add-retry-for-purchase-flow-errors into master

What does this MR do and why?

This MR is a temporary workaround for the HTTP status code: 500 and Request failed with status code 404 errors that have been occurring on the GitLab checkout page in Staging due to https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/8369. The root cause for these errors is due to a race condition, so this workaround will attempt to retry the purchase when one of these error messages occurs.

We should avoid retrying actions for vague error messages like these as much as possible, as this runs the risk of masking other bugs that may surface the same error message.

However, this bug has been open for several months now and is causing a substantial amount of noise in the pipelines. This places extra work on our on-call DRIs and decreases confidence in our tests due to the frequent flaky failures. By letting this bug continue to affect our tests, we also risk ignoring or missing new errors that need investigation.

Related test failure issues (non-exhaustive):

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

How to set up and validate locally

Important: The tests affected by these changes do not run in MR pipelines and can only run against Staging

To validate these changes, you will need to run the tests from your local machine and target Staging:

  1. Export the following ENV vars:
export GITLAB_USERNAME=gitlab-qa
export GITLAB_PASSWORD=<Found in GitLab-QA 1Password>
export GITLAB_QA_USER_AGENT=<Found in GitLab-QA 1Password>
export GITLAB_ADMIN_USERNAME=gitlab-qa-bot
export GITLAB_ADMIN_PASSWORD=<Found in GitLab-QA 1Password>
export GITLAB_QA_ACCESS_TOKEN=<Found in GitLab-QA 1Password>
export GITLAB_QA_ADMIN_ACCESS_TOKEN=<Found in GitLab-QA 1Password>
export QA_GITLAB_URL=https://staging.gitlab.com
  1. Then, run one of the tests from the qa directory, ex:
bundle exec rspec qa/specs/features/ee/browser_ui/11_fulfillment/purchase/purchase_ci_spec.rb
Edited by Valerie Burton

Merge request reports