Skip to content

Fixing flaky invite_flow spec

Ross Byrne requested to merge 419423-fixing-flaky-invite-flow-spec into master

What does this MR do and why?

Resolves #419423 (closed) and #423547 (closed)

This MR fixes flaky specs.

Based on the investigation carried out here, the root of the issue is API requests happening out of order.

I believe this is the same problem that has recently been fixed in !130752 (merged).

From gitlab-org/quality/engineering-productivity/master-broken-incidents#3344 (comment 1518944638), the order of the requests are:

  1. post to /users (the actual sign up click), time: 2023-08-18T14:05:35.047Z
  2. get to /users/user3/exists, time: 2023-08-18T14:05:35.099Z

The issue is the username validation call to /users/user3/exists should happen first, before the form is submitted. This is happening due to the debounce in the username validation js code.

The fix here is to expect the username field to show validation success before clicking the submit button.

Screenshots or screen recordings

No visible change

How to set up and validate locally

This is a very difficult timing issue to recreate locally, but one way to observe the problem is:

  1. Checkout master branch
  2. Run export WEBDRIVER='selenium_chrome' in terminal, to see feature spec execution
  3. Run bundle exec rspec ./ee/spec/features/registrations/saas/invite_flow_spec.rb
  4. See that the signup form is submitted before the username is validated

Then to see this fix:

  1. Checkout this MR branch or apply the git diff
  2. Run export WEBDRIVER='selenium_chrome' in terminal, to see feature spec execution
  3. Run bundle exec rspec ./ee/spec/features/registrations/saas/invite_flow_spec.rb
  4. See that the test waits for username validation before the signup form is submitted

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #419423 (closed)

Edited by Ross Byrne

Merge request reports