Skip to content

Fix web ide specs: ensure that element check responses checked

John McDonnell requested to merge jmd/fix-e2e-close-ide into master

What does this MR do and why?

I noted that the check for ide.has_file?(file_name) is silently failing at the moment as it doesn't include the within_vscode_editor The functionality is otherwise working as expected, so we don't observe any obvious failure/error other than adding a 10sec delay (default timeout) to the test execution time.

As this is a necessary precondition for this spec, lets instead just add an expectation to the check so it would fail if something did go wrong, and fix the method to ensure we use within_vscode_editor for this selector.

This is turn led to me to note a number of other cases within these specs with similar issues so I've tried to group them all together into this MR. By removing these silent failures, we should be able to speed up the test execution times. From local test runs it appears to improve the execution time of the set of 8 specs in browser_ui/3_create/web_ide by approx 2mins total.

WEBDRIVER_HEADLESS=true bundle exec rspec qa/specs/features/browser_ui/3_create/web_ide
Before results 
Top 8 slowest examples (364.17 seconds, 99.6% of total time):
  Create Upload a file in Web IDE when the file is a text file behaves like upload a file verifies it successfully uploads and commits to a MR
    68.75 seconds ./qa/specs/features/browser_ui/3_create/web_ide/upload_new_file_in_web_ide_spec.rb:30
  Create Upload a file in Web IDE when the file is an image behaves like upload a file verifies it successfully uploads and commits to a MR
    68.6 seconds ./qa/specs/features/browser_ui/3_create/web_ide/upload_new_file_in_web_ide_spec.rb:30
  Create Add a directory in Web IDE when user adds a new empty directory shows successfully but not able to be committed
    54.74 seconds ./qa/specs/features/browser_ui/3_create/web_ide/add_new_directory_in_web_ide_spec.rb:43
  Create Add first file in Web IDE when user adds a new file shows successfully added and visible in project
    47.26 seconds ./qa/specs/features/browser_ui/3_create/web_ide/add_first_file_in_web_ide_spec.rb:31
  Create Upload a file in Web IDE when a file with the same name already exists throws an error
    43.24 seconds ./qa/specs/features/browser_ui/3_create/web_ide/upload_new_file_in_web_ide_spec.rb:19
  Create Closing Web IDE shows an alert when there are unsaved changes
    33.42 seconds ./qa/specs/features/browser_ui/3_create/web_ide/closing_web_ide_with_unsaved_changes_spec.rb:16
  Create Add a directory in Web IDE when a directory with the same name already exists throws an error
    26.31 seconds ./qa/specs/features/browser_ui/3_create/web_ide/add_new_directory_in_web_ide_spec.rb:26
  Create Add first file in Web IDE when a file with the same name already exists throws an error
    21.85 seconds ./qa/specs/features/browser_ui/3_create/web_ide/add_first_file_in_web_ide_spec.rb:18

Finished in 6 minutes 5 seconds (files took 3.23 seconds to load)
After Results 
Top 8 slowest examples (225.55 seconds, 99.4% of total time):
  Create Add a directory in Web IDE when user adds a new empty directory shows successfully but not able to be committed
    43.72 seconds ./qa/specs/features/browser_ui/3_create/web_ide/add_new_directory_in_web_ide_spec.rb:43
  Create Upload a file in Web IDE when the file is a text file behaves like upload a file verifies it successfully uploads and commits to a MR
    30.36 seconds ./qa/specs/features/browser_ui/3_create/web_ide/upload_new_file_in_web_ide_spec.rb:31
  Create Upload a file in Web IDE when the file is an image behaves like upload a file verifies it successfully uploads and commits to a MR
    29.56 seconds ./qa/specs/features/browser_ui/3_create/web_ide/upload_new_file_in_web_ide_spec.rb:31
  Create Add first file in Web IDE when user adds a new file shows successfully added and visible in project
    27.81 seconds ./qa/specs/features/browser_ui/3_create/web_ide/add_first_file_in_web_ide_spec.rb:31
  Create Add a directory in Web IDE when a directory with the same name already exists throws an error
    26.57 seconds ./qa/specs/features/browser_ui/3_create/web_ide/add_new_directory_in_web_ide_spec.rb:26
  Create Upload a file in Web IDE when a file with the same name already exists throws an error
    24.81 seconds ./qa/specs/features/browser_ui/3_create/web_ide/upload_new_file_in_web_ide_spec.rb:19
  Create Closing Web IDE shows an alert when there are unsaved changes
    21.53 seconds ./qa/specs/features/browser_ui/3_create/web_ide/closing_web_ide_with_unsaved_changes_spec.rb:16
  Create Add first file in Web IDE when a file with the same name already exists throws an error
    21.2 seconds ./qa/specs/features/browser_ui/3_create/web_ide/add_first_file_in_web_ide_spec.rb:18

Finished in 3 minutes 46.9 seconds (files took 3.38 seconds to load)

As this is a reasonably substantial change to the overall flow the tests, I've decided to drop the :blocking tag here, as there's a risk that this change could introduce some flakiness that was masked by the longer waits.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

  1. WEBDRIVER_HEADLESS=true bundle exec rspec qa/specs/features/browser_ui/3_create/web_ide
Edited by John McDonnell

Merge request reports