Consistently clear localStorage before examples

What does this MR do and why?

Note

I wrote all these words myself with my own two bunny pawiwos 🐰 🐾 Please read them, they aren't slop!

Our test suite currently wipes localStorage between examples to stop e.g. work item drafts from carrying over into other examples and corrupting their initial state.

https://gitlab.com/gitlab-org/quality/test-failure-issues/-/work_items/44561 highlights some examples where flakes occur showing evidence that this isn't happening; see https://gitlab.com/gitlab-org/quality/test-failure-issues/-/work_items/44561#note_3839873436 in particular.

The problem is in the definition of "between examples"; it's currently done with execute_script("localStorage.clear();") in an after block. If any localStorage is written after this, before the next example, it'll pollute the next example run.

The content editor is susceptible to exactly this: it doesn't save drafts instantly on keypress; it debounces saves 100ms after the last edit. "Save" involves serialising the document, giving it to Apollo, and then writing to localStorage.

An unlucky example run that ends 30ms after the last edit can therefore clear localStorage in the after block, and then have the content editor commit its draft to localStorage.

Curiously enough, Capybara also has its own localStorage clear, and it uses the exact same mechanism; see capybara-3.40.0/lib/capybara/selenium/driver.rb's clear_local_storage, also called in an after hook. It's enabled by default, so our redundant version was probably added without noticing it, just seeing that sometimes localStorage wasn't cleared. Because it's using the same mechanism, our copy maybe reduced the window slightly, but didn't eliminate it.

Because localStorage is origin-specific, any example that navigated away from the test domain by the end of the example would fail to clear storage, which is another big no-no.

Accordingly, this commit removes the redundant mechanism, and adds a reliable clear before each example runs, using Chrome DevTools through CDP to clear data for the target, before the target is loaded. This means we clean up at a moment when the application is definitely not running, and therefore can't race.

It also rewords the rationale comment in a hack in one example which half-related to what we're fixing here; that hack's still needed for its own reasons.

Root cause identified with data and machinery via Remove all uses of `wait_for_requests`, `wait_f... (&22879).

References

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Asherah Connor

Merge request reports

Loading
Loading