SPIKE: Review Verify E2E tests to evaluate which tests we can keep and which ones can be merged with other tests.
Review the Verify E2E tests to determine which tests can be removed in favour of lower level tests. Once we have determined which tests can be removed, we will need to see if there are already existing lower level tests that can give us the confidence to remove these E2E tests.
After that, for the remaining E2E tests, some of these tests overlap with other tests where the steps performed might be identical or can be verified at the same time. We'll need to identify such tests and merge them if possible.
Tasks
-
Finalized list of E2E tests that can be removed in favour of lower-level tests. -
Dig into the lower-level tests to find tests that can replace the E2E test. If none exists or coverage is not enough, create issues to shore up test coverage. (This will be done on the individual spec removal issue) -
Review remaining E2E tests on which ones can be merged together. (#1652 (closed)) -
Create issues to merge such tests.
Current Status
✅ Phase 1: Identify specs
We've identified 4 specs we agreed should be removed.
add_remove_ci_variable_spec.rb
create_and_process_pipeline_spec.rb
mr_event_rule_pipeline_spec.rb
register_runner_spec.rb
There are an additional 7 issues that we need to understand better if it can be removed or not.
locked_artifacts_spec.rb
pipeline_editor_branch_switcher_spec.rb
pipeline_editor_tabs_spec.rb
view_code_coverage_spec.rb
new_discussion_not_dropping_merge_trains_mr_spec.rb
system_cancel_merge_request_in_merge_train_spec.rb
user_cancel_merge_request_in_merge_train_spec.rb
For these ones, we'll need to reach out to our counterparts to understand the system better to be able to make a decision.
👉 Phase 2: Potentially remove, move to lower levels
-
add_remove_ci_variable_spec.rb
– Remove `add_remove_ci_variable_spec.rb` (#1647 - closed) -
create_and_process_pipeline_spec.rb
– Remove `create_and_process_pipeline_spec.rb` (#1648 - closed) -
mr_event_rule_pipeline_spec.rb
– Remove `mr_event_rule_pipeline_spec.rb` (#1649 - closed) -
register_runner_spec.rb
– Update `register_runner_spec.rb` tag to `runner... (#1650)