Defining a criteria on when to write E2E tests

Problem

As part of the upcoming changes where the responsibility to write and maintain the E2E tests are shifting to each respective groups, we should look into having a more definitive criteria on when an end-to-end test should be created. We want to make it easier for groups to create and maintain E2E tests and having this clearer criteria can help make end-to-end tests feel less intimidating to pick up.

At the moment, we don't seem to have any guidelines and it's traditionally left up to the SET to decide what E2E test to write. The most I found is when not to write an end-to-end test. Without a clear guideline, we may start running into cases where groups may not know what E2E test cases to write and still have to rely on the remaining SETs in teamTest Engineering to define that or where the E2E test is left as an afterthought or just completely forgotten. This won't be sustainable when there are now only 1 to 3 SETs left to support entire sections. We wouldn't be able to keep up with every single initiative..

Proposal

Given that the end-to-end tests are tests where we want to exercise the live systems from start to finish ensuring that the various pieces are working together properly, we should tie these tests closely with what would be considered key workflows that are important for the business. This could be workflows that are considered core functionality that must work no matter what or could be critical workflows that help drive conversion. We can rely on PMs or maybe some data metrics to determine which workflows are critical.

By doing this, it becomes clear what should be an end-to-end test and, since the test cases have an associated business use case, it helps everyone understand the value of the test cases better. We can also then regularly revisit and review existing end-to-end tests and determine if it is still useful to keep as an end-to-end test given that the importance of that workflow could have changed.