Evaluate new environment for frontend tests
We are currently using Karma in combination with Jasmine to run our frontend tests. This has a few downsides, namely:
- we are relying on the now unmaintained jasmine-jquery (https://gitlab.com/gitlab-org/gitlab-ce/issues/24068)
- tests are all running in the same context and not sandboxed (https://gitlab.com/gitlab-org/gitlab-ce/issues/30322, https://gitlab.com/gitlab-org/gitlab-ce/issues/24100, https://gitlab.com/gitlab-org/gitlab-ce/issues/44238)
- we have no snapshots (https://gitlab.com/gitlab-org/gitlab-ce/issues/32524)
- there are no built-in async assertions / Promise based tests (https://gitlab.com/gitlab-org/gitlab-ce/issues/43323)
- Jasmine is not even listed as an option in the Vue testing guidelines: https://vue-test-utils.vuejs.org/guides/#choosing-a-test-runner
Alternatives to look at
I suggest we limit this evaluation to the tools that are described in the Vue testing guidelines:
- Jest: https://vue-test-utils.vuejs.org/guides/#testing-single-file-components-with-jest
- Mocha: https://vue-test-utils.vuejs.org/guides/#testing-single-file-components-with-mocha-webpack
- Karma + Mocha: https://vue-test-utils.vuejs.org/guides/#testing-single-file-components-with-karma
Criteria to evaluate
- solves the above mentioned points
- get it running with our tests takes less than one release cycle
- running the tests does not take much longer than currently (2 minutes)
- popularity (in general but also within the GitLab team)
- supports everything that we are currently using in our tests:
- HTML fixtures
- mocks / spies
- manipulating clock
- calculating test coverage
- assertions on the DOM
- run individual tests
Edited by Inactive Account