Evaluate new environment for frontend tests

We are currently using Karma in combination with Jasmine to run our frontend tests. This has a few downsides, namely:

  • we are relying on the now unmaintained jasmine-jquery (https://gitlab.com/gitlab-org/gitlab-ce/issues/24068)
  • tests are all running in the same context and not sandboxed (https://gitlab.com/gitlab-org/gitlab-ce/issues/30322, https://gitlab.com/gitlab-org/gitlab-ce/issues/24100, https://gitlab.com/gitlab-org/gitlab-ce/issues/44238)
  • we have no snapshots (https://gitlab.com/gitlab-org/gitlab-ce/issues/32524)
  • there are no built-in async assertions / Promise based tests (https://gitlab.com/gitlab-org/gitlab-ce/issues/43323)
  • Jasmine is not even listed as an option in the Vue testing guidelines: https://vue-test-utils.vuejs.org/guides/#choosing-a-test-runner

Alternatives to look at

I suggest we limit this evaluation to the tools that are described in the Vue testing guidelines:

  • Jest: https://vue-test-utils.vuejs.org/guides/#testing-single-file-components-with-jest
  • Mocha: https://vue-test-utils.vuejs.org/guides/#testing-single-file-components-with-mocha-webpack
  • Karma + Mocha: https://vue-test-utils.vuejs.org/guides/#testing-single-file-components-with-karma

Criteria to evaluate

  • solves the above mentioned points
  • get it running with our tests takes less than one release cycle
  • running the tests does not take much longer than currently (2 minutes)
  • popularity (in general but also within the GitLab team)
  • supports everything that we are currently using in our tests:
    • HTML fixtures
    • mocks / spies
    • manipulating clock
    • calculating test coverage
    • assertions on the DOM
    • run individual tests
Edited Jul 31, 2018 by Inactive Account
Assignee Loading
Time tracking Loading