Commit 676840ff authored by Rémy Coutable's avatar Rémy Coutable 🔴 Committed by Achilleas Pipinellis

Refactor the Development documentation, and divide the Testing documentation into multiple pages

parent 150f58de
...@@ -296,9 +296,9 @@ might be edited to make them small and simple. ...@@ -296,9 +296,9 @@ might be edited to make them small and simple.
Please submit Feature Proposals using the ['Feature Proposal' issue template](.gitlab/issue_templates/Feature Proposal.md) provided on the issue tracker. Please submit Feature Proposals using the ['Feature Proposal' issue template](.gitlab/issue_templates/Feature Proposal.md) provided on the issue tracker.
For changes in the interface, it is helpful to include a mockup. Issues that add to, or change, the interface should For changes in the interface, it is helpful to include a mockup. Issues that add to, or change, the interface should
be given the ~"UX" label. This will allow the UX team to provide input and guidance. You may be given the ~"UX" label. This will allow the UX team to provide input and guidance. You may
need to ask one of the [core team] members to add the label, if you do not have permissions to do it by yourself. need to ask one of the [core team] members to add the label, if you do not have permissions to do it by yourself.
If you want to create something yourself, consider opening an issue first to If you want to create something yourself, consider opening an issue first to
discuss whether it is interesting to include this in GitLab. discuss whether it is interesting to include this in GitLab.
...@@ -658,7 +658,7 @@ available at [http://contributor-covenant.org/version/1/1/0/](http://contributor ...@@ -658,7 +658,7 @@ available at [http://contributor-covenant.org/version/1/1/0/](http://contributor
[license-finder-doc]: doc/development/licensing.md [license-finder-doc]: doc/development/licensing.md
[GitLab Inc engineering workflow]: https://about.gitlab.com/handbook/engineering/workflow/#labelling-issues [GitLab Inc engineering workflow]: https://about.gitlab.com/handbook/engineering/workflow/#labelling-issues
[polling-etag]: https://docs.gitlab.com/ce/development/polling.html [polling-etag]: https://docs.gitlab.com/ce/development/polling.html
[testing]: doc/development/testing.md [testing]: doc/development/testing_guide/index.md
[^1]: Please note that specs other than JavaScript specs are considered backend [^1]: Please note that specs other than JavaScript specs are considered backend
code. code.
## GitLab Core Team & GitLab Inc. Contribution Process ## GitLab core team & GitLab Inc. contribution process
--- ---
......
# Development # GitLab development guides
## Outside of docs ## Get started!
- [CONTRIBUTING.md](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md) main contributing guide - Setup GitLab's development environment with [GitLab Development Kit (GDK)](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/README.md)
- [PROCESS.md](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/PROCESS.md) contributing process - [GitLab contributing guide](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/CONTRIBUTING.md)
- [GitLab Development Kit (GDK)](https://gitlab.com/gitlab-org/gitlab-development-kit/blob/master/doc/howto/README.md) to install a development version - [Architecture](architecture.md) of GitLab
- [Rake tasks](rake_tasks.md) for development
## Styleguides ## Processes
- [GitLab core team & GitLab Inc. contribution process](https://gitlab.com/gitlab-org/gitlab-ce/blob/master/PROCESS.md)
- [Generate a changelog entry with `bin/changelog`](changelog.md)
- [Code review guidelines](code_review.md) for reviewing code and having code reviewed.
- [Limit conflicts with EE when developing on CE](limit_ee_conflicts.md)
## UX and frontend guides
- [API styleguide](api_styleguide.md) Use this styleguide if you are
contributing to the API.
- [Documentation styleguide](doc_styleguide.md) Use this styleguide if you are
contributing to documentation.
- [Writing documentation](writing_documentation.md)
- [Distinction between general documentation and technical articles](writing_documentation.md#distinction-between-general-documentation-and-technical-articles)
- [SQL Migration Style Guide](migration_style_guide.md) for creating safe SQL migrations
- [Testing standards and style guidelines](testing.md)
- [UX guide](ux_guide/index.md) for building GitLab with existing CSS styles and elements - [UX guide](ux_guide/index.md) for building GitLab with existing CSS styles and elements
- [Frontend guidelines](fe_guide/index.md) - [Frontend guidelines](fe_guide/index.md)
- [SQL guidelines](sql.md) for working with SQL queries
## Backend guides
- [Testing standards and style guidelines](testing_guide/index.md)
- [API styleguide](api_styleguide.md) Use this styleguide if you are
contributing to the API.
- [Sidekiq guidelines](sidekiq_style_guide.md) for working with Sidekiq workers - [Sidekiq guidelines](sidekiq_style_guide.md) for working with Sidekiq workers
- [Working with Gitaly](gitaly.md)
- [Manage feature flags](feature_flags.md)
- [View sent emails or preview mailers](emails.md)
- [Shell commands](shell_commands.md) in the GitLab codebase
- [`Gemfile` guidelines](gemfile.md) - [`Gemfile` guidelines](gemfile.md)
- [Sidekiq debugging](sidekiq_debugging.md)
- [Gotchas](gotchas.md) to avoid
- [Issue and merge requests state models](object_state_models.md)
- [How to dump production data to staging](db_dump.md)
## Process ## Performance guides
- [Generate a changelog entry with `bin/changelog`](changelog.md) - [Instrumentation](instrumentation.md)
- [Limit conflicts with EE when developing on CE](limit_ee_conflicts.md) - [Performance guidelines](performance.md)
- [Code review guidelines](code_review.md) for reviewing code and having code reviewed.
- [Merge request performance guidelines](merge_request_performance_guidelines.md) - [Merge request performance guidelines](merge_request_performance_guidelines.md)
for ensuring merge requests do not negatively impact GitLab performance for ensuring merge requests do not negatively impact GitLab performance
## Backend howtos ## Databases guides
- [Architecture](architecture.md) of GitLab
- [Gotchas](gotchas.md) to avoid
- [How to dump production data to staging](db_dump.md)
- [Instrumentation](instrumentation.md)
- [Performance guidelines](performance.md)
- [Rake tasks](rake_tasks.md) for development
- [Shell commands](shell_commands.md) in the GitLab codebase
- [Sidekiq debugging](sidekiq_debugging.md)
- [Object state models](object_state_models.md)
- [Building a package for testing purposes](build_test_package.md)
- [Manage feature flags](feature_flags.md)
- [View sent emails or preview mailers](emails.md)
- [Working with Gitaly](gitaly.md)
## Databases ### Migrations
- [Merge Request Checklist](database_merge_request_checklist.md)
- [What requires downtime?](what_requires_downtime.md) - [What requires downtime?](what_requires_downtime.md)
- [SQL guidelines](sql.md) for working with SQL queries
- [Migrations style guide](migration_style_guide.md) for creating safe SQL migrations
- [Post deployment migrations](post_deployment_migrations.md)
- [Background migrations](background_migrations.md)
- [Swapping tables](swapping_tables.md)
### Best practices
- [Merge Request checklist](database_merge_request_checklist.md)
- [Adding database indexes](adding_database_indexes.md) - [Adding database indexes](adding_database_indexes.md)
- [Post Deployment Migrations](post_deployment_migrations.md) - [Foreign keys & associations](foreign_keys.md)
- [Foreign Keys & Associations](foreign_keys.md) - [Single table inheritance](single_table_inheritance.md)
- [Serializing Data](serializing_data.md) - [Polymorphic associations](polymorphic_associations.md)
- [Polymorphic Associations](polymorphic_associations.md) - [Serializing data](serializing_data.md)
- [Single Table Inheritance](single_table_inheritance.md) - [Hash indexes](hash_indexes.md)
- [Background Migrations](background_migrations.md) - [Storing SHA1 hashes as binary](sha1_as_binary.md)
- [Storing SHA1 Hashes As Binary](sha1_as_binary.md) - [Iterating tables in batches](iterating_tables_in_batches.md)
- [Iterating Tables In Batches](iterating_tables_in_batches.md) - [Ordering table columns](ordering_table_columns.md)
- [Ordering Table Columns](ordering_table_columns.md) - [Verifying database capabilities](verifying_database_capabilities.md)
- [Verifying Database Capabilities](verifying_database_capabilities.md)
- [Hash Indexes](hash_indexes.md) ## Documentation guides
- [Swapping Tables](swapping_tables.md)
- [Documentation styleguide](doc_styleguide.md): Use this styleguide if you are
## Internationalization (i18n) contributing to the documentation.
- [Writing documentation](writing_documentation.md)
- [Distinction between general documentation and technical articles](writing_documentation.md#distinction-between-general-documentation-and-technical-articles)
## Internationalization (i18n) guides
- [Introduction](i18n/index.md) - [Introduction](i18n/index.md)
- [Externalization](i18n/externalization.md) - [Externalization](i18n/externalization.md)
- [Translation](i18n/translation.md) - [Translation](i18n/translation.md)
## Build guides
- [Building a package for testing purposes](build_test_package.md)
## Compliance ## Compliance
- [Licensing](licensing.md) for ensuring license compliance - [Licensing](licensing.md) for ensuring license compliance
...@@ -54,7 +54,8 @@ or make changes to our frontend development guidelines. ...@@ -54,7 +54,8 @@ or make changes to our frontend development guidelines.
--- ---
## [Testing](testing.md) ## [Testing](../testing_guide/frontend_testing.md)
How we write frontend tests, run the GitLab test suite, and debug test related How we write frontend tests, run the GitLab test suite, and debug test related
issues. issues.
......
# Frontend Testing This document was moved to [../testing_guide/frontend_testing.md](../testing_guide/frontend_testing.md).
There are two types of test suites you'll encounter while developing frontend code
at GitLab. We use Karma and Jasmine for JavaScript unit and integration testing, and RSpec
feature tests with Capybara for e2e (end-to-end) integration testing.
Unit and feature tests need to be written for all new features.
Most of the time, you should use rspec for your feature tests.
There are cases where the behaviour you are testing is not worth the time spent running the full application,
for example, if you are testing styling, animation, edge cases or small actions that don't involve the backend,
you should write an integration test using Jasmine.
![Testing priority triangle](img/testing_triangle.png)
_This diagram demonstrates the relative priority of each test type we use_
Regression tests should be written for bug fixes to prevent them from recurring in the future.
See [the Testing Standards and Style Guidelines](../testing.md)
for more information on general testing practices at GitLab.
## Karma test suite
GitLab uses the [Karma][karma] test runner with [Jasmine][jasmine] as its test
framework for our JavaScript unit and integration tests. For integration tests,
we generate HTML files using RSpec (see `spec/javascripts/fixtures/*.rb` for examples).
Some fixtures are still HAML templates that are translated to HTML files using the same mechanism (see `static_fixtures.rb`).
Adding these static fixtures should be avoided as they are harder to keep up to date with real views.
The existing static fixtures will be migrated over time.
Please see [gitlab-org/gitlab-ce#24753](https://gitlab.com/gitlab-org/gitlab-ce/issues/24753) to track our progress.
Fixtures are served during testing by the [jasmine-jquery][jasmine-jquery] plugin.
JavaScript tests live in `spec/javascripts/`, matching the folder structure
of `app/assets/javascripts/`: `app/assets/javascripts/behaviors/autosize.js`
has a corresponding `spec/javascripts/behaviors/autosize_spec.js` file.
Keep in mind that in a CI environment, these tests are run in a headless
browser and you will not have access to certain APIs, such as
[`Notification`](https://developer.mozilla.org/en-US/docs/Web/API/notification),
which will have to be stubbed.
### Best practice
#### Naming unit tests
When writing describe test blocks to test specific functions/methods,
please use the method name as the describe block name.
```javascript
// Good
describe('methodName', () => {
it('passes', () => {
expect(true).toEqual(true);
});
});
// Bad
describe('#methodName', () => {
it('passes', () => {
expect(true).toEqual(true);
});
});
// Bad
describe('.methodName', () => {
it('passes', () => {
expect(true).toEqual(true);
});
});
```
#### Testing Promises
When testing Promises you should always make sure that the test is asynchronous and rejections are handled.
Your Promise chain should therefore end with a call of the `done` callback and `done.fail` in case an error occurred.
```javascript
// Good
it('tests a promise', (done) => {
promise
.then((data) => {
expect(data).toBe(asExpected);
})
.then(done)
.catch(done.fail);
});
// Good
it('tests a promise rejection', (done) => {
promise
.then(done.fail)
.catch((error) => {
expect(error).toBe(expectedError);
})
.then(done)
.catch(done.fail);
});
// Bad (missing done callback)
it('tests a promise', () => {
promise
.then((data) => {
expect(data).toBe(asExpected);
})
});
// Bad (missing catch)
it('tests a promise', (done) => {
promise
.then((data) => {
expect(data).toBe(asExpected);
})
.then(done)
});
// Bad (use done.fail in asynchronous tests)
it('tests a promise', (done) => {
promise
.then((data) => {
expect(data).toBe(asExpected);
})
.then(done)
.catch(fail)
});
// Bad (missing catch)
it('tests a promise rejection', (done) => {
promise
.catch((error) => {
expect(error).toBe(expectedError);
})
.then(done)
});
```
#### Stubbing
For unit tests, you should stub methods that are unrelated to the current unit you are testing.
If you need to use a prototype method, instantiate an instance of the class and call it there instead of mocking the instance completely.
For integration tests, you should stub methods that will effect the stability of the test if they
execute their original behaviour. i.e. Network requests.
### Vue.js unit tests
See this [section][vue-test].
### Running frontend tests
`rake karma` runs the frontend-only (JavaScript) tests.
It consists of two subtasks:
- `rake karma:fixtures` (re-)generates fixtures
- `rake karma:tests` actually executes the tests
As long as the fixtures don't change, `rake karma:tests` (or `yarn karma`)
is sufficient (and saves you some time).
### Live testing and focused testing
While developing locally, it may be helpful to keep karma running so that you
can get instant feedback on as you write tests and modify code. To do this
you can start karma with `npm run karma-start`. It will compile the javascript
assets and run a server at `http://localhost:9876/` where it will automatically
run the tests on any browser which connects to it. You can enter that url on
multiple browsers at once to have it run the tests on each in parallel.
While karma is running, any changes you make will instantly trigger a recompile
and retest of the entire test suite, so you can see instantly if you've broken
a test with your changes. You can use [jasmine focused][jasmine-focus] or
excluded tests (with `fdescribe` or `xdescribe`) to get karma to run only the
tests you want while you're working on a specific feature, but make sure to
remove these directives when you commit your code.
## RSpec Feature Integration Tests
Information on setting up and running RSpec integration tests with
[Capybara][capybara] can be found in the
[general testing guide](../testing.md).
## Gotchas
### Errors due to use of unsupported JavaScript features
Similar errors will be thrown if you're using JavaScript features not yet
supported by the PhantomJS test runner which is used for both Karma and RSpec
tests. We polyfill some JavaScript objects for older browsers, but some
features are still unavailable:
- Array.from
- Array.first
- Async functions
- Generators
- Array destructuring
- For..Of
- Symbol/Symbol.iterator
- Spread
Until these are polyfilled appropriately, they should not be used. Please
update this list with additional unsupported features.
### RSpec errors due to JavaScript
By default RSpec unit tests will not run JavaScript in the headless browser
and will simply rely on inspecting the HTML generated by rails.
If an integration test depends on JavaScript to run correctly, you need to make
sure the spec is configured to enable JavaScript when the tests are run. If you
don't do this you'll see vague error messages from the spec runner.
To enable a JavaScript driver in an `rspec` test, add `:js` to the
individual spec or the context block containing multiple specs that need
JavaScript enabled:
```ruby
# For one spec
it 'presents information about abuse report', :js do
# assertions...
end
describe "Admin::AbuseReports", :js do
it 'presents information about abuse report' do
# assertions...
end
it 'shows buttons for adding to abuse report' do
# assertions...
end
end
```
### Spinach errors due to missing JavaScript
> **Note:** Since we are discouraging the use of Spinach when writing new
> feature tests, you shouldn't ever need to use this. This information is kept
> available for legacy purposes only.
In Spinach, the JavaScript driver is enabled differently. In the `*.feature`
file for the failing spec, add the `@javascript` flag above the Scenario:
```
@javascript
Scenario: Developer can approve merge request
Given I am a "Shop" developer
And I visit project "Shop" merge requests page
And merge request 'Bug NS-04' must be approved
And I click link "Bug NS-04"
When I click link "Approve"
Then I should see approved merge request "Bug NS-04"
```
[capybara]: http://teamcapybara.github.io/capybara/
[jasmine]: https://jasmine.github.io/
[jasmine-focus]: https://jasmine.github.io/2.5/focused_specs.html
[jasmine-jquery]: https://github.com/velesin/jasmine-jquery
[karma]: http://karma-runner.github.io/
[vue-test]:https://docs.gitlab.com/ce/development/fe_guide/vue.html#testing-vue-components
This diff is collapsed.
This diff is collapsed.
# GitLab tests in the Continuous Integration (CI) context
### Test suite parallelization on the CI
Our current CI parallelization setup is as follows:
1. The `knapsack` job in the prepare stage that is supposed to ensure we have a
`knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file:
- The `knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file is fetched
from S3, if it's not here we initialize the file with `{}`.
1. Each `rspec x y` job are run with `knapsack rspec` and should have an evenly
distributed share of tests:
- It works because the jobs have access to the
`knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` since the "artifacts
from all previous stages are passed by default". [^1]
- the jobs set their own report path to
`KNAPSACK_REPORT_PATH=knapsack/${CI_PROJECT_NAME}/${JOB_NAME[0]}_node_${CI_NODE_INDEX}_${CI_NODE_TOTAL}_report.json`.
- if knapsack is doing its job, test files that are run should be listed under
`Report specs`, not under `Leftover specs`.
1. The `update-knapsack` job takes all the
`knapsack/${CI_PROJECT_NAME}/${JOB_NAME[0]}_node_${CI_NODE_INDEX}_${CI_NODE_TOTAL}_report.json`
files from the `rspec x y` jobs and merge them all together into a single
`knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file that is then
uploaded to S3.
After that, the next pipeline will use the up-to-date
`knapsack/${CI_PROJECT_NAME}/rspec_report-master.json` file. The same strategy
is used for Spinach tests as well.
### Monitoring
The GitLab test suite is [monitored] for the `master` branch, and any branch
that includes `rspec-profile` in their name.
A [public dashboard] is available for everyone to see. Feel free to look at the
slowest test files and try to improve them.
[monitored]: ../performance.md#rspec-profiling
[public dashboard]: https://redash.gitlab.com/public/dashboards/l1WhHXaxrCWM5Ai9D7YDqHKehq6OU3bx5gssaiWe?org_slug=default
## CI setup
- On CE and EE, the test suite runs both PostgreSQL and MySQL.
- Rails logging to `log/test.log` is disabled by default in CI [for
performance reasons][logging]. To override this setting, provide the
`RAILS_ENABLE_TEST_LOG` environment variable.
[logging]: https://jtway.co/speed-up-your-rails-test-suite-by-6-in-1-line-13fedb869ec4
---
[Return to Testing documentation](index.md)
# Flaky tests
## What's a flaky test?
It's a test that sometimes fails, but if you retry it enough times, it passes,
eventually.
## Automatic retries and flaky tests detection
On our CI, we use [rspec-retry] to automatically retry a failing example a few
times (see [`spec/spec_helper.rb`] for the precise retries count).
We also use a home-made `RspecFlaky::Listener` listener which records flaky
examples in a JSON report file on `master` (`retrieve-tests-metadata` and `update-tests-metadata` jobs), and warns when a new flaky example
is detected in any other branch (`flaky-examples-check` job). In the future, the
`flaky-examples-check` job will not be allowed to fail.
This was originally implemented in: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/13021.
[rspec-retry]: https://github.com/NoRedInk/rspec-retry
[`spec/spec_helper.rb`]: https://gitlab.com/gitlab-org/gitlab-ce/blob/master/spec/spec_helper.rb
## Problems we had in the past at GitLab
- [`rspec-retry` is bitting us when some API specs fail](https://gitlab.com/gitlab-org/gitlab-ce/issues/29242): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/9825
- [Sporadic RSpec failures due to `PG::UniqueViolation`](https://gitlab.com/gitlab-org/gitlab-ce/issues/28307#note_24958837): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/9846
- Follow-up: https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10688
- [Capybara.reset_session! should be called before requests are blocked](https://gitlab.com/gitlab-org/gitlab-ce/issues/33779): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12224
- FFaker generates funky data that tests are not ready to handle (and tests should be predictable so that's bad!):
- [Make `spec/mailers/notify_spec.rb` more robust](https://gitlab.com/gitlab-org/gitlab-ce/issues/20121): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10015
- [Transient failure in spec/requests/api/commits_spec.rb](https://gitlab.com/gitlab-org/gitlab-ce/issues/27988#note_25342521): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/9944
- [Replace FFaker factory data with sequences](https://gitlab.com/gitlab-org/gitlab-ce/issues/29643): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10184
- [Transient failure in spec/finders/issues_finder_spec.rb](https://gitlab.com/gitlab-org/gitlab-ce/issues/30211#note_26707685): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10404
### Time-sensitive flaky tests
- https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10046
- https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10306
### Array order expectation
- https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10148
### Feature tests
- [Be sure to create all the data the test need before starting exercize](https://gitlab.com/gitlab-org/gitlab-ce/issues/32622#note_31128195): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12059
- [Bis](https://gitlab.com/gitlab-org/gitlab-ce/issues/34609#note_34048715): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12604
- [Bis](https://gitlab.com/gitlab-org/gitlab-ce/issues/34698#note_34276286): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12664
- [Assert against the underlying database state instead of against a page's content](https://gitlab.com/gitlab-org/gitlab-ce/issues/31437): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10934
#### Capybara viewport size related issues
- [Transient failure of spec/features/issues/filtered_search/filter_issues_spec.rb](https://gitlab.com/gitlab-org/gitlab-ce/issues/29241#note_26743936): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10411
#### Capybara JS driver related issues
- [Don't wait for AJAX when no AJAX request is fired](https://gitlab.com/gitlab-org/gitlab-ce/issues/30461): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/10454
- [Bis](https://gitlab.com/gitlab-org/gitlab-ce/issues/34647): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12626
#### PhantomJS / WebKit related issues
- Memory is through the roof! (TL;DR: Load images but block images requests!): https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/12003
## Resources
- [Flaky Tests: Are You Sure You Want to Rerun Them?](http://semaphoreci.com/blog/2017/04/20/flaky-tests.html)
- [How to Deal With and Eliminate Flaky Tests](https://semaphoreci.com/community/tutorials/how-to-deal-with-and-eliminate-flaky-tests)
- [Tips on Treating Flakiness in your Rails Test Suite](http://semaphoreci.com/blog/2017/08/03/tips-on-treating-flakiness-in-your-test-suite.html)
- ['Flaky' tests: a short story](https://www.ombulabs.com/blog/rspec/continuous-integration/how-to-track-down-a-flaky-test.html)
- [Using Insights to Discover Flaky, Slow, and Failed Tests](https://circleci.com/blog/using-insights-to-discover-flaky-slow-and-failed-tests/)
---
[Return to Testing documentation](index.md)
# Frontend testing standards and style guidelines
There are two types of test suites you'll encounter while developing frontend code
at GitLab. We use Karma and Jasmine for JavaScript unit and integration testing,
and RSpec feature tests with Capybara for e2e (end-to-end) integration testing.
Unit and feature tests need to be written for all new features.
Most of the time, you should use [RSpec] for your feature tests.
Regression tests should be written for bug fixes to prevent them from recurring
in the future.
See the [Testing Standards and Style Guidelines](index.md) page for more
information on general testing practices at GitLab.
## Karma test suite
GitLab uses the [Karma][karma] test runner with [Jasmine] as its test
framework for our JavaScript unit and integration tests. For integration tests,
we generate HTML files using RSpec (see `spec/javascripts/fixtures/*.rb` for examples).
Some fixtures are still HAML templates that are translated to HTML files using the same mechanism (see `static_fixtures.rb`).
Adding these static fixtures should be avoided as they are harder to keep up to date with real views.
The existing static fixtures will be migrated over time.
Please see [gitlab-org/gitlab-ce#24753](https://gitlab.com/gitlab-org/gitlab-ce/issues/24753) to track our progress.
Fixtures are served during testing by the [jasmine-jquery][jasmine-jquery] plugin.
JavaScript tests live in `spec/javascripts/`, matching the folder structure
of `app/assets/javascripts/`: `app/assets/javascripts/behaviors/autosize.js`
has a corresponding `spec/javascripts/behaviors/autosize_spec.js` file.
Keep in mind that in a CI environment, these tests are run in a headless
browser and you will not have access to certain APIs, such as
[`Notification`](https://developer.mozilla.org/en-US/docs/Web/API/notification),
which will have to be stubbed.
### Best practices
#### Naming unit tests
When writing describe test blocks to test specific functions/methods,
please use the method name as the describe block name.
```javascript
// Good
describe('methodName', () => {
it('passes', () => {
expect(true).toEqual(true);
});
});
// Bad
describe('#methodName', () => {
it('passes', () => {
expect(true).toEqual(true);
});
});
// Bad
describe('.methodName', () => {
it('passes', () => {
expect(true).toEqual(true);
});
});
```
#### Testing promises
When testing Promises you should always make sure that the test is asynchronous and rejections are handled.
Your Promise chain should therefore end with a call of the `done` callback and `done.fail` in case an error occurred.
```javascript
// Good
it('tests a promise', (done) => {
promise
.then((data) => {
expect(data).toBe(asExpected);
})
.then(done)
.catch(done.fail);
});
// Good
it('tests a promise rejection', (done) => {
promise
.then(done.fail)
.catch((error) => {
expect(error).toBe(expectedError);
})
.then(done)
.catch(done.fail);
});
// Bad (missing done callback)
it('tests a promise', () => {
promise
.then((data) => {
expect(data).toBe(asExpected);
})
});
// Bad (missing catch)
it('tests a promise', (done) => {
promise
.then((data) => {
expect(data).toBe(asExpected);
})
.then(done)
});
// Bad (use done.fail in asynchronous tests)
it('tests a promise', (done) => {
promise
.then((data) => {
expect(data).toBe(asExpected);
})
.then(done)
.catch(fail)
});
// Bad (missing catch)
it('tests a promise rejection', (done) => {
promise
.catch((error) => {
expect(error).toBe(expectedError);
})
.then(done)
});
```
#### Stubbing
For unit tests, you should stub methods that are unrelated to the current unit you are testing.
If you need to use a prototype method, instantiate an instance of the class and call it there instead of mocking the instance completely.
For integration tests, you should stub methods that will effect the stability of the test if they
execute their original behaviour. i.e. Network requests.
### Vue.js unit tests
See this [section][vue-test].
### Running frontend tests