GitLab project pipeline improvements > Reduce test suite duration
GitLab project pipeline improvements > Reduce test suite duration
I've created a script to extract GitLab CE master
pipelines duration (https://gitlab.com/gitlab-org/gitlab-insights/blob/master/bin/pipelines-duration), and here are the data and graph: https://docs.google.com/spreadsheets/d/1sdppNaGYh1TkhOFdDkSz20YVhc9z9YKtusW-0pmBOtQ/edit#gid=11873453
Right now, the average pipeline duration for CE on master
is 45 minutes, we should aim to reduce that to 30 minutes at the end of Q1 2018.
Possible improvements:
-
Split some jobs into several jobs
-
Increase parallelization of specs jobs
Prior work at https://gitlab.com/gitlab-org/gitlab-ce/issues/24899.
Some additional ideas that don't currently have dedicated issues:
Group expectations
Even though in theory we should have one expectation per test, in practice it's
often OK to group several expectations (possibly using aggregate_failures
),
particularly when the test setup is heavy.
In the same idea you can use the have_attributes
matcher, so instead of:
# Here the setup is run twice
it { expect(model.name).to eq('foo') }
it { expect(model.size).to eq(2) }
you can do:
# Here the setup is run only once
it { expect(model).to have_attributes(name: 'foo', size: 2) }
Chain expectations
RSpec allows to use compund matchers, so instead of:
# Here the setup is run twice
it { is_expected.to start_with('Foo') }
it { is_expected.to end_with('Bar') }
you can do:
# Here the setup is run only once
it { is_expected.to start_with('Noel').and end_with('Rappin') }
Move some callbacks to the Service layer
Callbacks that are only executed on :create
or on :update
are good
candidates to go into the Service layer for a few reasons:
-
Models shouldn't care about lifecycle-specific behaviors
-
These extra behaviors won't be triggered when using spec factories (at least until we use the services in factories...) meaning less side-effects and a less slower test suite in the end.
Examples:
before_validation :generate_password, on: :create
before_validation :signup_domain_valid?, on: :create, if: ->(user) { !user.created_by_id }
after_create :post_create_hook
from app/models/user.rb
.
Consider removing tests that are duplicated
We already have a very good coverage and that probably means that we're over-testing some parts of the application. We might want to remove some tests.
Merge requests
- https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/13321
- https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/16593
- https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/16621
- https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/16675
- https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/16700
- Show closed items