Skip to content
Snippets Groups Projects
Open GitLab project pipeline improvements > Reduce test suite duration
  • GitLab project pipeline improvements > Reduce test suite duration

  • GitLab project pipeline improvements > Reduce test suite duration

    Open Epic created by Rémy Coutable

    I've created a script to extract GitLab CE master pipelines duration (https://gitlab.com/gitlab-org/gitlab-insights/blob/master/bin/pipelines-duration), and here are the data and graph: https://docs.google.com/spreadsheets/d/1sdppNaGYh1TkhOFdDkSz20YVhc9z9YKtusW-0pmBOtQ/edit#gid=11873453

    Right now, the average pipeline duration for CE on master is 45 minutes, we should aim to reduce that to 30 minutes at the end of Q1 2018.

    Possible improvements:

    • Split some jobs into several jobs

    • Increase parallelization of specs jobs


    Prior work at https://gitlab.com/gitlab-org/gitlab-ce/issues/24899.

    Some additional ideas that don't currently have dedicated issues:

    Group expectations

    Even though in theory we should have one expectation per test, in practice it's often OK to group several expectations (possibly using aggregate_failures), particularly when the test setup is heavy.

    In the same idea you can use the have_attributes matcher, so instead of:

    # Here the setup is run twice
    it { expect(model.name).to eq('foo') }
    it { expect(model.size).to eq(2) }

    you can do:

    # Here the setup is run only once
    it { expect(model).to have_attributes(name: 'foo', size: 2) }

    Chain expectations

    RSpec allows to use compund matchers, so instead of:

    # Here the setup is run twice
    it { is_expected.to start_with('Foo') }
    it { is_expected.to end_with('Bar') }

    you can do:

    # Here the setup is run only once
    it { is_expected.to start_with('Noel').and end_with('Rappin') }

    Move some callbacks to the Service layer

    Callbacks that are only executed on :create or on :update are good candidates to go into the Service layer for a few reasons:

    1. Models shouldn't care about lifecycle-specific behaviors

    2. These extra behaviors won't be triggered when using spec factories (at least until we use the services in factories...) meaning less side-effects and a less slower test suite in the end.

    Examples:

    before_validation :generate_password, on: :create
    before_validation :signup_domain_valid?, on: :create, if: ->(user) { !user.created_by_id }
    after_create :post_create_hook

    from app/models/user.rb.

    Consider removing tests that are duplicated

    We already have a very good coverage and that probably means that we're over-testing some parts of the application. We might want to remove some tests.

    Merge requests

    Edited by Rémy Coutable

    Linked items 0

  • Link items together to show that they're related or that one is blocking others.

    Activity

    • All activity
    • Comments only
    • History only
    • Newest first
    • Oldest first