Skip to content

Add timing information and optimize the `setup-test-env` job

What does this MR do?

  1. Add timing information to the setup-test-env job and some CI steps in general with a dedicated helper.
  2. De-deduplicate the setup-test-env pg* jobs with bundle pristine pg. Adding 10 seconds. Since we now only test against PG 11, we don't need bundle pristine pg for now (but we'll do once we'll test against multiple versions again in the future).
  3. Group bundle exec rake db:drop db:create db:structure:load db:migrate and bundle exec rake gitlab:db:setup_ee into a single command. Saving 13 seconds.
  4. Disable eager-loading in the test environment + replace bundle exec ruby -Ispec -e 'require "spec_helper" ; TestEnv.init' with bundle exec ruby -I. -e 'require "config/environment"; TestEnv.init'. Saving 49 seconds.
  5. Reduce the folders we cache. Saving 23 seconds in upload and 16 seconds in download.
  6. Reduce the folders we save as artifacts (remove vendor/gitaly-ruby). Saving 17 seconds in upload.

Total expected time saved per setup-test-env job so far: 16 - 10 + 13 + 49 + 23 + 17 = 118 = 1 minute 58 seconds! That would represent a 108/562 = 20.99% improvement.

These numbers are optimistic and by looking at the latest pipelines from this MR, we are getting a 10% improvement rate: https://app.periscopedata.com/app/gitlab/647974/WIP:-R%C3%A9my-Coutable-Scratchpad?widget=8597685&udv=1002157

Before change

Section Duration (seconds) Cumulative duration (seconds) % of total Can be improved at the job level?
Preparing the "docker+machine" executor 6 6 1 % No
Preparing environment 3 9 0.5 % No
Getting source from Git repository 56 65 9.9 % No
Restoring cache 102 167 18.1 % Yes
Downloading artifacts 4 171 0.7 % Yes
Running before_script and script 251 422 44.6 % Yes
Running after_script 3 425 0.5 % Yes
Saving cache 78 503 13.8% Yes
Uploading artifacts for failed job 59 562 10.4 % Yes

Restoring cache

1 minute 39 seconds.

DB setup step

==> 'bundle exec rake db:drop db:create db:structure:load db:migrate' succeeded in 22 seconds.
==> 'bundle exec rake gitlab:db:setup_ee' succeeded in 13 seconds.

TestEnv.init step

 $ time bundle exec ruby -Ispec -e 'require "spec_helper" ; TestEnv.init'
 ==> Setting up Gitaly...
  Warning 
   You are running as user root, we hope you know what you are doing.
   Things may work/fail for the wrong reasons.
   For correct results you should run this as user git.
     Gitaly set up in 14.945335431 seconds...
 ==> Setting up GitLab Workhorse...
  Warning 
   You are running as user root, we hope you know what you are doing.
   Things may work/fail for the wrong reasons.
   For correct results you should run this as user git.
     GitLab Workhorse set up in 38.620998438 seconds...
 ==> Setting up GitLab Elasticsearch Indexer...
     GitLab Elasticsearch Indexer set up in 32.122182656 seconds...
 real	2m26.653s
 user	2m39.842s
 sys	0m14.670s

scripts/gitaly-test-build step

 real	0m42.870s

Saving cache step

1 minute 18 seconds.

Uploading artifacts step

49 seconds.

After change

Restoring cache

1 minute 23 seconds: 16 seconds saved.

Extra bundle pristine pg step (that allows to de-duplicate the setup-test-env pg* jobs)

==> 'bundle pristine pg' succeeded in 10 seconds.: 10 seconds added.

DB setup step

==> 'bundle exec rake db:drop db:create db:structure:load db:migrate gitlab:db:setup_ee' succeeded in 22 seconds.: 13 seconds saved.

TestEnv.init step

==> 'bundle exec ruby -I. -e 'require "config/environment"; TestEnv.init'' succeeded in 97 seconds.: 49 seconds saved.

scripts/gitaly-test-build step

==> 'scripts/gitaly-test-build' succeeded in 42 seconds.: Same duration.

Saving cache step

55 seconds: 23 seconds saved.

Uploading artifacts step

32 seconds: 17 seconds saved.

Additional potential savings

  • We can save 20 seconds on Uploading artifacts by not uploading vendor/gitaly-ruby since it's already present in the cache. I need to ensure tests still pass without it, though.

Total saved so far: 16 - 10 + 13 + 49 + 23 + 17 = 118 = 1 minute 48 seconds!

Relates to #215005 (closed).

Edited by Rémy Coutable

Merge request reports