Improve CI job speed through additional caching

In many cases, we should be able to short-circuit a lot of repeated tasks for which the input and output has not changed from the last time a pipeline has run.

For instance, we can easily cache the karma fixtures which are currently re-generated in every pipeline, like so:

  1. check for existing cache of karma fixtures
  2. create a digest string based on the concatenated contents of /app/views, /ee/app/views, and /spec/javascripts/fixtures
  3. compare this digest string to the one saved within the cache
  4. if it matches
    1. use the cached fixtures
  5. if it does not match
    1. re-generate the fixtures
    2. if on the master branch
      1. store the newly-generated fixtures along with our digest string and a timestamp
      2. use FIFO policy to remove the 21st-oldest cache of fixtures, keeping the 20 most recent revisions

In 90% of branches, where no changes are made to the files in those directories, the karma job will run extremely fast.


The setup-test-env task could also be sped up by caching the compiled output of both our asset pipeline and utilizing several caching layers within our webpack config. The cache maintenance and invalidation could work similar to the one outlined above for the karma fixtures.

If these prove to be really effective, we could perhaps turn this concept into an actual feature of gitlab runner, rather than implementing this cache invalidation within our CI config manually.

Assignee Loading
Time tracking Loading