Skip to content

Resolve "Random out of memory and timeout errors running integration tests in GitLab CI"

Problem

Please see #11 (closed) for context.

The following integration tests are putting a lot of pressure on the CI runners:

  1. DriverSuite.TestConcurrentFileStreams: Tests that multiple *os.File objects can be passed in to Writer concurrently without hanging.
  2. DriverSuite.TestConcurrentStreamReads: Tests that multiple clients can safely read from the same file simultaneously with various offsets.
  3. DriverSuite.TestWriteReadLargeStreams: Tests that a 5GB file may be written to the storage.

Proposed Solution

Tests 1 and 2 support the "short" mode flag, in which case a smaller number of streams and smaller files are used, respectively.

Test 3 also supports "short" mode, in which case the test is skipped (there are other integration tests that use smaller files, so it doesn't make sense to reduce the file size when in short mode).

Considering the above, the proposed solution is to use the "short" mode flag for integration tests run in GitLab CI.

This not only solves the random errors issue but it also drops the execution time from +15 minutes (example) to ~3 minutes (example), which improves the development and review process as well.

Edited by João Pereira

Merge request reports