Skip to content

test(integration): do not preallocate 5Gb of memory for in memory test

What does this MR do?

Context

All “inmemory” integration tests have been consistently failing on the master branch for a very long time now with no identifiable-logged reason (ref: https://gitlab.com/gitlab-org/container-registry/-/jobs/5376542409).

Findings

After some preliminary analysis (in !1489 (commits)) it was discovered that the reason they fail only on the master branch and no other branch was because only the master branch ran the full “inmemory” test package and did not “skip” any tests.

The problematic test that was being skipped in other branches (and hence giving us a false positive) was “TestWriteReadLargeStreams” . The reason this test fails is it requests too much memory from the runner (i.e 5Gb) and gets OOM killed. The runner does not surface this reason weirdly enough.

Solution

We don’t necessarily need to pre-allocate 5Gb for this test so I’ve reduced it to 2Gb and the test seems to consistently pass now: 2f3924b4

Bonus

This should likely fix the consistently failing swift jobs on master which appear to be plagued by the same issue: https://gitlab.com/gitlab-org/container-registry/-/jobs/5376542415

Author checklist

  • Feature flags
    • Added feature flag:
    • This feature does not require a feature flag
  • I added unit tests or they are not required
  • I added documentation (or it's not required)
  • I followed code review guidelines
  • I followed Go Style guidelines
  • For database changes including schema migrations:
    • Manually run up and down migrations in a postgres.ai production database clone and post a screenshot of the result here.
    • If adding new queries, extract a query plan from postgres.ai and post the link here. If changing existing queries, also extract a query plan for the current version for comparison.
    • Do not include code that depends on the schema migrations in the same commit. Split the MR into two or more.
  • Ensured this change is safe to deploy to individual stages in the same environment (cny -> prod). State-related changes can be troublesome due to having parts of the fleet processing (possibly related) requests in different ways.

Reviewer checklist

  • Ensure the commit and MR tittle are still accurate.
  • If the change contains a breaking change, apply the breaking change label.
  • If the change is considered high risk, apply the label high-risk-change
  • Identify if the change can be rolled back safely. (note: all other reasons for not being able to rollback will be sufficiently captured by major version changes).

If the MR introduces database schema migrations:

  • Ensure the commit and MR tittle start with fix:, feat:, or perf: so that the change appears on the Changelog
If the changes cannot be rolled back follow these steps:
  • If not, apply the label cannot-rollback.
  • Add a section to the MR description that includes the following details:
    • The reasoning behind why a release containing the presented MR can not be rolled back (e.g. schema migrations or changes to the FS structure)
    • Detailed steps to revert/disable a feature introduced by the same change where a migration cannot be rolled back. (note: ideally MRs containing schema migrations should not contain feature changes.)
    • Ensure this MR does not add code that depends on these changes that cannot be rolled back.
Edited by Hayley Swimelar

Merge request reports

Loading