Only allow failures with exit code 99 in Linux tests (!2704) · Merge requests · GitLab.org / gitlab-runner

Pedro Pombeiro requested to merge 27474-allow_failure_exit_code_1 into main Jan 22, 2021

What does this MR do?

It leverages the new allow_failures:exit_codes functionality in GitLab Runner 13.8 in order to narrow down the types of failures that will be allowed.

We need to return a well-known exit code from the scripts that run tests (in this case 99) and only accept that error code (or success). Anything else (like a timeout) should cause the job to fail.

Why was this MR needed?

Currently, a system failure like a timeout will be allowed (and therefore considered as a "good" build), which is not right.

What's the best way to test this MR?

Check if the Linux race unit tests still pass. I guess we'll need to keep an eye open to see if there are other error codes that will need to be allowed, but so far it doesn't seem so.