Report `panic` failures in CI tests
What does this MR do?
- Adds
check * test panic failures
jobs to detected if apanic
log line is found in the test job logs - Introduces
test.SkipIfGitLabCI
helper methods - Skips
TestInteractiveTerminal
when running in CI - Skips some broken tests. Follow-up issues added:
- Skips tests that resulted in
ERROR: Job failed: executor requires OSType=linux, but Docker Engine supports only OSType=windows
, removing them from the Windows ignore files.
Why was this MR needed?
Some tests have been failing for a long time without being detected, reducing confidence in the tests as well as causing a 10 minute delay in the build (e.g. here, here and here).
Once a panic happens in a test run, the subsequent tests in the suite are not run, so in a sense, we've been flying blind in recent times with our CI tests.
What's the best way to test this MR?
- Confirm that the
panic
problems are being reported with the new script changes. -
windows 1809 tests 3/8
now is green, where previously it had ignored failures - Confirm that there are no more
ERROR: Job failed: executor requires OSType=linux, but Docker Engine supports only OSType=windows
failures in the Windows CI test logs.
What are the relevant issue numbers?
Closes #26294 (closed) #26440 (closed)