[steps] Deflake steps command tests
What does this MR do?
[steps] Deflake steps command tests
The test binary might be compiled with coverage enabled. Thus, when the test binary runs mimicking external commands, it also runs with coverage enabled. Esp. on windows this might lead to errors like:
error: coverage meta-data emit failed: writing C:\Users\GITLAB~1\AppData\Local\Temp\go-build1443630877\b1861\gocoverdir\covmeta.4db6b605483bb6557665265358fbe732: rename from C:\Users\GITLAB~1\AppData\Local\Temp\go-build1443630877\b1861\gocoverdir\tmp.covmeta.4db6b605483bb6557665265358fbe7321761171284598833100 failed: rename C:\Users\GITLAB~1\AppData\Local\Temp\go-build1443630877\b1861\gocoverdir\tmp.covmeta.4db6b605483bb6557665265358fbe7321761171284598833100 C:\Users\GITLAB~1\AppData\Local\Temp\go-build1443630877\b1861\gocoverdir\covmeta.4db6b605483bb6557665265358fbe732: The process cannot access the file because it is being used by another process.(see e.g. https://gitlab.com/gitlab-org/gitlab-runner/-/jobs/11814901603#L624)
We can't disable coverage once the test binary is compiled. Therefore we change the directory for coverage files to something else when the test binary runs mimicking an external binary. The coverage data is still created, but somewhere else.
We need to ensure not to clean up the coverage directory too early, else
we might see errors where the expected raw coverage data can't be
accessed anymore. Thus, a t.TempDir isn't a good fit here.
So we create a temporary directory in the main process, and set an env
var with the path to that. When we run the test binary in "external
binary mode", we pick that up, create a temp directory in there and use
that for GOCOVERDIR. In the main process, after all tests ran, we then
clean up this temporary directory.
Why was this MR needed?
To deflake tests
What's the best way to test this MR?
run the respective tests
- with coverage
- multipe times
- esp. on windows
go test -failfast -cover -coverprofile ./prof -coverpkg gitlab.com/gitlab-org/gitlab-runner/... -v -count=100 -parallel=10 .\commands\steps\What are the relevant issue numbers?
-