Flaky tests due to bind: address already in use
Job #1003694350 failed for f6e3cae7:
func TestMultipleListenersFromEnvironmentVariables(t *testing.T) {
skipUnlessEnabled(t)
listenSpecs := []ListenSpec{{"http", "127.0.0.1", "37001"}, {"http", "127.0.0.1", "37002"}}
envVarValue := fmt.Sprintf("LISTEN_HTTP=%s,%s", net.JoinHostPort("127.0.0.1", "37001"), net.JoinHostPort("127.0.0.1", "37002"))
RunPagesProcessWithStubGitLabServer(t,
withoutWait,
withListeners([]ListenSpec{}), // explicitly disable listeners for this test
withEnv([]string{envVarValue}),
)
for _, listener := range listenSpecs {
require.NoError(t, listener.WaitUntilRequestSucceeds(nil))
rsp, err := GetPageFromListener(t, listener, "group.gitlab-example.com", "project/")
require.NoError(t, err)
rsp.Body.Close()
require.Equal(t, http.StatusOK, rsp.StatusCode)
}
}
We set two http listeners on ports 37001 and 37002. The test usually fails to bind port 37002. The port is not used in any other tests. However, we run the tests against multiple Go versions concurrently. My suspicion is that different jobs run on the same host concurrently.
Looking closer at the job logs:
- Job
test-acceptance:1.14
https://gitlab.com/gitlab-org/gitlab-pages/-/jobs/1003694353 - ran successfully on runnerrunner-bd091556-project-734943-concurrent-0
- Job
test:1.13
https://gitlab.com/gitlab-org/gitlab-pages/-/jobs/1003694350 - ran and failed on the same runnerrunner-bd091556-project-734943-concurrent-0
One idea was to find an open port prior to running the tests, for example https://github.com/phayes/freeport. However I found that some jobs may get the same open port but I didn't test very closely.
Retrying jobs usually fix the pipeline
Edited by Jaime Martinez