Flaky acceptance tests due to port already in use
Since we split the acceptance tests execution in !403 (merged), we now get a bunch of errors due to the ports being already in use.
Retrying the jobs one by one in a failing pipeline seem to be enough to make the tests pass.
Sample job https://gitlab.com/gitlab-org/gitlab-pages/-/jobs/912690661
msg="could not create socket" error="listen tcp [::1]:37000: bind: cannot assign requested address"
A potential solution is to use TCP port 0 and let the OS assign an available port. However, there is no easy way to communicate the port that was assigned to the Pages acceptance tests. This causes WaitUntilRequestSucceeds
to wait until timeout to get a response from the binary currently running.
Sample faling pipeline https://gitlab.com/gitlab-org/gitlab-pages/-/pipelines/237350515
Update 2020-01-07
IPv6 seems to be disabled in some gitlab-org-docker
runners https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/12258.