sync-docker-images contains a data race when building sync commands
Summary
While investigating a comment on !4139 (merged), I found a data race in building the sync commands.
Steps to reproduce
I described how I encountered the data race here, which includes a commit I made on a separate branch to add a --dry-run argument to the sync, so no actual syncing is done. I've added also added that commit to my proposed fix branch to ease the testing.
To reproduce:
- Checkout gitlab-community/gitlab-runner@97568f78
cd scripts/sync-docker-images
go build -race
PUSH_TO_DOCKER_HUB=true ./sync-docker-images -version v16.1.0 --dry-run
- Note the reported data race in the output. Also, you can grep for the Cmd: lines and see they sometimes have repetitions.
Actual behavior
The Cmd: output from a `--dry-run` sometimes has repeated images (e.g. alpine3.13) and reports a DATA RACE
$ go build -race
$ PUSH_TO_DOCKER_HUB=true ./sync-docker-images -version v16.1.0 --dry-run 2>&1 | grep -E '(WARNING|gitlab-runner:)'
2023/07/01 10:14:23 Copying image registry.gitlab.com/gitlab-org/gitlab-runner:alpine-v16.1.0 => registry.hub.docker.com/gitlab/gitlab-runner:alpine-v16.1.0
WARNING: DATA RACE
Cmd: skopeo copy --all docker://registry.gitlab.com/gitlab-org/gitlab-runner:alpine-v16.1.0 docker://registry.hub.docker.com/gitlab/gitlab-runner:alpine-v16.1.0
2023/07/01 10:14:23 Copying image registry.gitlab.com/gitlab-org/gitlab-runner:alpine3.12-v16.1.0 => registry.hub.docker.com/gitlab/gitlab-runner:alpine3.12-v16.1.0
Cmd: skopeo copy --all docker://registry.gitlab.com/gitlab-org/gitlab-runner:alpine3.13-v16.1.0 docker://registry.hub.docker.com/gitlab/gitlab-runner:alpine3.13-v16.1.0
2023/07/01 10:14:23 Copying image registry.gitlab.com/gitlab-org/gitlab-runner:alpine3.13-v16.1.0 => registry.hub.docker.com/gitlab/gitlab-runner:alpine3.13-v16.1.0
Cmd: skopeo copy --all docker://registry.gitlab.com/gitlab-org/gitlab-runner:alpine3.13-v16.1.0 docker://registry.hub.docker.com/gitlab/gitlab-runner:alpine3.13-v16.1.0
2023/07/01 10:14:23 Copying image registry.gitlab.com/gitlab-org/gitlab-runner:alpine3.14-v16.1.0 => registry.hub.docker.com/gitlab/gitlab-runner:alpine3.14-v16.1.0
Cmd: skopeo copy --all docker://registry.gitlab.com/gitlab-org/gitlab-runner:alpine3.14-v16.1.0 docker://registry.hub.docker.com/gitlab/gitlab-runner:alpine3.15-v16.1.0
2023/07/01 10:14:23 Copying image registry.gitlab.com/gitlab-org/gitlab-runner:alpine3.15-v16.1.0 => registry.hub.docker.com/gitlab/gitlab-runner:alpine3.15-v16.1.0
Cmd: skopeo copy --all docker://registry.gitlab.com/gitlab-org/gitlab-runner:alpine3.15-v16.1.0 docker://registry.hub.docker.com/gitlab/gitlab-runner:ubi-fips-v16.1.0
2023/07/01 10:14:23 Copying image registry.gitlab.com/gitlab-org/gitlab-runner:ubi-fips-v16.1.0 => registry.hub.docker.com/gitlab/gitlab-runner:ubi-fips-v16.1.0
Cmd: skopeo copy --all docker://registry.gitlab.com/gitlab-org/gitlab-runner:ubi-fips-v16.1.0 docker://registry.hub.docker.com/gitlab/gitlab-runner:ubi-fips-v16.1.0
2023/07/01 10:14:23 Copying image registry.gitlab.com/gitlab-org/gitlab-runner:ubuntu-v16.1.0 => registry.hub.docker.com/gitlab/gitlab-runner:ubuntu-v16.1.0
Cmd: skopeo copy --all docker://registry.gitlab.com/gitlab-org/gitlab-runner:ubuntu-v16.1.0 docker://registry.hub.docker.com/gitlab/gitlab-runner:latest
2023/07/01 10:14:23 Copying image registry.gitlab.com/gitlab-org/gitlab-runner:latest => registry.hub.docker.com/gitlab/gitlab-runner:latest
Expected behavior
In the command above, we'd expect the image reported from the Cmd: line to always and without fail match the log line above it.
Proper Execution, no duplicates, no DATA RACE reported
$ go build -race
$ PUSH_TO_DOCKER_HUB=true ./sync-docker-images -version v16.1.0 --dry-run 2>&1 | grep -E '(WARNING|gitlab-runner:)'
2023/07/01 10:15:57 Copying image registry.gitlab.com/gitlab-org/gitlab-runner:alpine-v16.1.0 => registry.hub.docker.com/gitlab/gitlab-runner:alpine-v16.1.0
Cmd: skopeo copy --all docker://registry.gitlab.com/gitlab-org/gitlab-runner:alpine-v16.1.0 docker://registry.hub.docker.com/gitlab/gitlab-runner:alpine-v16.1.0
2023/07/01 10:15:57 Copying image registry.gitlab.com/gitlab-org/gitlab-runner:alpine3.12-v16.1.0 => registry.hub.docker.com/gitlab/gitlab-runner:alpine3.12-v16.1.0
Cmd: skopeo copy --all docker://registry.gitlab.com/gitlab-org/gitlab-runner:alpine3.12-v16.1.0 docker://registry.hub.docker.com/gitlab/gitlab-runner:alpine3.12-v16.1.0
2023/07/01 10:15:57 Copying image registry.gitlab.com/gitlab-org/gitlab-runner:alpine3.13-v16.1.0 => registry.hub.docker.com/gitlab/gitlab-runner:alpine3.13-v16.1.0
Cmd: skopeo copy --all docker://registry.gitlab.com/gitlab-org/gitlab-runner:alpine3.13-v16.1.0 docker://registry.hub.docker.com/gitlab/gitlab-runner:alpine3.13-v16.1.0
2023/07/01 10:15:57 Copying image registry.gitlab.com/gitlab-org/gitlab-runner:alpine3.14-v16.1.0 => registry.hub.docker.com/gitlab/gitlab-runner:alpine3.14-v16.1.0
Cmd: skopeo copy --all docker://registry.gitlab.com/gitlab-org/gitlab-runner:alpine3.14-v16.1.0 docker://registry.hub.docker.com/gitlab/gitlab-runner:alpine3.14-v16.1.0
2023/07/01 10:15:57 Copying image registry.gitlab.com/gitlab-org/gitlab-runner:alpine3.15-v16.1.0 => registry.hub.docker.com/gitlab/gitlab-runner:alpine3.15-v16.1.0
Cmd: skopeo copy --all docker://registry.gitlab.com/gitlab-org/gitlab-runner:alpine3.15-v16.1.0 docker://registry.hub.docker.com/gitlab/gitlab-runner:alpine3.15-v16.1.0
2023/07/01 10:15:57 Copying image registry.gitlab.com/gitlab-org/gitlab-runner:ubi-fips-v16.1.0 => registry.hub.docker.com/gitlab/gitlab-runner:ubi-fips-v16.1.0
Cmd: skopeo copy --all docker://registry.gitlab.com/gitlab-org/gitlab-runner:ubi-fips-v16.1.0 docker://registry.hub.docker.com/gitlab/gitlab-runner:ubi-fips-v16.1.0
2023/07/01 10:15:57 Copying image registry.gitlab.com/gitlab-org/gitlab-runner:ubuntu-v16.1.0 => registry.hub.docker.com/gitlab/gitlab-runner:ubuntu-v16.1.0
Cmd: skopeo copy --all docker://registry.gitlab.com/gitlab-org/gitlab-runner:ubuntu-v16.1.0 docker://registry.hub.docker.com/gitlab/gitlab-runner:ubuntu-v16.1.0
2023/07/01 10:15:57 Copying image registry.gitlab.com/gitlab-org/gitlab-runner:latest => registry.hub.docker.com/gitlab/gitlab-runner:latest
Cmd: skopeo copy --all docker://registry.gitlab.com/gitlab-org/gitlab-runner:latest docker://registry.hub.docker.com/gitlab/gitlab-runner:latest