PackageCloud reports a package does not exists when it definitely exists
Over in https://gitlab.com/gitlab-org/ci-cd/runner-tools/gitlab-runner-package-tests we have aproject to tests the rpm and deb runner and runner-helper packages we create. Lately we've seen a number of failures, e.g.
- https://gitlab.com/gitlab-org/ci-cd/runner-tools/gitlab-runner-package-tests/-/jobs/11130604174
- https://gitlab.com/gitlab-org/ci-cd/runner-tools/gitlab-runner-package-tests/-/pipelines/2003673822
- https://gitlab.com/gitlab-org/ci-cd/runner-tools/gitlab-runner-package-tests/-/pipelines/2010112048
The gist of the failures is this:
...
#8 4.957 Detected operating system as debian/11.
...
#8 13.50 + apt install -y --allow-unauthenticated gitlab-runner=18.4.0~pre.55.g46e05432-1 gitlab-runner-helper-images=18.4.0~pre.55.g46e05432-1
#8 13.51
#8 13.51 WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
#8 13.51
#8 13.51 Reading package lists...
#8 13.92 Building dependency tree...
#8 14.03 Reading state information...
#8 14.09 E: Version '18.4.0~pre.55.g46e05432-1' for 'gitlab-runner' was not found
#8 14.09 E: Version '18.4.0~pre.55.g46e05432-1' for 'gitlab-runner-helper-images' was not found
...
But if one navigates to packagecloud and searches for those packages, they exist:
- https://packages.gitlab.com/runner/unstable/packages/debian/bullseye/gitlab-runner_18.4.0~pre.55.g46e05432-1_amd64.deb
- https://packages.gitlab.com/runner/unstable/packages/debian/bullseye/gitlab-runner-helper-images_18.4.0~pre.55.g46e05432-1_all.deb
It's worth noting that so far we have only seen this in the unstable repository, which is only used for MRs/tests. We have not seen it in the public-facing gitlab-runner repository, so customers have not been impacted yet.
Reproducing
docker run -it --rm debian:bullseye
apt install -y curl wget // yes, both.
curl -s https://packages.gitlab.com/install/repositories/runner/unstable/script.deb.sh | bash
// this will fail as in the pipelines
apt install -y --allow-unauthenticated gitlab-runner=18.4.0~pre.55.g46e05432-1 gitlab-runner-helper-images=18.4.0~pre.55.g46e05432-1
But the packages are there! This succeeds:
curl -Lo gitlab-runner_18.4.0~pre.55.g46e05432-1_amd64.deb https://packages.gitlab.com/runner/unstable/packages/debian/bullseye/gitlab-runner_18.4.0~pre.55.g46e05432-1_amd64.deb/download.deb
curl -Lo gitlab-runner-helper-images_18.4.0~pre.55.g46e05432-1_all.deb https://packages.gitlab.com/runner/unstable/packages/debian/bullseye/gitlab-runner-helper-images_18.4.0~pre.55.g46e05432-1_all.deb/download.deb
apt install -y ./gitlab-runner_18.4.0~pre.55.g46e05432-1_amd64.deb ./gitlab-runner-helper-images_18.4.0~pre.55.g46e05432-1_all.deb
So, the packages exist and are valid (they can be installed and run), but they cannot be installed via the package manager. apt-cache madison gitlab-runner will show all the gitlab-runner packages available for installation; note that ~pre.55.g46e05432-1 does not appear there.
