Issue with artifact timeout on private runners.
Summary
My runners (only the ones I run on kubernetes, not the ones I have running on a standard vps with only docker) seems to have some issues with the artifacts and cache. The issues started after updating to 12.1.0 runners instead of earlier 12.0.1.
In the logs it seems like the job downloads the cache objects:
Checking cache for ***.gems...
Downloading cache.zip from http://rook-ceph-rgw-ceph-s3-store.rook-ceph.svc.cluster.local/gitlab-ci-cache/cache/project/***/***.gems
Successfully extracted cache
but then it behaves like if there is no cache (it's supposed to use the downloaded gem files as cache, but it re-installs all).
When the job is done, it tries to upload cache and fails 1-2 times and then produces a Created cache
message, the artifacts seems to upload fine (from reading the logs).
The step after (which uses the artifacts) the runner breaks after trying to fetch artifacts 3 times:
Downloading artifacts for build:jekyll (***)...
ERROR: Downloading artifacts from coordinator... error couldn't execute GET against https://gitlab.com/api/v4/jobs/***/artifacts: Get https://gitlab.com/api/v4/jobs/***/artifacts: dial tcp: i/o timeout id=*** token=***
WARNING: Retrying... error=invalid argument
ERROR: Downloading artifacts from coordinator... error couldn't execute GET against https://gitlab.com/api/v4/jobs/***/artifacts: Get https://gitlab.com/api/v4/jobs/***/artifacts: dial tcp: i/o timeout id=*** token=***
WARNING: Retrying... error=invalid argument
ERROR: Downloading artifacts from coordinator... error couldn't execute GET against https://gitlab.com/api/v4/jobs/***/artifacts: Get https://gitlab.com/api/v4/jobs/***/artifacts: dial tcp: i/o timeout id=*** token=***
FATAL: invalid argument
Ive ran the pipeline about 5 times today, and it keeps on doing it. I tried to clean up my ceph cluster to make sure it had nothing to do with that, and it doesn't. Also, my other runners uses the same cache storage (ceph) and it works just fine from there.
Steps to reproduce
Not sure, please contact me if you wish to have access to the repository to test (if you are a GitLab employee that is, hehe...).
What is the current bug behavior?
Artifacts can not be fetched from the dependency.
What is the expected correct behavior?
That the artifacts downloads fine.
Output of checks
This bug happens on GitLab.com