502 Bad Gateway when uploading artifacts - regression?
If you experience a problem with CI/CD on GitLab.com, please raise an > issue in https://gitlab.com/gitlab-com/support-forum/issues
That URL in the issue template is broken ^^^
And searching for "timeout" on https://gitlab.com/groups/gitlab-org/-/issues times out...
So I will create my issue here instead.
Using gitlab-runner v18.4.0 with GitLab SaaS.
I've seen a significant number of “Bad Gateway” errors when a job tries to upload artifacts, recently. This did not happen previously and seems to have become an issue in the last week or two:
WARNING: Uploading artifacts as "junit" to coordinator... 502 Bad Gateway correlation_id=416df81d02df41a696dfcd8d2ee8xxxx id=1166443xxxx responseStatus=502 Bad Gateway status=502 token=6b_yxxxxIt seems to happen on approximately 1% of jobs, which for us is about once or twice a day.
Uploading artifacts...
pytest_result.xml: found 1 matching artifact files and directories
Waiting before making the next call context=ratelimit-requester-gitlab-request duration=100ms method=POST url=https://gitlab.com/api/v4/jobs/11664430759/artifacts?artifact_format=gzip&artifact_type=junit
Waiting before making the next call context=ratelimit-requester-gitlab-request duration=165.921748ms method=POST url=https://gitlab.com/api/v4/jobs/11664430759/artifacts?artifact_format=gzip&artifact_type=junit
Waiting before making the next call context=ratelimit-requester-gitlab-request duration=130.965998ms method=POST url=https://gitlab.com/api/v4/jobs/11664430759/artifacts?artifact_format=gzip&artifact_type=junit
Waiting before making the next call context=ratelimit-requester-gitlab-request duration=369.052814ms method=POST url=https://gitlab.com/api/v4/jobs/11664430759/artifacts?artifact_format=gzip&artifact_type=junit
Waiting before making the next call context=ratelimit-requester-gitlab-request duration=1.12370186s method=POST url=https://gitlab.com/api/v4/jobs/11664430759/artifacts?artifact_format=gzip&artifact_type=junit
WARNING: Uploading artifacts as "junit" to coordinator... 502 Bad Gateway correlation_id=77e5ab77e1c244b4b1b514aae9320765 id=11664430759 responseStatus=502 Bad Gateway status=502 token=6b_yqo6v4
WARNING: Retrying... context=artifacts-uploader error=invalid argument
Waiting before making the next call context=ratelimit-requester-gitlab-request duration=100ms method=POST url=https://gitlab.com/api/v4/jobs/11664430759/artifacts?artifact_format=gzip&artifact_type=junit
Waiting before making the next call context=ratelimit-requester-gitlab-request duration=164.892156ms method=POST url=https://gitlab.com/api/v4/jobs/11664430759/artifacts?artifact_format=gzip&artifact_type=junit
Waiting before making the next call context=ratelimit-requester-gitlab-request duration=193.441029ms method=POST url=https://gitlab.com/api/v4/jobs/11664430759/artifacts?artifact_format=gzip&artifact_type=junit
Waiting before making the next call context=ratelimit-requester-gitlab-request duration=309.441607ms method=POST url=https://gitlab.com/api/v4/jobs/11664430759/artifacts?artifact_format=gzip&artifact_type=junit
Waiting before making the next call context=ratelimit-requester-gitlab-request duration=924.406696ms method=POST url=https://gitlab.com/api/v4/jobs/11664430759/artifacts?artifact_format=gzip&artifact_type=junit
WARNING: Uploading artifacts as "junit" to coordinator... 502 Bad Gateway correlation_id=4adc200ae4d249d0a4bf000aa62b3317 id=11664430759 responseStatus=502 Bad Gateway status=502 token=6b_yqo6v4
WARNING: Retrying... context=artifacts-uploader error=invalid argument
Waiting before making the next call context=ratelimit-requester-gitlab-request duration=100ms method=POST url=https://gitlab.com/api/v4/jobs/11664430759/artifacts?artifact_format=gzip&artifact_type=junit
Waiting before making the next call context=ratelimit-requester-gitlab-request duration=136.595577ms method=POST url=https://gitlab.com/api/v4/jobs/11664430759/artifacts?artifact_format=gzip&artifact_type=junit
Waiting before making the next call context=ratelimit-requester-gitlab-request duration=178.620988ms method=POST url=https://gitlab.com/api/v4/jobs/11664430759/artifacts?artifact_format=gzip&artifact_type=junit
Waiting before making the next call context=ratelimit-requester-gitlab-request duration=792.826142ms method=POST url=https://gitlab.com/api/v4/jobs/11664430759/artifacts?artifact_format=gzip&artifact_type=junit
Waiting before making the next call context=ratelimit-requester-gitlab-request duration=1.023020772s method=POST url=https://gitlab.com/api/v4/jobs/11664430759/artifacts?artifact_format=gzip&artifact_type=junit
WARNING: Uploading artifacts as "junit" to coordinator... 502 Bad Gateway correlation_id=416df81d02df41a696dfcd8d2ee80551 id=11664430759 responseStatus=502 Bad Gateway status=502 token=6b_yqo6v4
FATAL: invalid argument
Cleaning up project directory and file based variables 00:00
ERROR: Job failed: exit code 1