15.x GitLab with direct upload, fails to upload big job artifacts (> 20Gb) to AWS s3
After upgrade to v15.x large (21GiB) artifact file fails to upload to S3
Steps to reproduce:
- Install omnibus GitLab version 15.x or greater.
- Configure AWS s3 as a cloud storage for artifacts.
- Create
.gitlab-ci.ymlwith job artifacts more than 20Gb - Run pipeline
- Check
exceptions_json.log
Current findings:
- omnibus 14.10.4-ee - it works, I was able to upload job artifact 21G to S3 (runner 15.6.1)
- omnibus 15.4-ee - failed to upload job artifact 21G to S3 (runner 15.6.1)
- omnibus 15.5.4-ee - failed to upload job artifact 21G to S3 (runner 15.6.1)
Rack::Timeout::RequestTimeoutException”,“exception.message”:“Request ran for longer than 60000ms
copy_to’“,”app/uploaders/object_storage.rb:399:
Looks like we hit a limit on CarrierWave S3 specific copy/move operation