Skip to content

task-runner backup-utility : increase s3 multipart chunk size

https://gitlab.com/gitlab-org/build/CNG/blob/master/gitlab-task-runner/scripts/bin/backup-utility

Large uploads fail to be uploaded to the s3 backup bucket. This can be resoled by choosing larger chunks for multipart uplaods, eg --multipart-chunk-size-mb=128 or by disabling multipart alltogether --disable-multipart (not recommended)

ERROR: Parameter problem: Chunk size 15 MB results in more than 10000 chunks. Please increase --multipart-chunk-size-mb

And for some reason s3cmd also runs DeUnicodising on the binary file contents (and that's not because python-magic isn't installed), but that's not what's me preventing to get backups.

cc: @WarheadsSE @twk3