backup_multipart_chunk_size ignored on Openstack Swift upload
Summary
I've configured the Backup upload connection to Openstack Swift (private Openstack Cluster) with Keystone v3, I've tested a simple backup without pages and registry and worked smooth (150MB of backup). After I've tried a full backup, involving also the registry and the pages (10GB) and it failed because Swift has a maximum file limit of 5GB (413 Entity too large), so I've configured the backup_multipart_chunk_size to 1GB (1048576000) but it fails always with the same error.
Steps to reproduce
Configure the Openstack Swift upload connection with a huge backup (more than 5GB) and backup_multipart_chunk_size settled, do a backup and check the
What is the current bug behavior?
The backup fails with a 413 Entity Too Large error
What is the expected correct behavior?
The backup correctly uploaded
Relevant logs
Relevant logs
Uploading backup archive to remote storage gitlab ... rake aborted! Excon::Error::RequestEntityTooLarge: Expected(201) <=> Actual(413 Request Entity Too Large) excon.error.response :body => "Your request is too large." :cookies => [ ] :headers => { "Content-Length" => "26" "Content-Type" => "text/plain" "Date" => "Wed, 30 Jan 2019 14:43:14 GMT" "X-Openstack-Request-Id" => "" "X-Trans-Id" => "" } :host => "http://controller" :local_address => "localip" :local_port => 44780 :path => "/v1/AUTH_bf232309890b474281eb95a472445946/gitlab/1548856514_2019_01_30_11.7.0-ee_gitlab_backup.tar" :port => 8080 :reason_phrase => "Request Entity Too Large" :remote_ip => "controller-ip" :status => 413 :status_line => "HTTP/1.1 413 Request Entity Too Large\r\n" /opt/gitlab/embedded/service/gitlab-rails/lib/backup/manager.rb:50:in `upload' /opt/gitlab/embedded/service/gitlab-rails/lib/backup/manager.rb:35:in `block in pack' /opt/gitlab/embedded/service/gitlab-rails/lib/backup/manager.rb:19:in `chdir' /opt/gitlab/embedded/service/gitlab-rails/lib/backup/manager.rb:19:in `pack' /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/backup.rake:20:in `block (3 levels) in ' /opt/gitlab/embedded/bin/bundle:23:in `load' /opt/gitlab/embedded/bin/bundle:23:in `'
Details of package version
Provide the package version installation details
rpm -qa | grep 'gitlab' => gitlab-ee-11.7.0-ee.0.el7.x86_64
Environment details
- Operating System:
CentOS 7
- Installation Target, remove incorrect
- VM: Openstack
- Installation Type, remove incorrect values:
- New Installation
- Is there any other software running on the machine:
No
- Is this a single or multiple node installation? Single
- Resources
- CPU:
4 cores
- Memory total:
8gb of ram
- CPU:
Configuration details
Provide the relevant sections of `/etc/gitlab/gitlab.rb`
``ruby gitlab_rails['backup_keep_time'] = 604800gitlab_rails['backup_upload_connection'] = { 'provider' => 'Openstack', 'openstack_auth_url' => 'http://keystone(redacted):5000/v3/auth/tokens', # workaround for keystone v3 https://github.com/fog/fog/pull/3506 'openstack_username' => 'user(redacted)', 'openstack_api_key' => 'password(redacted)', 'openstack_project_name' => 'project(redacted)', 'openstack_domain_id' => 'domain(redacted)' # Same domain of the project } gitlab_rails['backup_upload_remote_directory'] = 'gitlab' gitlab_rails['backup_multipart_chunk_size'] = 1048576000 ``