Runner won't upload artifacts to Azure object storage

Summary

I have just enabled Azure object storage for our GitLab instance using the consolidated configuration. Config snippet attached. When running a simple CI/CD pipeline, I get the following error:

WARNING: Uploading artifacts to coordinator... failed id=55 responseStatus=500 Internal Server Error status=500 Internal Server Error token=2P2x68Wq
WARNING: Retrying...
WARNING: Uploading artifacts to coordinator... failed id=55 responseStatus=500 Internal Server Error status=500 Internal Server Error token=2P2x68Wq
WARNING: Retrying...
WARNING: Uploading artifacts to coordinator... failed id=55 responseStatus=500 Internal Server Error status=500 Internal Server Error token=2P2x68Wq
FATAL: invalid argument
ERROR: Job failed: exit code 1

This pipeline worked before I enabled Azure object storage, and logs below definitely point to Azure storage being involved in the problem. The pipeline seems to fail after receiving 400 An HTTP header that's mandatory for this request is not specified from a PUT request to Azure object storage.

Steps to reproduce

I have a very simple repo that publishes to GitLab Pages. When running the pipeline, it fails as above.

.gitlab-ci.yml

pages:
  stage: deploy
  script:
  - mkdir .public
  - cp -r * .public
  - mv .public public
  artifacts:
    paths:
    - public
  only:
  - master

Actual behavior

Job fails with 500 Internal Server Error as above

Expected behavior

Job completes successfully and updates the Pages page

Relevant logs and/or screenshots

job log

Running with gitlab-runner 11.8.0 (4745a6f3)
  on azae-gitlabce-01 7ead9ceb
Using Docker executor with image alpine:latest ...
Pulling docker image alpine:latest ...
Using docker image sha256:a24bb4013296f61e89ba57005a7b3e52274d8edd3ae2077d04395f806b63d83e for alpine:latest ...
Running on runner-7ead9ceb-project-69-concurrent-0 via azae-gitlabce-01...
00:03
Fetching changes...
00:02
Removing public/
HEAD is now at f39ee9f Update index.html
Checking out f39ee9f7 as master...
Skipping Git submodules setup
$ mkdir .public
00:02
$ cp -r * .public
$ mv .public public
Uploading artifacts...
00:07
public: found 3 matching files                     
WARNING: Uploading artifacts to coordinator... failed  id=56 responseStatus=500 Internal Server Error status=500 Internal Server Error token=6ZeihmpX
WARNING: Retrying...                               
WARNING: Uploading artifacts to coordinator... failed  id=56 responseStatus=500 Internal Server Error status=500 Internal Server Error token=6ZeihmpX
WARNING: Retrying...                               
WARNING: Uploading artifacts to coordinator... failed  id=56 responseStatus=500 Internal Server Error status=500 Internal Server Error token=6ZeihmpX
FATAL: invalid argument                            
ERROR: Job failed: exit code 1

gitlab-workhorse/current

==> /var/log/gitlab/gitlab-workhorse/current <==
{"copied_bytes":1391,"correlation_id":"XgaEDcJI8K2","is_local":false,"is_multipart":false,"is_remote":true,"level":"info","msg":"saved file","remote_id":"1599451338-61797-0003-1559-be13879aeb90b0f273bc07dbe16e69ca","temp_file_prefix":"artifacts.zip","time":"2020-09-07T16:02:18+12:00","use_s3_client":false}
{"correlation_id":"XgaEDcJI8K2","error":"handleFileUploads: extract files from multipart: persisting multipart file: PUT request https://accountnamechanged.blob.core.windows.net/artifacts/tmp/uploads/1599451338-61797-4358-1559-be13879aeb95886273bc07dbe16e69ca?sp=c\u0026sv=2016-05-31\u0026sr=b\u0026se=2020-09-07T08%3A17%3A18Z\u0026spr=https\u0026sig=DsjHFxse%2Br3NrLuh1NCw5A1FlXXtFicpL8tk9fw%3D returned: 400 An HTTP header that's mandatory for this request is not specified.","level":"error","method":"POST","msg":"error","time":"2020-09-07T16:02:18+12:00","uri":"/api/v4/jobs/55/artifacts?artifact_format=zip\u0026artifact_type=archive"}
{"correlation_id":"XgaEDcJI8K2","duration_ms":61,"host":"gitlab.mycompany.com","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","status":500,"system":"http","time":"2020-09-07T16:02:18+12:00","uri":"/api/v4/jobs/55/artifacts?artifact_format=zip\u0026artifact_type=archive","user_agent":"gitlab-runner 11.8.0 (11-8-stable; go1.8.7; linux/amd64)","written_bytes":22}

grep -r 'status":500' /var/log/gitlab

gitlab-workhorse/current:{"correlation_id":"iLTzvde5383","duration_ms":1568,"host":"gitlab.mycompany.com","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","status":500,"system":"http","time":"2020-09-08T09:36:37+12:00","uri":"/api/v4/jobs/56/artifacts?artifact_format=zip\u0026artifact_type=archive","user_agent":"gitlab-runner 11.8.0 (11-8-stable; go1.8.7; linux/amd64)","written_bytes":22}
gitlab-workhorse/current:{"correlation_id":"MSB3vM1WYn9","duration_ms":49,"host":"gitlab.mycompany.com","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","status":500,"system":"http","time":"2020-09-08T09:36:38+12:00","uri":"/api/v4/jobs/56/artifacts?artifact_format=zip\u0026artifact_type=archive","user_agent":"gitlab-runner 11.8.0 (11-8-stable; go1.8.7; linux/amd64)","written_bytes":22}
gitlab-workhorse/current:{"correlation_id":"Sk8T5YLrjR6","duration_ms":408,"host":"gitlab.mycompany.com","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","status":500,"system":"http","time":"2020-09-08T09:36:40+12:00","uri":"/api/v4/jobs/56/artifacts?artifact_format=zip\u0026artifact_type=archive","user_agent":"gitlab-runner 11.8.0 (11-8-stable; go1.8.7; linux/amd64)","written_bytes":22}

sudo grep -r 'iLTzvde5383' /var/log/gitlab

/var/log/gitlab/gitlab-workhorse/current:{"copied_bytes":1391,"correlation_id":"iLTzvde5383","is_local":false,"is_multipart":false,"is_remote":true,"level":"info","msg":"saved file","remote_id":"1599514596-93903-0001-2519-896148178cda5591a9577ae2af0f0ed7","temp_file_prefix":"artifacts.zip","time":"2020-09-08T09:36:37+12:00","use_s3_client":false}
/var/log/gitlab/gitlab-workhorse/current:{"correlation_id":"iLTzvde5383","error":"handleFileUploads: extract files from multipart: persisting multipart file: PUT request https://accountnamechanged.blob.core.windows.net/artifacts/tmp/uploads/1599514596-93903-0001-2519-896148178cda5591a9577ae2af0f0ed7?sp=c\u0026sv=2016-05-31\u0026sr=b\u0026se=2020-09-08T01%3A51%3A37Z\u0026spr=https\u0026sig=q9DE%2FMQpdy%2BnfTgeby7yQ6FC0IWiCY50oUz3YC2m0ow%3D returned: 400 An HTTP header that's mandatory for this request is not specified.","level":"error","method":"POST","msg":"error","time":"2020-09-08T09:36:37+12:00","uri":"/api/v4/jobs/56/artifacts?artifact_format=zip\u0026artifact_type=archive"}
/var/log/gitlab/gitlab-workhorse/current:{"correlation_id":"iLTzvde5383","duration_ms":1568,"host":"gitlab.mycompany.com","level":"info","method":"POST","msg":"access","proto":"HTTP/1.1","referrer":"","remote_addr":"127.0.0.1:0","remote_ip":"127.0.0.1","status":500,"system":"http","time":"2020-09-08T09:36:37+12:00","uri":"/api/v4/jobs/56/artifacts?artifact_format=zip\u0026artifact_type=archive","user_agent":"gitlab-runner 11.8.0 (11-8-stable; go1.8.7; linux/amd64)","written_bytes":22}

Environment description

Running self-hosted GitLab EE 13.3.5

object storage config

################
# Object Store #
################
# Consolidated object storage configuration
gitlab_rails['object_store']['enabled'] = true
gitlab_rails['object_store']['proxy_download'] = false
gitlab_rails['object_store']['connection'] = {
  'provider' => 'AzureRM',
  'azure_storage_account_name' => 'accountnamechanged',
  'azure_storage_access_key' => 'keyremoved'
}
gitlab_rails['object_store']['objects']['artifacts']['bucket'] = 'artifacts'
gitlab_rails['object_store']['objects']['external_diffs']['bucket'] = 'external-diffs'
gitlab_rails['object_store']['objects']['lfs']['bucket'] = 'lfs-objects'
gitlab_rails['object_store']['objects']['uploads']['bucket'] = 'uploads'
gitlab_rails['object_store']['objects']['packages']['bucket'] = 'packages'
gitlab_rails['object_store']['objects']['dependency_proxy']['bucket'] = 'dependency-proxy'
gitlab_rails['object_store']['objects']['terraform_state']['bucket'] = 'terraform-state'