Skip to content

GitLab Next

Closed
Created by vmignot@vmignot

Cannot upload cache to Azure Blob (getting : "413 The request body is too large and exceeds the maximum permissible limit.")

Summary

When using the [runners.cache.azure], and if the file is exceeding a certain size (not sure yet what size, I'm trying to find some elements in the documentation), the cache will fail with the following message:

Actual behavior

The cache might be failing because of Azure blob storage limitation.

Expected behavior

We might have to use "chunked upload" to upload the cache when exceeding a certain limit

Relevant logs and/or screenshots

Capture_d_écran_2020-10-20_à_15.21.14

(text:)

Saving cache
02:02
Creating cache e5c3ef9838d165f51d91c13e47efd7df65ee8eef...
node_modules/: found 82486 matching files and directories 
.yarn-cache/: found 76412 matching files and directories 
.cypress-cache/: found 39144 matching files and directories 
.node-sass-cache/: found 3 matching files and directories 
Uploading cache.zip to https://xxxxx.blob.core.windows.net/gitlab-cache/runner/J1jMMku-/project/103/e5c3ef9838d165f51d91c13e47efd7df65ee8eef 
FATAL: received: 413 The request body is too large and exceeds the maximum permissible limit. 
Failed to create cache
Uploading artifacts for successful job
00:39
Uploading artifacts...

Environment description

config.toml contents 
log_format = "json"
listen_address = "0.0.0.0:9252"

[session_server]
  session_timeout = 1800

[[runners]]
  name = "aks-runner"
  url = "https://xxxxx/"
  token = "xxxx"
  executor = "kubernetes"
  environment = ["K8S_AUTH_KUBECONFIG=/home/ops/.kube/config"]
  [runners.custom_build_dir]
  [runners.cache]
    Type = "azure"
    [runners.cache.azure]
      StorageDomain = "blob.core.windows.net"
      AccountName = "xxxxx"
      AccountKey = "xxxxx"
      ContainerName = "gitlab-cache"
  [runners.kubernetes]
    namespace = "gitlab-jobs"
    namespace_overwrite_allowed = ""
    cpu_request = "200m"    # https://docs.gitlab.com/runner/executors/kubernetes.html#overwriting-build-resources
    cpu_request_overwrite_max_allowed = "2"
    memory_request = "200Mi"
    memory_request_overwrite_max_allowed = "2Gi"
    privileged = false
    poll_timeout = 360
    host = ""
    bearer_token_overwrite_allowed = false
    image = ""
    pull_policy = "always"
    service_account = ""
    service_account_overwrite_allowed = ""
    pod_annotations_overwrite_allowed = ""
    image_pull_secrets = [ "acrpullsecret" ]
    cpu_limit = "2"
    cpu_limit_overwrite_max_allowed = "4"
    memory_limit = "2Gi"
    memory_limit_overwrite_max_allowed = "4Gi"
    [runners.kubernetes.pod_security_context]
    [runners.kubernetes.volumes]

Used GitLab Runner version

Running with gitlab/gitlab-runner:v13.4.1

Possible fixes

Implement chunk upload for Azure blob.

Edited by vmignot