Azure cache incorrect authentication information
Hello, I'm having the issue with gitlab runner for some time now, and I'm not able to resove it follwing official doc or any issue conversation stream. Gitlab helm version: 0.84.0 Gitlab runner version: v18.7.0
I tried to configure gitlab runner using acount name and account key from azure storage account which resulted in
FATAL: blob (key "project/3069/sonar-cache-main-protected") (code=Unknown): PUT https://[MASKED].blob.core.windows.net/gitlab-runner-cache/project/3069/sonar-cache-main-protected
--------------------------------------------------------------------------------
RESPONSE 400: 400 Authentication information is not given in the correct format. Check the value of Authorization header.
ERROR CODE: InvalidAuthenticationInfo
--------------------------------------------------------------------------------
<?xml version="1.0" encoding="utf-8"?>
<Error><Code>InvalidAuthenticationInfo</Code><Message>Authentication information is not given in the correct format. Check the value of Authorization header.
RequestId:76e2a57b-501e-00d2-469c-75d9f1000000
Time:2025-12-25T12:49:44.0841894Z</Message></Error>
--------------------------------------------------------------------------------I have double check that account name and key are correctly parset in config toml file which was the case (key has the same value as one on azure stora account) I have use same credentials to connect to storage account which worked correctly I have network access to storage account from k8s
This is my deployment
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: gitlab-runner
namespace: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: infra
sources:
- repoURL: https://gitlab.xxx.xxx/xxx/devops.git
targetRevision: HEAD
path: argocd/aks-eastus-xxx-dev/deployments/infrastructure/gitlab-runner
- repoURL: http://charts.gitlab.io/
chart: gitlab-runner
targetRevision: 0.84.0
helm:
values: |
replicas: 3
extraEnv:
FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY: "true"
gitlabUrl: https://gitlab.xxx.xxx/
runnerRegistrationToken: ""
concurrent: 10
rbac:
create: true
rules:
- resources: ["events"]
verbs: ["list", "watch"]
- resources: ["pods"]
verbs: ["create","delete","get", "list", "watch"]
- apiGroups: [""]
resources: ["pods/attach","pods/exec"]
verbs: ["get","create","patch","delete"]
- apiGroups: [""]
resources: ["pods/log"]
verbs: ["get","list"]
- resources: ["secrets"]
verbs: ["create","delete","get","update"]
- resources: ["serviceaccounts"]
verbs: ["get"]
- resources: ["services"]
verbs: ["create","get"]
# Schedule runner manager pods on specific nodes
nodeSelector:
purpose: runners
kubernetes.azure.com/scalesetpriority: spot
tolerations:
- key: purpose
operator: Equal
value: runners
effect: NoSchedule
- key: kubernetes.azure.com/scalesetpriority
operator: Equal
value: spot
effect: NoSchedule
# Resource limits for runner manager
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 100m
memory: 128Mi
runners:
secret: gitlab-runner-secret
cache:
secretName: gitlab-runner-cache
config: |
[[runners]]
environment = ["FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY=true"]
request_concurrency = 4
[runners.cache]
Type = "azure"
Shared = true
[runners.cache.azure]
ContainerName = "gitlab-runner-cache"
[runners.kubernetes]
image = "ubuntu:24.04"
privileged = true
namespace = "{{.Release.Namespace}}"
cpu_limit = "1"
memory_limit = "3Gi"
pull_policy = ["if-not-present"]
[[runners.kubernetes.volumes.empty_dir]]
name = "docker-certs"
mount_path = "/certs/client"
medium = "Memory"
# Schedule job pods on the same node pool
[runners.kubernetes.node_selector]
"purpose" = "runners"
"kubernetes.azure.com/scalesetpriority" = "spot"
[runners.kubernetes.node_tolerations]
"purpose=runners" = "NoSchedule"
"kubernetes.azure.com/scalesetpriority=spot" = "NoSchedule"
destination:
server: https://kubernetes.default.svc
namespace: gitlab-runner
syncPolicy:
syncOptions:
- CreateNamespace=true
- ServerSideApply=true
automated:
prune: true
selfHeal: true
In [runners.cache.azure] account key and name are populated from secret
runners:
cache:
secretName: gitlab-runner-cacheAnd this works correclty
I tried to switch to managed identity, i have created everything needed assigned correct role and confirmed access, tried following approach for managed identity
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: gitlab-runner
namespace: argocd
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
project: infra
sources:
- repoURL: https://gitlab.xxx.xxx/xxx/devops.git
targetRevision: HEAD
path: argocd/aks-eastus-xxx-dev/deployments/infrastructure/gitlab-runner
- repoURL: http://charts.gitlab.io/
chart: gitlab-runner
targetRevision: 0.84.0
helm:
values: |
replicas: 3
extraEnv:
FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY: "true"
gitlabUrl: https://gitlab.xxx.xxx/
runnerRegistrationToken: ""
concurrent: 10
serviceAccount:
name: "gitlab-runner"
annotations:
azure.workload.identity/client-id: xxx
podLabels:
azure.workload.identity/use: "true"
rbac:
create: true
rules:
- resources: ["events"]
verbs: ["list", "watch"]
- resources: ["pods"]
verbs: ["create","delete","get", "list", "watch"]
- apiGroups: [""]
resources: ["pods/attach","pods/exec"]
verbs: ["get","create","patch","delete"]
- apiGroups: [""]
resources: ["pods/log"]
verbs: ["get","list"]
- resources: ["secrets"]
verbs: ["create","delete","get","update"]
- resources: ["serviceaccounts"]
verbs: ["get"]
- resources: ["services"]
verbs: ["create","get"]
# Schedule runner manager pods on specific nodes
nodeSelector:
purpose: runners
kubernetes.azure.com/scalesetpriority: spot
tolerations:
- key: purpose
operator: Equal
value: runners
effect: NoSchedule
- key: kubernetes.azure.com/scalesetpriority
operator: Equal
value: spot
effect: NoSchedule
# Resource limits for runner manager
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 100m
memory: 128Mi
runners:
secret: gitlab-runner-secret
config: |
[[runners]]
environment = [
"FF_USE_LEGACY_KUBERNETES_EXECUTION_STRATEGY=true"
]
request_concurrency = 4
[runners.cache]
Type = "azure"
Shared = true
[runners.cache.azure]
AccountName = "astraexpipelinecachedev"
ContainerName = "gitlab-runner-cache"
[runners.kubernetes]
image = "ubuntu:24.04"
privileged = true
namespace = "{{.Release.Namespace}}"
cpu_limit = "1"
memory_limit = "3Gi"
pull_policy = ["if-not-present"]
service_account = "gitlab-runner"
[runners.kubernetes.pod_labels]
"azure.workload.identity/use" = "true"
[[runners.kubernetes.volumes.empty_dir]]
name = "docker-certs"
mount_path = "/certs/client"
medium = "Memory"
# Schedule job pods on the same node pool
[runners.kubernetes.node_selector]
"purpose" = "runners"
"kubernetes.azure.com/scalesetpriority" = "spot"
[runners.kubernetes.node_tolerations]
"purpose=runners" = "NoSchedule"
"kubernetes.azure.com/scalesetpriority=spot" = "NoSchedule"
destination:
server: https://kubernetes.default.svc
namespace: gitlab-runner
syncPolicy:
syncOptions:
- CreateNamespace=true
- ServerSideApply=true
automated:
prune: true
selfHeal: trueAgain I have same error with managed identities approach as the account name and key, same error. Any idea is this related to particular version of runner or I'm doing it wrong (I have followed documentation)