Skip to content

Gitlab Minio and Gitaly FailedMount NFS

I have the issue with Mount to AWS NFS: Output: mount.nfs: Connection timed out I checked firewall and allow TCP/UPD ports: 111, 2049, 20048 also, and tried mount directly and it ok.

Here is error log: Minio:

 Warning  FailedMount  9m44s (x2377 over 3d17h)  kubelet, ip-10-102-8-21.ap-southeast-1.compute.internal  Unable to mount volumes for pod "gitlab-minio-78cc9b68b7-jfv42_gitlab(93661dc1-1659-419d-aa4d-34a699687ed9)": timeout expired waiting for volumes to attach or mount for pod "gitlab"/"gitlab-minio-78cc9b68b7-jfv42". list of unmounted volumes=[export]. list of unattached volumes=[podinfo export minio-configuration minio-server-config default-token-qmgdm]
  Warning  FailedMount  3m (x1285 over 3d17h)     kubelet, ip-10-102-8-21.ap-southeast-1.compute.internal  MountVolume.SetUp failed for volume "gitlab-minio" : mount failed: exit status 32
Mounting command: mount
Mounting arguments: -t nfs fs-921e2ed3.efs.ap-southeast-1.amazonaws.com:/gitlab/minio /var/lib/kubelet/pods/93661dc1-1659-419d-aa4d-34a699687ed9/volumes/kubernetes.io~nfs/gitlab-minio
Output: mount.nfs: Connection timed out

Gitaly:

  Warning  FailedMount  9m44s (x2379 over 3d17h)  kubelet, ip-10-102-8-21.ap-southeast-1.compute.internal  Unable to mount volumes for pod "gitlab-gitaly-0_gitlab(1fcd46b6-35ce-4a63-9d13-1cc5af365f82)": timeout expired waiting for volumes to attach or mount for pod "gitlab"/"gitlab-gitaly-0". list of unmounted volumes=[repo-data]. list of unattached volumes=[repo-data gitaly-config gitaly-secrets init-gitaly-secrets etc-ssl-certs default-token-qmgdm]
  Warning  FailedMount  3m59s (x1285 over 3d17h)  kubelet, ip-10-102-8-21.ap-southeast-1.compute.internal  MountVolume.SetUp failed for volume "repo-data-gitlab-gitaly-0" : mount failed: exit status 32
Mounting command: mount
Mounting arguments: -t nfs fs-921e2ed3.efs.ap-southeast-1.amazonaws.com:/gitlab/gitaly /var/lib/kubelet/pods/1fcd46b6-35ce-4a63-9d13-1cc5af365f82/volumes/kubernetes.io~nfs/repo-data-gitlab-gitaly-0
Output: mount.nfs: Connection timed out

image

Versions

  • Chart: 4.0.5 - master
  • Platform:
    • Self-hosted: (OpenShift | Minikube | Rancher RKE | ?): RKE
  • Kubernetes: (kubectl version)
    • Client: v1.16.8
    • Server: v1.15.11
  • Helm: (helm version)
    • Client: v2.14.3
    • Server: v2.14.3

Relevant logs

(Please provide any relevant log snippets you have collected, using code blocks (```) to format)