gitlab-ce pod in kubernetes can't change permission to volume (NFS)

Summary

Can't start gitlab-ce in Kubernetes environment because it's not able to modify a volume's permission and gitlab-ce just shuts down.

Steps to reproduce

1.- Setup a configured Kubernetes cluster in Ubuntu
2.- Install an NFS Server to store Kubernetes volumes
3.- Setup the Persistent Volume and Persistent Volume Claim for the gitlab data volume
4.- Setup Gitlab deployment

Example Project

pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: gitlab-data
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1G
pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: gitlab-data
spec:
  capacity:
    storage: 1G
  accessModes:
    - ReadWriteMany
  claimRef:
    namespace: default
    name: gitlab-data
  mountOptions:
    - nfsvers=4.1
  nfs:
    path: /volumes/gitlab/data
    server: nfsserver.pe
    readOnly: false
deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: gitlab
  labels:
    app: gitlab
spec:
  replicas: 1
  selector:
    matchLabels:
      app: gitlab
  template:
    metadata:
      labels:
        app: gitlab
    spec:
      terminationGracePeriodSeconds: 180
      containers:
        - name: gitlab
          image: gitlab/gitlab-ce
          imagePullPolicy: Always
          ports:
            - containerPort: 22
            - containerPort: 80
            - containerPort: 443
          volumeMounts:
            - mountPath: /var/opt/gitlab
              name: data
      volumes:
        - name: data
          persistentVolumeClaim:
            claimName: gitlab-data

What is the current bug behavior?

GitLab does not work with errors.

What is the expected correct behavior?

GitLab starts properly.

Relevant logs and/or screenshots

Expand for logs of gitlab-ce container
Installing Cookbook Gems:
Compiling Cookbooks...
Recipe: gitlab::default
  * directory[/etc/gitlab] action create
    - change mode from '0755' to '0775'
  Converging 272 resources
  * directory[/etc/gitlab] action create (up to date)
  * directory[Create /var/opt/gitlab] action create
================================================================================
Error executing action `create` on resource 'directory[Create /var/opt/gitlab]'
================================================================================

Errno::EPERM
------------
Operation not permitted @ apply2files - /var/opt/gitlab

Resource Declaration:
---------------------
# In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb

 41: directory "Create /var/opt/gitlab" do
 42:   path "/var/opt/gitlab"
 43:   owner "root"
 44:   group "root"
 45:   mode "0755"
 46:   recursive true
 47:   action :create
 48: end
 49: 

Compiled Resource:
------------------
# Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb:41:in `from_file'

directory("Create /var/opt/gitlab") do
  action [:create]
  default_guard_interpreter :default
  declared_type :directory
  cookbook_name "gitlab"
  recipe_name "default"
  owner "root"
  group "root"
  mode "0755"
  path "/var/opt/gitlab"
  recursive true
end

System Info:
------------
chef_version=14.13.11
platform=ubuntu
platform_version=16.04
ruby=ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/chef-client
executable=/opt/gitlab/embedded/bin/chef-client

Running handlers: There was an error running gitlab-ctl reconfigure:

directory[Create /var/opt/gitlab] (gitlab::default line 41) had an error: Errno::EPERM: Operation not permitted @ apply2files - /var/opt/gitlab

Running handlers complete Chef Client failed. 1 resources updated in 05 seconds

Edited Dec 11, 2019 by Cristian
Assignee Loading
Time tracking Loading