gitlab-ce pod in kubernetes can't change permission to volume (NFS)
Summary
Can't start gitlab-ce in Kubernetes environment because it's not able to modify a volume's permission and gitlab-ce just shuts down.
Steps to reproduce
1.- Setup a configured Kubernetes cluster in Ubuntu
2.- Install an NFS Server to store Kubernetes volumes
3.- Setup the Persistent Volume and Persistent Volume Claim for the gitlab data volume
4.- Setup Gitlab deployment
Example Project
pvc.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: gitlab-data
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1G
pv.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: gitlab-data
spec:
capacity:
storage: 1G
accessModes:
- ReadWriteMany
claimRef:
namespace: default
name: gitlab-data
mountOptions:
- nfsvers=4.1
nfs:
path: /volumes/gitlab/data
server: nfsserver.pe
readOnly: false
deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: gitlab
labels:
app: gitlab
spec:
replicas: 1
selector:
matchLabels:
app: gitlab
template:
metadata:
labels:
app: gitlab
spec:
terminationGracePeriodSeconds: 180
containers:
- name: gitlab
image: gitlab/gitlab-ce
imagePullPolicy: Always
ports:
- containerPort: 22
- containerPort: 80
- containerPort: 443
volumeMounts:
- mountPath: /var/opt/gitlab
name: data
volumes:
- name: data
persistentVolumeClaim:
claimName: gitlab-data
What is the current bug behavior?
GitLab does not work with errors.
What is the expected correct behavior?
GitLab starts properly.
Relevant logs and/or screenshots
Expand for logs of gitlab-ce container
Installing Cookbook Gems:
Compiling Cookbooks...
Recipe: gitlab::default
* directory[/etc/gitlab] action create
- change mode from '0755' to '0775'
Converging 272 resources
* directory[/etc/gitlab] action create (up to date)
* directory[Create /var/opt/gitlab] action create
================================================================================
Error executing action `create` on resource 'directory[Create /var/opt/gitlab]'
================================================================================
Errno::EPERM
------------
Operation not permitted @ apply2files - /var/opt/gitlab
Resource Declaration:
---------------------
# In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb
41: directory "Create /var/opt/gitlab" do
42: path "/var/opt/gitlab"
43: owner "root"
44: group "root"
45: mode "0755"
46: recursive true
47: action :create
48: end
49:
Compiled Resource:
------------------
# Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/default.rb:41:in `from_file'
directory("Create /var/opt/gitlab") do
action [:create]
default_guard_interpreter :default
declared_type :directory
cookbook_name "gitlab"
recipe_name "default"
owner "root"
group "root"
mode "0755"
path "/var/opt/gitlab"
recursive true
end
System Info:
------------
chef_version=14.13.11
platform=ubuntu
platform_version=16.04
ruby=ruby 2.6.3p62 (2019-04-16 revision 67580) [x86_64-linux]
program_name=/opt/gitlab/embedded/bin/chef-client
executable=/opt/gitlab/embedded/bin/chef-client
Running handlers:
There was an error running gitlab-ctl reconfigure:
directory[Create /var/opt/gitlab] (gitlab::default line 41) had an error: Errno::EPERM: Operation not permitted @ apply2files - /var/opt/gitlab
Running handlers complete
Chef Client failed. 1 resources updated in 05 seconds
Edited by Cristian