Can't persist storage var/opt - var/log

Summary

I'm unable to persist storage of the var/*/gitlab path into a kubernetes local cluster. If I comment out the storage monthPath for the /var/ folders the installation works perfectly fine. Similar to this issue #3432 but on kubernetes.

Steps to reproduce

CentOS 7 enviroment, rbac enable. Storage permission chown kubernetesUser:kubernetesUser /folder External Postgres database.

Deployment & Service yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
  creationTimestamp: null
  generation: 1
  labels:
    app: gitlab
  name: gitlab
  selfLink: /apis/extensions/v1beta1/namespaces/default/deployments/gitlab
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: gitlab
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: gitlab
    spec:
      containers: 
      - image: gitlab/gitlab-ce:11.10.4-ce.0
        # command: [ "/bin/bash", "-c", "--" ]
        # args: [ "while true; do sleep 30; done;" ]
        imagePullPolicy: Always
        name: gitlab
        ports:
        - containerPort: 22
          name: ssh
          protocol: TCP
        - containerPort: 80
          name: http
          protocol: TCP
        - containerPort: 443
          name: https
          protocol: TCP
        resources:
          requests:
            cpu: 500m
            memory: 1024Mi
          limits:
            cpu: 2000m
            memory: 3Gi
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - name: gitlab-volume
          mountPath: /etc/gitlab
          subPath: storage/etc/gitlab
        - name: gitlab-volume
          mountPath: /var/opt/gitlab
          subPath: storage/var/opt/gitlab
        - name: gitlab-volume
          mountPath: /var/log/gitlab
          subPath: storage/var/log/gitlab
      volumes:
      - name: gitlab-volume
        persistentVolumeClaim:
          claimName: gitlab-volume-claim
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
status: {}
---
apiVersion: v1
kind: Service
metadata:
  creationTimestamp: null
  labels:
    app: gitlab
  name: gitlab
  selfLink: /api/v1/namespaces/default/services/gitlab
spec:
  externalTrafficPolicy: Cluster
  ports:
  - name: ssh
    port: 22
    protocol: TCP
    targetPort: ssh
  - name: http
    port: 80
    protocol: TCP
    targetPort: http
  - name: https
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    app: gitlab
  sessionAffinity: None
  type: NodePort
status:
  loadBalancer: {}

What is the current bug behavior?

Permission denied @ apply2files - /var/opt/gitlab/.chef-.gitconfig20190808-25-1917fjo.gitconfig

What is the expected correct behavior?

Having no problem to persist storage on local host.

Relevant logs

Relevant logs
Recipe: gitlab::users
  * directory[/var/opt/gitlab] action create (up to date)
  * account[GitLab user and group] action create (up to date)
  * template[/var/opt/gitlab/.gitconfig] action create
    ================================================================================
    Error executing action `create` on resource 'template[/var/opt/gitlab/.gitconfig]'
    ================================================================================
    Errno::EACCES
    -------------
    Permission denied @ apply2files - /var/opt/gitlab/.gitconfig
    Resource Declaration:
    ---------------------
    # In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/users.rb
     40: template File.join(gitlab_home, ".gitconfig") do
     41:   source "gitconfig.erb"
     42:   owner gitlab_username
     43:   group gitlab_group
     44:   mode "0644"
     45:   variables(node['gitlab']['user'].to_hash)
     46: end
     47:
    Compiled Resource:
    ------------------
    # Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/users.rb:40:in `from_file'
    template("/var/opt/gitlab/.gitconfig") do
      action [:create]
      default_guard_interpreter :default
      source "gitconfig.erb"
      variables {"username"=>"git", "group"=>"git", "uid"=>nil, "gid"=>nil, "shell"=>"/bin/sh", "home"=>"/var/opt/gitlab", "git_user_name"=>"GitLab", "git_user_email"=>"gitlab@gitlab.gyf.com"}
      declared_type :template
      cookbook_name "gitlab"
      recipe_name "users"
      mode "0644"
      owner "git"
      group "git"
      path "/var/opt/gitlab/.gitconfig"
      verifications []
    end
    System Info:
    ------------
    chef_version=13.6.4
    platform=ubuntu
    platform_version=16.04
    ruby=ruby 2.5.3p105 (2018-10-18 revision 65156) [x86_64-linux]
    program_name=/opt/gitlab/embedded/bin/chef-client
    executable=/opt/gitlab/embedded/bin/chef-client
Running handlers:
There was an error running gitlab-ctl reconfigure:

template[/var/opt/gitlab/.gitconfig] (gitlab::users line 40) had an error: Errno::EACCES: Permission denied @ apply2files - /var/opt/gitlab/.gitconfig

Running handlers complete Chef Client failed. 2 resources updated in 05 seconds

Details of package version

Provide the package version installation details
gitlab/gitlab-ce:11.10.4-ce.0

Configuration details

Provide the relevant sections of `/etc/gitlab/gitlab.rb`
Nothing relevant to permissions.

Workaround: Change the default git-data path within the etc folder.

Edited Aug 15, 2019 by Agustin Lare
Assignee Loading
Time tracking Loading