Skip to content

gitlab-shell container not starting due to permission issues with SSH keys

Summary

I updated my GET Hybrid Cloud environment to use main-fips (registry.gitlab.com/gitlab-org/build/cng/gitlab-shell:main-fips) for gitlab-shell. It's now not starting due to:

Begin parsing .tpl templates from /etc/gitlab-shell
Writing /srv/gitlab-shell/config.yml
Copying other config files found in /etc/gitlab-shell to /srv/gitlab-shell
Using existing Host Keys
cp: cannot create regular file '/etc/ssh/ssh_host_dsa_key': Permission denied
cp: cannot create regular file '/etc/ssh/ssh_host_dsa_key.pub': Permission denied
cp: cannot create regular file '/etc/ssh/ssh_host_ecdsa_key': Permission denied
cp: cannot create regular file '/etc/ssh/ssh_host_ecdsa_key.pub': Permission denied
cp: cannot create regular file '/etc/ssh/ssh_host_ed25519_key': Permission denied
cp: cannot create regular file '/etc/ssh/ssh_host_ed25519_key.pub': Permission denied
cp: cannot create regular file '/etc/ssh/ssh_host_rsa_key': Permission denied
cp: cannot create regular file '/etc/ssh/ssh_host_rsa_key.pub': Permission denied

Steps to reproduce

I don't remember what I did here. kubectl get secrets gitlab-gitlab-shell-host-keys -o json shows all the keys in place.

  • Previous image: registry.gitlab.com/gitlab-org/build/cng/gitlab-shell@sha256:973c6f5f7c9339a5a4192a73e61a1ce614528e42ba8893cda7b91957279ba589
  • Current image : registry.gitlab.com/gitlab-org/build/cng/gitlab-shell@sha256:027c482eff3e026bfcc44c3371c45e74667678a306041e79e193fbe0d0fa2bc3

Configuration used

(Please provide a sanitized version of the configuration used wrapped in a code block (```yaml))

(Paste sanitized configuration here)

Current behavior

(What you're experiencing happening)

Expected behavior

(What you're expecting to happen)

Versions

  • Chart: 138c146a
  • Platform:
    • Cloud: AWS
    • Self-hosted: EKS
  • Kubernetes: (kubectl version)
    • Client: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.5", GitCommit:"c285e781331a3785a7f436042c65c5641ce8a9e9", GitTreeState:"clean", BuildDate:"2022-03-16T15:51:05Z", GoVersion:"go1.17.8", Compiler:"gc", Platform:"darwin/arm64"}
    • Server: version.Info{Major:"1", Minor:"21+", GitVersion:"v1.21.12-eks-a64ea69", GitCommit:"d4336843ba36120e9ed1491fddff5f2fec33eb77", GitTreeState:"clean", BuildDate:"2022-05-12T18:29:27Z", GoVersion:"go1.16.15", Compiler:"gc", Platform:"linux/amd64"}
  • Helm: version.BuildInfo{Version:"v3.8.0", GitCommit:"d14138609b01886f544b2025f5000351c9eb092e", GitTreeState:"clean", GoVersion:"go1.17.6"}
    • Client:
    • Server:

Relevant logs

(Please provide any relevate log snippets you have collected, using code blocks (```) to format)