Skip to content

Installing Helm Tiller in k3s cluster fails with NoMethodError in cluster_role_binding.rb

Summary

Environment:

  • Arch Linux
  • GitLab 12.10.2 (installation from source)
  • k3s 1.17.4+k3s1

Installing Helm Tiller fails with the following errors:

install_helm_tiller_error

Output in exceptions_json.log 
{
    "severity": "ERROR",
    "time": "2020-05-03T11:25:13.897Z",
    "correlation_id": "rIDldM8L8k4",
    "extra.server": {
        "os": {
            "name": "Linux",
            "version": "#1 ZEN SMP PREEMPT Wed, 29 Apr 2020 16:23:00 +0000",
            "build": "5.6.8-zen1-1-zen",
            "kernel_version": "Linux devenv 5.6.8-zen1-1-zen #1 ZEN SMP PREEMPT Wed, 29 Apr 2020 16:23:00 +0000 x86_64 GNU/Linux"
        },
        "runtime": {
            "name": "ruby",
            "version": "ruby 2.7.1p83 (2020-03-31 revision a0c7c23c9c) [x86_64-linux]"
        }
    },
    "extra.sidekiq": {
        "class": "ClusterInstallAppWorker",
        "args": [
            "helm",
            7
        ],
        "retry": 3,
        "queue": "gcp_cluster:cluster_install_app",
        "queue_namespace": "gcp_cluster",
        "jid": "4d36679ec97310879a223706",
        "created_at": 1588505113.4317422,
        "meta.user": "dNhax",
        "meta.project": "dNhax/bookstore",
        "meta.root_namespace": "dNhax",
        "meta.caller_id": "Projects::Clusters::ApplicationsController#create",
        "correlation_id": "rIDldM8L8k4",
        "enqueued_at": 1588505113.435215
    },
    "extra.error_code": null,
    "extra.service": "Clusters::Applications::InstallService",
    "extra.app_id": 7,
    "extra.app_name": "helm",
    "extra.project_ids": [
        1
    ],
    "extra.group_ids": [],
    "exception.class": "NoMethodError",
    "exception.message": "undefined method `[]=' for nil:NilClass",
    "exception.backtrace": [
        "lib/gitlab/kubernetes/cluster_role_binding.rb:16:in `block in generate'",
        "lib/gitlab/kubernetes/cluster_role_binding.rb:15:in `tap'",
        "lib/gitlab/kubernetes/cluster_role_binding.rb:15:in `generate'",
        "lib/gitlab/kubernetes/helm/base_command.rb:42:in `cluster_role_binding_resource'",
        "lib/gitlab/kubernetes/helm/api.rb:99:in `create_cluster_role_binding'",
        "lib/gitlab/kubernetes/helm/api.rb:20:in `install'",
        "app/services/clusters/applications/install_service.rb:18:in `install'",
        "app/services/clusters/applications/install_service.rb:11:in `execute'",
        "app/workers/cluster_install_app_worker.rb:12:in `block in perform'",
        "app/workers/concerns/cluster_applications.rb:8:in `find_application'",
        "app/workers/cluster_install_app_worker.rb:11:in `perform'",
        "lib/gitlab/with_request_store.rb:7:in `with_request_store'",
        "lib/gitlab/sidekiq_daemon/monitor.rb:49:in `within_job'"
    ]
}

Output in production.log

Processing by Projects::Clusters::ApplicationsController#create as JSON
Parameters: {"namespace_id"=>"dNhax", "project_id"=>"bookstore", "id"=>"7", "application"=>"helm"}
Completed 204 No Content in 1293ms (ActiveRecord: 20.3ms | Elasticsearch: 0.0ms | Allocations: 9918)

The namespace gitlab-managed-apps is created though.

k3s_namespaces

Steps to reproduce

  • install GitLab from source (Arch Linux)
  • install k3s from AUR or from get.k3s.io
  • start k3s: k3s server --docker --disable traefik --disable servicelb
  • add the cluster to a project (see documentation)
  • try to install Helm Tiller

What is the current bug behavior?

Installing Helm Tiller fails with the error above.

What is the expected correct behavior?

Helm Tiller is installed successfully.

Relevant logs and/or screenshots

see in summary

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info 

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

System information
System:		
Current User:	gitlab
Using RVM:	no
Ruby Version:	2.7.1p83
Gem Version:	3.1.2
Bundler Version:2.1.4
Rake Version:	12.3.3
Redis Version:	6.0.0
Git Version:	2.26.2
Sidekiq Version:5.2.7
Go Version:	unknown

GitLab information
Version:	12.10.2
Revision:	
Directory:	/usr/share/webapps/gitlab
DB Adapter:	PostgreSQL
DB Version:	12.2
URL:		https://gitlab.devenv.home
HTTP Clone URL:	https://gitlab.devenv.home/some-group/some-project.git
SSH Clone URL:	gitlab@gitlab.devenv.home:some-group/some-project.git
Using LDAP:	no
Using Omniauth:	yes
Omniauth Providers: 

GitLab Shell
Version:	12.2.0
Repository storage paths:
- default: 	/var/lib/gitlab/repositories
GitLab Shell path:		/usr/share/webapps/gitlab-shell
Git:		/usr/bin/git

Results of GitLab application Check

Expand for output related to the GitLab application check 
(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true`)

Checking GitLab subtasks ...


Checking GitLab Shell ...


GitLab Shell: ... GitLab Shell version >= 12.2.0 ? ... OK (12.2.0)
Running /usr/share/webapps/gitlab-shell/bin/check
Internal API available: OK
Redis available via internal API: OK
gitlab-shell self-check successful


Checking GitLab Shell ... Finished


Checking Gitaly ...


Gitaly: ... default ... OK


Checking Gitaly ... Finished


Checking Sidekiq ...


Sidekiq: ... Running? ... yes
Number of Sidekiq processes ... 1


Checking Sidekiq ... Finished


Checking Incoming Email ...


Incoming Email: ... Reply by email is disabled in config/gitlab.yml


Checking Incoming Email ... Finished


Checking LDAP ...


LDAP: ... LDAP is disabled in config/gitlab.yml


Checking LDAP ... Finished


Checking GitLab App ...


Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... skipped (no tmp uploads folder yet)
Init script exists? ... no
Try fixing it:
Install the init script
For more information see:
doc/install/installation.md in section "Install Init Script"
Please fix the error above and rerun the checks.
Init script up-to-date? ... can't check because of previous errors
Projects have namespace: ...


dNhax / bookstore ... yes
Redis version >= 4.0.0? ... yes
Ruby version >= 2.5.3 ? ... yes (2.7.1)
Git version >= 2.22.0 ? ... yes (2.26.2)
Git user has default SSH configuration? ... yes
Active users: ... 2
Is authorized keys file accessible? ... yes


Checking GitLab App ... Finished


Checking GitLab subtasks ... Finished

Possible fixes

https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/kubernetes/cluster_role_binding.rb#L16