Problem installing gitlab-runner on Kubernetes
### Summary
Problem installing gitlab-runner on kubernetes cluster via gitlab. I have behind a restricted firewall. A lot of endpoints has been whitelisted. I guess it is because of our firewall restriction but as I cannot see deeply the error in logs, I cannot whitelist it.
### Steps to reproduce
1. Configure existing kubernetes cluster
2. Install Helm tiller application from Gitlab
3. Install Giltab-runner application from Gitlab (fail)
### What is the current *bug* behavior?
Something went wrong while installing GitLab Runner
Operation failed. Check pod logs for install-runner for more details.
### Relevant logs and/or screenshots
```
:~$ kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:40:16Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.0", GitCommit:"641856db18352033a0d96dbc99153fa3b27298e5", GitTreeState:"clean", BuildDate:"2019-03-25T15:45:25Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"linux/amd64"}
```
```
:~$ kubectl logs install-runner -n gitlab-managed-apps
+ helm init --upgrade
Creating /root/.helm
Creating /root/.helm/repository
Creating /root/.helm/repository/cache
Creating /root/.helm/repository/local
Creating /root/.helm/plugins
Creating /root/.helm/starters
Creating /root/.helm/cache/archive
Creating /root/.helm/repository/repositories.yaml
Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com
Adding local repo with URL: http://127.0.0.1:8879/charts
$HELM_HOME has been configured at /root/.helm.
Tiller (the Helm server-side component) has been upgraded to the current version.
Happy Helming!
+ seq 1 30
+ helm version
Client: &version.Version{SemVer:"v2.12.3", GitCommit:"eecf22f77df5f65c823aacd2dbd30ae6c65f186e", GitTreeState:"clean"}
Error: cannot connect to Tiller
+ sleep 1s
+ echo 'Retrying (1)...'
+ helm version
Retrying (1)...
Client: &version.Version{SemVer:"v2.12.3", GitCommit:"eecf22f77df5f65c823aacd2dbd30ae6c65f186e", GitTreeState:"clean"}
Error: cannot connect to Tiller
+ sleep 1s
Retrying (2)...
+ echo 'Retrying (2)...'
+ helm version
Client: &version.Version{SemVer:"v2.12.3", GitCommit:"eecf22f77df5f65c823aacd2dbd30ae6c65f186e", GitTreeState:"clean"}
Error: cannot connect to Tiller
+ sleep 1s
.....
.....
+ echo 'Retrying (24)...'
+ helm version
Retrying (24)...
Client: &version.Version{SemVer:"v2.12.3", GitCommit:"eecf22f77df5f65c823aacd2dbd30ae6c65f186e", GitTreeState:"clean"}
Error: cannot connect to Tiller
+ sleep 1s
+ echo 'Retrying (25)...'
+ helm version
Retrying (25)...
Client: &version.Version{SemVer:"v2.12.3", GitCommit:"eecf22f77df5f65c823aacd2dbd30ae6c65f186e", GitTreeState:"clean"}
Error: cannot connect to Tiller
+ sleep 1s
Retrying (26)...
+ echo 'Retrying (26)...'
+ helm version
Client: &version.Version{SemVer:"v2.12.3", GitCommit:"eecf22f77df5f65c823aacd2dbd30ae6c65f186e", GitTreeState:"clean"}
Error: cannot connect to Tiller
+ sleep 1s
+ echo 'Retrying (27)...'
+ helm version
Retrying (27)...
Client: &version.Version{SemVer:"v2.12.3", GitCommit:"eecf22f77df5f65c823aacd2dbd30ae6c65f186e", GitTreeState:"clean"}
Error: cannot connect to Tiller
+ sleep 1s
Retrying (28)...
+ echo 'Retrying (28)...'
+ helm version
Client: &version.Version{SemVer:"v2.12.3", GitCommit:"eecf22f77df5f65c823aacd2dbd30ae6c65f186e", GitTreeState:"clean"}
Error: cannot connect to Tiller
+ sleep 1s
+ echo 'Retrying (29)...'
+ helm version
Retrying (29)...
Client: &version.Version{SemVer:"v2.12.3", GitCommit:"eecf22f77df5f65c823aacd2dbd30ae6c65f186e", GitTreeState:"clean"}
Error: cannot connect to Tiller
+ sleep 1s
+ echo 'Retrying (30)...'
+ helm repo add runner https://charts.gitlab.io
Retrying (30)...
"runner" has been added to your repositories
+ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Skip local chart repository
...Successfully got an update from the "runner" chart repository
...Successfully got an update from the "stable" chart repository
Update Complete. ⎈ Happy Helming!⎈
+ helm upgrade runner runner/gitlab-runner --install --reset-values --tls --tls-ca-cert /data/helm/runner/config/ca.pem --tls-cert /data/helm/runner/config/cert.pem --tls-key /data/helm/runner/config/key.pem --version 0.5.2 --set 'rbac.create=true,rbac.enabled=true' --namespace gitlab-managed-apps -f /data/helm/runner/config/values.yaml
Error: failed to download "runner/gitlab-runner" (hint: running `helm repo update` may help)
```
```
:~$ kubectl --namespace gitlab-managed-apps logs tiller-deploy-7fb68896db-bhdvf
[main] 2019/07/10 13:33:52 Starting Tiller v2.12.3 (tls=true)
[main] 2019/07/10 13:33:52 GRPC listening on :44134
[main] 2019/07/10 13:33:52 Probes listening on :44135
[main] 2019/07/10 13:33:52 Storage driver is ConfigMap
[main] 2019/07/10 13:33:52 Max history per release is 0
[tiller] 2019/07/10 13:39:42 getting history for release prometheus
[storage] 2019/07/10 13:39:42 getting release history for "prometheus"
[tiller] 2019/07/10 13:39:42 preparing install for prometheus
[storage] 2019/07/10 13:39:42 getting release history for "prometheus"
[tiller] 2019/07/10 13:39:42 rendering prometheus chart using values
2019/07/10 13:39:42 info: manifest "prometheus/templates/alertmanager-pvc.yaml" is empty. Skipping.
2019/07/10 13:39:42 info: manifest "prometheus/templates/server-networkpolicy.yaml" is empty. Skipping.
2019/07/10 13:39:42 info: manifest "prometheus/templates/node-exporter-service.yaml" is empty. Skipping.
2019/07/10 13:39:42 info: manifest "prometheus/templates/alertmanager-configmap.yaml" is empty. Skipping.
2019/07/10 13:39:42 info: manifest "prometheus/templates/alertmanager-deployment.yaml" is empty. Skipping.
2019/07/10 13:39:42 info: manifest "prometheus/templates/server-ingress.yaml" is empty. Skipping.
2019/07/10 13:39:42 info: manifest "prometheus/templates/pushgateway-deployment.yaml" is empty. Skipping.
2019/07/10 13:39:42 info: manifest "prometheus/templates/kube-state-metrics-networkpolicy.yaml" is empty. Skipping.
2019/07/10 13:39:42 info: manifest "prometheus/templates/alertmanager-ingress.yaml" is empty. Skipping.
2019/07/10 13:39:42 info: manifest "prometheus/templates/pushgateway-ingress.yaml" is empty. Skipping.
2019/07/10 13:39:42 info: manifest "prometheus/templates/alertmanager-networkpolicy.yaml" is empty. Skipping.
2019/07/10 13:39:42 info: manifest "prometheus/templates/pushgateway-service.yaml" is empty. Skipping.
2019/07/10 13:39:42 info: manifest "prometheus/templates/node-exporter-daemonset.yaml" is empty. Skipping.
2019/07/10 13:39:42 info: manifest "prometheus/templates/alertmanager-service.yaml" is empty. Skipping.
[tiller] 2019/07/10 13:39:42 performing install for prometheus
[tiller] 2019/07/10 13:39:42 executing 0 crd-install hooks for prometheus
[tiller] 2019/07/10 13:39:42 hooks complete for crd-install prometheus
[tiller] 2019/07/10 13:39:42 executing 0 pre-install hooks for prometheus
[tiller] 2019/07/10 13:39:42 hooks complete for pre-install prometheus
[storage] 2019/07/10 13:39:42 getting release history for "prometheus"
[storage] 2019/07/10 13:39:42 creating release "prometheus.v1"
[kube] 2019/07/10 13:39:42 building resources from manifest
[kube] 2019/07/10 13:39:42 creating 15 resource(s)
[tiller] 2019/07/10 13:39:43 executing 0 post-install hooks for prometheus
[tiller] 2019/07/10 13:39:43 hooks complete for post-install prometheus
[storage] 2019/07/10 13:39:43 updating release "prometheus.v1"
[storage] 2019/07/10 13:39:43 getting last revision of "prometheus"
[storage] 2019/07/10 13:39:43 getting release history for "prometheus"
[kube] 2019/07/10 13:39:43 Doing get for ConfigMap: "prometheus-prometheus-server"
[kube] 2019/07/10 13:39:43 get relation pod of object: gitlab-managed-apps/ConfigMap/prometheus-prometheus-server
[kube] 2019/07/10 13:39:43 Doing get for PersistentVolumeClaim: "prometheus-prometheus-server"
[kube] 2019/07/10 13:39:43 get relation pod of object: gitlab-managed-apps/PersistentVolumeClaim/prometheus-prometheus-server
[kube] 2019/07/10 13:39:43 Doing get for ServiceAccount: "prometheus-alertmanager"
[kube] 2019/07/10 13:39:43 get relation pod of object: gitlab-managed-apps/ServiceAccount/prometheus-alertmanager
[kube] 2019/07/10 13:39:43 Doing get for ServiceAccount: "prometheus-kube-state-metrics"
[kube] 2019/07/10 13:39:43 get relation pod of object: gitlab-managed-apps/ServiceAccount/prometheus-kube-state-metrics
[kube] 2019/07/10 13:39:43 Doing get for ServiceAccount: "prometheus-node-exporter"
[kube] 2019/07/10 13:39:43 get relation pod of object: gitlab-managed-apps/ServiceAccount/prometheus-node-exporter
[kube] 2019/07/10 13:39:43 Doing get for ServiceAccount: "prometheus-pushgateway"
[kube] 2019/07/10 13:39:43 get relation pod of object: gitlab-managed-apps/ServiceAccount/prometheus-pushgateway
[kube] 2019/07/10 13:39:43 Doing get for ServiceAccount: "prometheus-prometheus-server"
[kube] 2019/07/10 13:39:43 get relation pod of object: gitlab-managed-apps/ServiceAccount/prometheus-prometheus-server
[kube] 2019/07/10 13:39:43 Doing get for ClusterRole: "prometheus-kube-state-metrics"
[kube] 2019/07/10 13:39:43 get relation pod of object: /ClusterRole/prometheus-kube-state-metrics
[kube] 2019/07/10 13:39:43 Doing get for ClusterRole: "prometheus-prometheus-server"
[kube] 2019/07/10 13:39:43 get relation pod of object: /ClusterRole/prometheus-prometheus-server
[kube] 2019/07/10 13:39:43 Doing get for ClusterRoleBinding: "prometheus-kube-state-metrics"
[kube] 2019/07/10 13:39:43 get relation pod of object: /ClusterRoleBinding/prometheus-kube-state-metrics
[kube] 2019/07/10 13:39:43 Doing get for ClusterRoleBinding: "prometheus-prometheus-server"
[kube] 2019/07/10 13:39:43 get relation pod of object: /ClusterRoleBinding/prometheus-prometheus-server
[kube] 2019/07/10 13:39:43 Doing get for Service: "prometheus-kube-state-metrics"
[kube] 2019/07/10 13:39:43 get relation pod of object: gitlab-managed-apps/Service/prometheus-kube-state-metrics
[kube] 2019/07/10 13:39:43 Doing get for Service: "prometheus-prometheus-server"
[kube] 2019/07/10 13:39:43 get relation pod of object: gitlab-managed-apps/Service/prometheus-prometheus-server
[kube] 2019/07/10 13:39:43 Doing get for Deployment: "prometheus-kube-state-metrics"
[kube] 2019/07/10 13:39:43 get relation pod of object: gitlab-managed-apps/Deployment/prometheus-kube-state-metrics
[kube] 2019/07/10 13:39:43 Doing get for Deployment: "prometheus-prometheus-server"
[kube] 2019/07/10 13:39:43 get relation pod of object: gitlab-managed-apps/Deployment/prometheus-prometheus-server
[tiller] 2019/07/10 13:39:47 getting history for release certmanager
[storage] 2019/07/10 13:39:47 getting release history for "certmanager"
[tiller] 2019/07/10 13:39:48 preparing install for certmanager
[storage] 2019/07/10 13:39:48 getting release history for "certmanager"
[tiller] 2019/07/10 13:39:48 rendering cert-manager chart using values
2019/07/10 13:39:48 info: manifest "cert-manager/templates/00-namespace.yaml" is empty. Skipping.
[tiller] 2019/07/10 13:39:48 performing install for certmanager
[tiller] 2019/07/10 13:39:48 executing 3 crd-install hooks for certmanager
[kube] 2019/07/10 13:39:48 building resources from manifest
[kube] 2019/07/10 13:39:48 creating 1 resource(s)
[kube] 2019/07/10 13:39:48 building resources from manifest
[kube] 2019/07/10 13:39:48 creating 1 resource(s)
[kube] 2019/07/10 13:39:48 building resources from manifest
[kube] 2019/07/10 13:39:48 creating 1 resource(s)
[tiller] 2019/07/10 13:39:48 hooks complete for crd-install certmanager
[tiller] 2019/07/10 13:39:48 executing 3 pre-install hooks for certmanager
[tiller] 2019/07/10 13:39:48 hooks complete for pre-install certmanager
[storage] 2019/07/10 13:39:48 getting release history for "certmanager"
[storage] 2019/07/10 13:39:48 creating release "certmanager.v1"
[kube] 2019/07/10 13:39:48 building resources from manifest
[kube] 2019/07/10 13:39:48 creating 4 resource(s)
[tiller] 2019/07/10 13:39:48 executing 3 post-install hooks for certmanager
[tiller] 2019/07/10 13:39:48 hooks complete for post-install certmanager
[storage] 2019/07/10 13:39:48 updating release "certmanager.v1"
[storage] 2019/07/10 13:39:48 getting last revision of "certmanager"
[storage] 2019/07/10 13:39:48 getting release history for "certmanager"
[kube] 2019/07/10 13:39:48 Doing get for ServiceAccount: "certmanager-cert-manager"
[kube] 2019/07/10 13:39:48 get relation pod of object: gitlab-managed-apps/ServiceAccount/certmanager-cert-manager
[kube] 2019/07/10 13:39:48 Doing get for ClusterRole: "certmanager-cert-manager"
[kube] 2019/07/10 13:39:48 get relation pod of object: /ClusterRole/certmanager-cert-manager
[kube] 2019/07/10 13:39:48 Doing get for ClusterRoleBinding: "certmanager-cert-manager"
[kube] 2019/07/10 13:39:48 get relation pod of object: /ClusterRoleBinding/certmanager-cert-manager
[kube] 2019/07/10 13:39:48 Doing get for Deployment: "certmanager-cert-manager"
[kube] 2019/07/10 13:39:48 get relation pod of object: gitlab-managed-apps/Deployment/certmanager-cert-manager
```
### Output of checks
#### Results of GitLab environment info
<details>
<summary>Expand for output related to GitLab environment info</summary>
<pre>
# sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production
System information
System: Ubuntu 18.04
Current User: git
Using RVM: no
Ruby Version: 2.6.3p62
Gem Version: 3.0.3
Bundler Version:1.17.3
Rake Version: 12.3.2
Redis Version: 4.0.9
Git Version: 2.22.0
Sidekiq Version:5.2.7
Go Version: go1.11.10 linux/amd64
GitLab information
Version: 12.0.2
Revision: 1a9fd38a4ca
Directory: /home/git/gitlab
DB Adapter: PostgreSQL
DB Version: 10.9
Using LDAP: yes
Using Omniauth: yes
Omniauth Providers: saml
GitLab Shell
Version: 9.3.0
Repository storage paths:
- default: /home/git/repositories
GitLab Shell path: /home/git/gitlab-shell
Git: /usr/bin/git
</pre>
</details>
#### Results of GitLab application Check
<details>
<summary>Expand for output related to the GitLab application check</summary>
<pre>
# sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true
Checking GitLab subtasks ...
Checking GitLab Shell ...
GitLab Shell: ... GitLab Shell version >= 9.3.0 ? ... OK (9.3.0)
Running /home/git/gitlab-shell/bin/check
Check GitLab API access: OK
Redis available via internal API: OK
Access to /home/git/.ssh/authorized_keys: OK
gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Gitaly ...
Gitaly: ... default ... OK
Checking Gitaly ... Finished
Checking Sidekiq ...
Sidekiq: ... Running? ... yes
Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Incoming Email ...
Incoming Email: ... Reply by email is disabled in config/gitlab.yml
Checking Incoming Email ... Finished
Checking LDAP ...
LDAP: ... Server: ldapmain
LDAP authentication... Success
LDAP users with access to your GitLab server (only showing the first 100 results)
Checking LDAP ... Finished
Checking GitLab App ...
Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... yes
Init script up-to-date? ... yes
Projects have namespace: ...
5/2 ... yes
6/3 ... yes
12/4 ... yes
8/6 ... yes
12/7 ... yes
12/9 ... yes
2/12 ... yes
14/13 ... yes
12/14 ... yes
14/15 ... yes
8/16 ... yes
14/17 ... yes
12/18 ... yes
12/21 ... yes
12/22 ... yes
12/23 ... yes
12/24 ... yes
4/25 ... yes
12/26 ... yes
12/28 ... yes
12/29 ... yes
46/30 ... yes
12/31 ... yes
46/32 ... yes
6/33 ... yes
12/34 ... yes
12/35 ... yes
12/36 ... yes
6/37 ... yes
46/39 ... yes
12/41 ... yes
12/42 ... yes
6/186 ... yes
6/187 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.5.3 ? ... yes (2.6.3)
Git version >= 2.21.0 ? ... yes (2.22.0)
Git user has default SSH configuration? ... yes
Active users: ... 19
Checking GitLab App ... Finished
Checking GitLab subtasks ... Finished
</pre>
</details>
issue