EKS Rewiew App fails deploy (permission issues)
Summary
When I try to deploy a review app to an EKS cluster, I get the following error:
Error: UPGRADE FAILED: configmaps is forbidden: User "system:serviceaccount:<my-repo>-14820762-review-feat-auto-5yhobx:default" cannot list resource "configmaps" in API group "" in the namespace "<my-repo>-14820762-review-feat-auto-5yhobx"I created the EKS cluster via Terraform and I am using gitlab.com (Cloud) Gold version.
Steps to reproduce
- Create an EKS Cluster with Terraform using the module
terraform-aws-modules/eks/aws. - Use https://docs.gitlab.com/ee/user/project/clusters/ to add the cluster to your gitlab project and install helm, ingress, cert and prometheus via the UI in gitlab.
- Create a new branch and use the AutoDevops
.gitlab-ci.ymlto deploy your branch as a review app.
Example Project
What is the current bug behavior?
The pipelines gets stuck at stage deploy and in the review job.
What is the expected correct behavior?
The pipeline goes through and creates the review app correctly.
Relevant logs and/or screenshots
Running with gitlab-runner 12.3.0 (a8a019e0)
on docker-auto-scale fa6cab46
Using Docker executor with image alpine:latest ...
Pulling docker image alpine:latest ...
Using docker image sha256:965ea09ff2ebd2b9eeec88cd822ce156f6674c7e99be082c7efac3c62f3ff652 for alpine:latest ...
Running on runner-fa6cab46-project-14820762-concurrent-0 via runner-fa6cab46-srm-1571876285-f1358155...
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/<my-project>/<my-repo>/.git/
Created fresh repository.
From https://gitlab.com/<my-project>/<my-repo>
* [new ref] refs/pipelines/91100353 -> refs/pipelines/91100353
* [new branch] feat/auto-devops -> origin/feat/auto-devops
* [new tag] v1.10.0 -> v1.10.0
Checking out a01d2c58 as feat/auto-devops...
Skipping Git submodules setup
$ # Auto DevOps variables and functions # collapsed multi-line command
$ check_kube_domain
$ install_dependencies
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/x86_64/APKINDEX.tar.gz
(1/15) Installing ncurses-terminfo-base (6.1_p20190518-r0)
(2/15) Installing ncurses-terminfo (6.1_p20190518-r0)
(3/15) Installing ncurses-libs (6.1_p20190518-r0)
(4/15) Installing readline (8.0.0-r0)
(5/15) Installing bash (5.0.0-r0)
Executing bash-5.0.0-r0.post-install
(6/15) Installing ca-certificates (20190108-r0)
(7/15) Installing nghttp2-libs (1.39.2-r0)
(8/15) Installing libcurl (7.66.0-r0)
(9/15) Installing curl (7.66.0-r0)
(10/15) Installing expat (2.2.8-r0)
(11/15) Installing pcre2 (10.33-r0)
(12/15) Installing git (2.22.0-r0)
(13/15) Installing gzip (1.10-r0)
(14/15) Installing openssl (1.1.1d-r0)
(15/15) Installing tar (1.32-r0)
Executing busybox-1.30.1-r2.trigger
Executing ca-certificates-20190108-r0.trigger
OK: 31 MiB in 29 packages
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 451 100 451 0 0 884 0 --:--:-- --:--:-- --:--:-- 884
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 622 0 622 0 0 4897 0 --:--:-- --:--:-- --:--:-- 4897
2 2219k 2 68133 0 0 268k 0 0:00:08 --:--:-- 0:00:08 268k
100 2219k 100 2219k 0 0 7044k 0 --:--:-- --:--:-- --:--:-- 31.3M
(1/1) Installing glibc (2.28-r0)
OK: 36 MiB in 30 packages
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
52 23.3M 52 12.3M 0 0 17.3M 0 0:00:01 --:--:-- 0:00:01 17.3M
100 23.3M 100 23.3M 0 0 19.4M 0 0:00:01 0:00:01 --:--:-- 19.4M
Client: &version.Version{SemVer:"v2.15.0", GitCommit:"c2440264ca6c078a06e088a838b0476d2fc14750", GitTreeState:"clean"}
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
52 41.1M 52 21.4M 0 0 58.0M 0 --:--:-- --:--:-- --:--:-- 57.9M
100 41.1M 100 41.1M 0 0 86.9M 0 --:--:-- --:--:-- --:--:-- 86.7M
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.0", GitCommit:"641856db18352033a0d96dbc99153fa3b27298e5", GitTreeState:"clean", BuildDate:"2019-03-25T15:53:57Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"linux/amd64"}
$ download_chart
Creating /root/.helm
Creating /root/.helm/repository
Creating /root/.helm/repository/cache
Creating /root/.helm/repository/local
Creating /root/.helm/plugins
Creating /root/.helm/starters
Creating /root/.helm/cache/archive
Creating /root/.helm/repository/repositories.yaml
Adding stable repo with URL: https://kubernetes-charts.storage.googleapis.com
Adding local repo with URL: http://127.0.0.1:8879/charts
$HELM_HOME has been configured at /root/.helm.
Not installing Tiller due to 'client-only' flag having been set
"gitlab" has been added to your repositories
Hang tight while we grab the latest from your chart repositories...
...Unable to get an update from the "local" chart repository (http://127.0.0.1:8879/charts):
Get http://127.0.0.1:8879/charts/index.yaml: dial tcp 127.0.0.1:8879: connect: connection refused
...Successfully got an update from the "gitlab" chart repository
...Successfully got an update from the "stable" chart repository
Update Complete.
Saving 1 charts
Downloading postgresql from repo https://kubernetes-charts.storage.googleapis.com/
Deleting outdated charts
Hang tight while we grab the latest from your chart repositories...
...Unable to get an update from the "local" chart repository (http://127.0.0.1:8879/charts):
Get http://127.0.0.1:8879/charts/index.yaml: dial tcp 127.0.0.1:8879: connect: connection refused
...Successfully got an update from the "gitlab" chart repository
...Successfully got an update from the "stable" chart repository
Update Complete.
Saving 1 charts
Downloading postgresql from repo https://kubernetes-charts.storage.googleapis.com/
Deleting outdated charts
$ ensure_namespace
Name: <my-repo>-14820762-review-feat-auto-5yhobx
Labels: <none>
Annotations: <none>
Status: Active
No resource quota.
No resource limits.
$ install_tiller
Checking Tiller...
$HELM_HOME has been configured at /root/.helm.
Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.
Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.
To prevent this, run `helm init` with the --tiller-tls-verify flag.
For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation
Waiting for deployment "tiller-deploy" rollout to finish: 0 of 1 updated replicas are available...
deployment "tiller-deploy" successfully rolled out
Client: &version.Version{SemVer:"v2.15.0", GitCommit:"c2440264ca6c078a06e088a838b0476d2fc14750", GitTreeState:"clean"}
[debug] Created tunnel using local port: '45461'
[debug] SERVER: "127.0.0.1:45461"
Kubernetes: &version.Info{Major:"1", Minor:"14+", GitVersion:"v1.14.6-eks-5047ed", GitCommit:"5047edce664593832e9b889e447ac75ab104f527", GitTreeState:"clean", BuildDate:"2019-08-21T22:32:40Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Server: &version.Version{SemVer:"v2.15.0", GitCommit:"c2440264ca6c078a06e088a838b0476d2fc14750", GitTreeState:"clean"}
$ create_secret
Create secret...
secret/gitlab-registry replaced
$ deploy
UPGRADE FAILED
Error: configmaps is forbidden: User "system:serviceaccount:<my-repo>-14820762-review-feat-auto-5yhobx:default" cannot list resource "configmaps" in API group "" in the namespace "<my-repo>-14820762-review-feat-auto-5yhobx"
Error: UPGRADE FAILED: configmaps is forbidden: User "system:serviceaccount:<my-repo>-14820762-review-feat-auto-5yhobx:default" cannot list resource "configmaps" in API group "" in the namespace "<my-repo>-14820762-review-feat-auto-5yhobx"Output of checks
This bug happens on GitLab.com
Edited by Özer Sahin