Skip to content

Improve Review Apps cleanup when previous deployment failed by only issuing an `helm delete` command

Rémy Coutable requested to merge improve-review-apps-cleanup-when-previous-deployment-failed into master

What does this MR do?

  1. Only run delete (Helm delete command) and not cleanup (Kubernetes delete command) when the previous Review App deployment failed.
    • The goal is to avoid errors like https://gitlab.com/gitlab-org/gitlab-ce/-/jobs/217794390 where a deployment would fail because resources are still being deleted somehow.
    • Normally helm delete

      removes all of the resources associated with the last release of the chart. so there shouldn't be any need to manually delete the Kubernetes resources, we should let Helm do its job here.

Race condition between `helm delete` and `kubectl delete` commands

As we can see, Tiller is starting a new deploy before Kubernetes has actually deleted all the resources.

** Checking for previous deployment of review-62151-brok-mkip7n **
Previous deployment found, checking status...
Deployment in bad state, cleaning up review-62151-brok-mkip7n
Previous deployment state: FAILED

** Deleting release 'review-62151-brok-mkip7n'... **
release "review-62151-brok-mkip7n" deleted

** Cleaning up 'review-62151-brok-mkip7n'... **
pod "review-62151-brok-mkip7n-gitaly-0" deleted
pod "review-62151-brok-mkip7n-gitlab-shell-7bf7b85875-ktvvb" deleted
pod "review-62151-brok-mkip7n-gitlab-shell-7bf7b85875-pslq8" deleted
pod "review-62151-brok-mkip7n-minio-7c7bfb457d-kq8cd" deleted
pod "review-62151-brok-mkip7n-nginx-ingress-controller-7b5bdfb59svd5" deleted
pod "review-62151-brok-mkip7n-nginx-ingress-controller-7b5bdfb5bkpsf" deleted
pod "review-62151-brok-mkip7n-nginx-ingress-default-backend-847hfc5h" deleted
pod "review-62151-brok-mkip7n-nginx-ingress-default-backend-847tsk8k" deleted
pod "review-62151-brok-mkip7n-postgresql-66d468f845-lbssv" deleted
pod "review-62151-brok-mkip7n-redis-64d5dbb8fb-2h59g" deleted
pod "review-62151-brok-mkip7n-registry-67969487fc-b5vfp" deleted
pod "review-62151-brok-mkip7n-registry-67969487fc-fldrt" deleted
pod "review-62151-brok-mkip7n-sidekiq-all-in-1-77cfbd9dc4-6xxcr" deleted
pod "review-62151-brok-mkip7n-task-runner-5664b8575-n2lct" deleted
pod "review-62151-brok-mkip7n-unicorn-79dbc4c9d9-hcr52" deleted
pod "review-62151-brok-mkip7n-unicorn-79dbc4c9d9-znk9p" deleted
secret "review-62151-brok-mkip7n-gitaly-secret" deleted
secret "review-62151-brok-mkip7n-gitlab-initial-root-password" deleted
secret "review-62151-brok-mkip7n-gitlab-runner-secret" deleted
secret "review-62151-brok-mkip7n-gitlab-shell-host-keys" deleted
secret "review-62151-brok-mkip7n-gitlab-shell-secret" deleted
secret "review-62151-brok-mkip7n-gitlab-workhorse-secret" deleted
secret "review-62151-brok-mkip7n-minio-secret" deleted
secret "review-62151-brok-mkip7n-postgresql-password" deleted
secret "review-62151-brok-mkip7n-rails-secret" deleted
secret "review-62151-brok-mkip7n-redis-secret" deleted
secret "review-62151-brok-mkip7n-registry-httpsecret" deleted
secret "review-62151-brok-mkip7n-registry-secret" deleted
persistentvolumeclaim "repo-data-review-62151-brok-mkip7n-gitaly-0" deleted
persistentvolumeclaim "review-62151-brok-mkip7n-minio" deleted
persistentvolumeclaim "review-62151-brok-mkip7n-postgresql" deleted
persistentvolumeclaim "review-62151-brok-mkip7n-redis" deleted

** Creating the review-62151-brok-mkip7n-gitlab-initial-root-password secret in the review-apps-ce namespace... **
secret "review-62151-brok-mkip7n-gitlab-initial-root-password" created

** Downloading the GitLab chart... **
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  208k  100  208k    0     0  2064k      0 --:--:-- --:--:-- --:--:-- 2064k
Adding the gitlab repo to Helm...
"gitlab" has been added to your repositories
Building the gitlab chart's dependencies...
Hang tight while we grab the latest from your chart repositories...
...Unable to get an update from the "local" chart repository (http://127.0.0.1:8879/charts):
	Get http://127.0.0.1:8879/charts/index.yaml: dial tcp 127.0.0.1:8879: connect: connection refused
...Successfully got an update from the "gitlab" chart repository
...Successfully got an update from the "stable" chart repository
Update Complete. ⎈Happy Helming!⎈
Saving 4 charts
Downloading cert-manager from repo https://kubernetes-charts.storage.googleapis.com/
Downloading prometheus from repo https://kubernetes-charts.storage.googleapis.com/
Downloading postgresql from repo https://kubernetes-charts.storage.googleapis.com/
Downloading gitlab-runner from repo https://charts.gitlab.io/
Deleting outdated charts
Deploying with:
  helm upgrade --install     --wait     --timeout 600     --set global.appConfig.enableUsagePing=false     --set releaseOverride="review-62151-brok-mkip7n"     --set global.hosts.hostSuffix="review-62151-brok-mkip7n"     --set global.hosts.domain="ce.gitlab-review.app"     --set certmanager.install=false     --set prometheus.install=false     --set global.ingress.configureCertmanager=false     --set global.ingress.tls.secretName=tls-cert     --set global.ingress.annotations."external-dns\.alpha\.kubernetes\.io/ttl"="10"
    --set nginx-ingress.controller.service.enableHttp=false     --set nginx-ingress.defaultBackend.resources.requests.memory=7Mi     --set nginx-ingress.controller.resources.requests.memory=440M     --set nginx-ingress.controller.replicaCount=2     --set gitlab.unicorn.resources.requests.cpu=200m     --set gitlab.sidekiq.resources.requests.cpu=100m     --set gitlab.sidekiq.resources.requests.memory=800M     --set gitlab.gitlab-shell.resources.requests.cpu=100m     --set redis.resources.requests.cpu=100m     --set minio.resources.requests.cpu=100m     --set gitlab.migrations.image.repository="registry.gitlab.com/gitlab-org/build/cng-mirror/gitlab-rails-ce"     --set gitlab.migrations.image.tag="62151-broken-master"     --set gitlab.sidekiq.image.repository="registry.gitlab.com/gitlab-org/build/cng-mirror/gitlab-sidekiq-ce"     --set gitlab.sidekiq.image.tag="62151-broken-master"     --set gitlab.unicorn.image.repository="registry.gitlab.com/gitlab-org/build/cng-mirror/gitlab-unicorn-ce"     --set gitlab.unicorn.image.tag="62151-broken-master"     --set gitlab.task-runner.image.repository="registry.gitlab.com/gitlab-org/build/cng-mirror/gitlab-task-runner-ce"     --set gitlab.task-runner.image.tag="62151-broken-master"     --set gitlab.gitaly.image.repository="registry.gitlab.com/gitlab-org/build/cng-mirror/gitaly"     --set gitlab.gitaly.image.tag="v1.42.1"     --set gitlab.gitlab-shell.image.repository="registry.gitlab.com/gitlab-org/build/cng-mirror/gitlab-shell"     --set gitlab.gitlab-shell.image.tag="v9.2.0"     --set gitlab.unicorn.workhorse.image="registry.gitlab.com/gitlab-org/build/cng-mirror/gitlab-workhorse-ce"     --set gitlab.unicorn.workhorse.tag="62151-broken-master"     --set nginx-ingress.controller.config.ssl-ciphers="ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA256:ECDHE-RSA-AES256-SHA:ECDHE-RSA-AES128-SHA:AES256-GCM-SHA384:AES128-GCM-SHA256:AES256-SHA256:AES128-SHA256:AES256-SHA:AES128-SHA:!aNULL:!eNULL:!EXPORT:!DES:!MD5:!PSK:!RC4"     --namespace="review-apps-ce"     --version="62681373-217794390"     "review-62151-brok-mkip7n"     .
Release "review-62151-brok-mkip7n" does not exist. Installing it now.
Error: release review-62151-brok-mkip7n failed: object is being deleted: persistentvolumeclaims "review-62151-brok-mkip7n-minio" already exists

When I look at Tiller's logs I see the following:

E  [tiller] 2019/05/23 07:48:16 uninstall: Deleting review-62151-brok-mkip7n
E  [tiller] 2019/05/23 07:48:16 executing 6 pre-delete hooks for review-62151-brok-mkip7n
E  [tiller] 2019/05/23 07:48:16 hooks complete for pre-delete review-62151-brok-mkip7n
E  [storage] 2019/05/23 07:48:16 updating release "review-62151-brok-mkip7n.v4"
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-unicorn" Ingress
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-minio" Ingress
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-registry" Ingress
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-gitaly" Service
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-gitlab-shell" Service
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-unicorn" Service
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-minio-svc" Service
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-nginx-ingress-controller-metrics" Service
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-nginx-ingress-controller" Service
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-nginx-ingress-controller-stats" Service
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-nginx-ingress-default-backend" Service
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-postgresql" Service
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-redis" Service
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-registry" Service
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-migrations.4" Job
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-minio-create-buckets.4" Job
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-gitaly" StatefulSet
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-gitlab-runner" Deployment
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-gitlab-shell" Deployment
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-sidekiq-all-in-1" Deployment
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-task-runner" Deployment
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-unicorn" Deployment
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-minio" Deployment
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-nginx-ingress-controller" Deployment
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-nginx-ingress-default-backend" Deployment
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-postgresql" Deployment
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-redis" Deployment
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-registry" Deployment
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-gitlab-runner" RoleBinding
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-nginx-ingress" RoleBinding
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-gitlab-runner" Role
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-nginx-ingress" Role
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-gitlab-runner" ServiceAccount
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-nginx-ingress" ServiceAccount
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-minio" PersistentVolumeClaim
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-postgresql" PersistentVolumeClaim
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-redis" PersistentVolumeClaim
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-gitaly" ConfigMap
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-gitlab-shell" ConfigMap
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-nginx-ingress-tcp" ConfigMap
E  [kube] 2019/05/23 07:48:16 Starting delete for "review-62151-brok-mkip7n-migrations" ConfigMap
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-sidekiq-all-in-1" ConfigMap
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-sidekiq" ConfigMap
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-task-runner" ConfigMap
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-unicorn" ConfigMap
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-workhorse-config" ConfigMap
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-unicorn-tests" ConfigMap
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-minio-config-cm" ConfigMap
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-nginx-ingress-controller" ConfigMap
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-nginx-ingress-custom-headers" ConfigMap
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-postgresql" ConfigMap
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-redis" ConfigMap
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-registry" ConfigMap
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-gitlab-runner" ConfigMap
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-gitaly" PodDisruptionBudget
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-gitlab-shell" PodDisruptionBudget
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-sidekiq" PodDisruptionBudget
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-unicorn" PodDisruptionBudget
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-minio-v1" PodDisruptionBudget
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-nginx-ingress-controller" PodDisruptionBudget
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-nginx-ingress-default-backend" PodDisruptionBudget
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-redis-v1" PodDisruptionBudget
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-registry-v1" PodDisruptionBudget
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-gitlab-shell" HorizontalPodAutoscaler
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-sidekiq-all-in-1" HorizontalPodAutoscaler
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-unicorn" HorizontalPodAutoscaler
E  [kube] 2019/05/23 07:48:17 Starting delete for "review-62151-brok-mkip7n-registry" HorizontalPodAutoscaler
E  [tiller] 2019/05/23 07:48:17 executing 6 post-delete hooks for review-62151-brok-mkip7n
E  [tiller] 2019/05/23 07:48:17 hooks complete for post-delete review-62151-brok-mkip7n
E  [tiller] 2019/05/23 07:48:17 purge requested for review-62151-brok-mkip7n
E  [storage] 2019/05/23 07:48:17 deleting release "review-62151-brok-mkip7n.v1"
E  [storage] 2019/05/23 07:48:17 deleting release "review-62151-brok-mkip7n.v2"
E  [storage] 2019/05/23 07:48:17 deleting release "review-62151-brok-mkip7n.v3"
E  [storage] 2019/05/23 07:48:17 deleting release "review-62151-brok-mkip7n.v4"
E  [tiller] 2019/05/23 07:48:39 getting history for release review-62151-brok-mkip7n
E  [storage] 2019/05/23 07:48:39 getting release history for "review-62151-brok-mkip7n"
E  [tiller] 2019/05/23 07:48:40 preparing install for review-62151-brok-mkip7n
E  [storage] 2019/05/23 07:48:40 getting release history for "review-62151-brok-mkip7n"
E  [tiller] 2019/05/23 07:48:41 rendering gitlab chart using values
E  [tiller] 2019/05/23 07:48:42 performing install for review-62151-brok-mkip7n
E  [tiller] 2019/05/23 07:48:42 executing 6 crd-install hooks for review-62151-brok-mkip7n
E  [tiller] 2019/05/23 07:48:42 hooks complete for crd-install review-62151-brok-mkip7n
E  [tiller] 2019/05/23 07:48:42 executing 6 pre-install hooks for review-62151-brok-mkip7n
E  [tiller] 2019/05/23 07:48:42 deleting pre-install hook review-62151-brok-mkip7n-shared-secrets for release review-62151-brok-mkip7n due to "before-hook-creation" policy
E  [kube] 2019/05/23 07:48:42 Starting delete for "review-62151-brok-mkip7n-shared-secrets" ServiceAccount
E  [kube] 2019/05/23 07:48:42 serviceaccounts "review-62151-brok-mkip7n-shared-secrets" not found
E  [kube] 2019/05/23 07:48:42 building resources from manifest
E  [kube] 2019/05/23 07:48:42 creating 1 resource(s)
E  [kube] 2019/05/23 07:48:42 Watching for changes to ServiceAccount review-62151-brok-mkip7n-shared-secrets with timeout of 10m0s
E  [kube] 2019/05/23 07:48:42 Add/Modify event for review-62151-brok-mkip7n-shared-secrets: ADDED
E  [tiller] 2019/05/23 07:48:42 deleting pre-install hook review-62151-brok-mkip7n-shared-secrets for release review-62151-brok-mkip7n due to "before-hook-creation" policy
E  [kube] 2019/05/23 07:48:42 Starting delete for "review-62151-brok-mkip7n-shared-secrets" Role
E  [kube] 2019/05/23 07:48:42 roles.rbac.authorization.k8s.io "review-62151-brok-mkip7n-shared-secrets" not found
E  [kube] 2019/05/23 07:48:42 building resources from manifest
E  [kube] 2019/05/23 07:48:42 creating 1 resource(s)
E  [kube] 2019/05/23 07:48:42 Watching for changes to Role review-62151-brok-mkip7n-shared-secrets with timeout of 10m0s
E  [kube] 2019/05/23 07:48:42 Add/Modify event for review-62151-brok-mkip7n-shared-secrets: ADDED
E  [tiller] 2019/05/23 07:48:42 deleting pre-install hook review-62151-brok-mkip7n-shared-secrets for release review-62151-brok-mkip7n due to "before-hook-creation" policy
E  [kube] 2019/05/23 07:48:42 Starting delete for "review-62151-brok-mkip7n-shared-secrets" RoleBinding
E  [kube] 2019/05/23 07:48:42 rolebindings.rbac.authorization.k8s.io "review-62151-brok-mkip7n-shared-secrets" not found
E  [kube] 2019/05/23 07:48:42 building resources from manifest
E  [kube] 2019/05/23 07:48:42 creating 1 resource(s)
E  [kube] 2019/05/23 07:48:42 Watching for changes to RoleBinding review-62151-brok-mkip7n-shared-secrets with timeout of 10m0s
E  [kube] 2019/05/23 07:48:42 Add/Modify event for review-62151-brok-mkip7n-shared-secrets: ADDED
E  [tiller] 2019/05/23 07:48:42 deleting pre-install hook review-62151-brok-mkip7n-shared-secrets for release review-62151-brok-mkip7n due to "before-hook-creation" policy
E  [kube] 2019/05/23 07:48:42 Starting delete for "review-62151-brok-mkip7n-shared-secrets" ConfigMap
E  [kube] 2019/05/23 07:48:42 configmaps "review-62151-brok-mkip7n-shared-secrets" not found
E  [kube] 2019/05/23 07:48:42 building resources from manifest
E  [kube] 2019/05/23 07:48:42 creating 1 resource(s)
E  [kube] 2019/05/23 07:48:42 Watching for changes to ConfigMap review-62151-brok-mkip7n-shared-secrets with timeout of 10m0s
E  [kube] 2019/05/23 07:48:42 Add/Modify event for review-62151-brok-mkip7n-shared-secrets: ADDED
E  [tiller] 2019/05/23 07:48:42 deleting pre-install hook review-62151-brok-mkip7n-shared-secrets.1-krb for release review-62151-brok-mkip7n due to "before-hook-creation" policy
E  [kube] 2019/05/23 07:48:42 Starting delete for "review-62151-brok-mkip7n-shared-secrets.1-krb" Job
E  [kube] 2019/05/23 07:48:42 jobs.batch "review-62151-brok-mkip7n-shared-secrets.1-krb" not found
E  [kube] 2019/05/23 07:48:42 building resources from manifest
E  [kube] 2019/05/23 07:48:42 creating 1 resource(s)
E  [kube] 2019/05/23 07:48:42 Watching for changes to Job review-62151-brok-mkip7n-shared-secrets.1-krb with timeout of 10m0s
E  [kube] 2019/05/23 07:48:42 Add/Modify event for review-62151-brok-mkip7n-shared-secrets.1-krb: ADDED
E  [kube] 2019/05/23 07:48:42 review-62151-brok-mkip7n-shared-secrets.1-krb: Jobs active: 0, jobs failed: 0, jobs succeeded: 0
E  [kube] 2019/05/23 07:48:42 Add/Modify event for review-62151-brok-mkip7n-shared-secrets.1-krb: MODIFIED
E  [kube] 2019/05/23 07:48:42 review-62151-brok-mkip7n-shared-secrets.1-krb: Jobs active: 1, jobs failed: 0, jobs succeeded: 0
E  [kube] 2019/05/23 07:49:07 Add/Modify event for review-62151-brok-mkip7n-shared-secrets.1-krb: MODIFIED
E  [tiller] 2019/05/23 07:49:07 hooks complete for pre-install review-62151-brok-mkip7n
E  [tiller] 2019/05/23 07:49:07 deleting pre-install hook review-62151-brok-mkip7n-shared-secrets for release review-62151-brok-mkip7n due to "hook-succeeded" policy
E  [kube] 2019/05/23 07:49:07 Starting delete for "review-62151-brok-mkip7n-shared-secrets" ServiceAccount
E  [tiller] 2019/05/23 07:49:07 deleting pre-install hook review-62151-brok-mkip7n-shared-secrets for release review-62151-brok-mkip7n due to "hook-succeeded" policy
E  [kube] 2019/05/23 07:49:07 Starting delete for "review-62151-brok-mkip7n-shared-secrets" Role
E  [tiller] 2019/05/23 07:49:07 deleting pre-install hook review-62151-brok-mkip7n-shared-secrets for release review-62151-brok-mkip7n due to "hook-succeeded" policy
E  [kube] 2019/05/23 07:49:07 Starting delete for "review-62151-brok-mkip7n-shared-secrets" RoleBinding
E  [tiller] 2019/05/23 07:49:07 deleting pre-install hook review-62151-brok-mkip7n-shared-secrets for release review-62151-brok-mkip7n due to "hook-succeeded" policy
E  [kube] 2019/05/23 07:49:07 Starting delete for "review-62151-brok-mkip7n-shared-secrets" ConfigMap
E  [tiller] 2019/05/23 07:49:07 deleting pre-install hook review-62151-brok-mkip7n-shared-secrets.1-krb for release review-62151-brok-mkip7n due to "hook-succeeded" policy
E  [kube] 2019/05/23 07:49:07 Starting delete for "review-62151-brok-mkip7n-shared-secrets.1-krb" Job
E  [storage] 2019/05/23 07:49:07 getting release history for "review-62151-brok-mkip7n"
E  [storage] 2019/05/23 07:49:07 creating release "review-62151-brok-mkip7n.v1"
E  [kube] 2019/05/23 07:49:07 building resources from manifest
E  [kube] 2019/05/23 07:49:08 creating 67 resource(s)
E  [tiller] 2019/05/23 07:49:08 warning: Release "review-62151-brok-mkip7n" failed: object is being deleted: persistentvolumeclaims "review-62151-brok-mkip7n-minio" already exists
E  [storage] 2019/05/23 07:49:08 updating release "review-62151-brok-mkip7n.v1"
E  [tiller] 2019/05/23 07:49:08 failed install perform step: release review-62151-brok-mkip7n failed: object is being deleted: persistentvolumeclaims "review-62151-brok-mkip7n-minio" already exists
  1. Deploy the tiller deployment to a dedicated node pool for more stability.
  2. Remove the set -e / set +e from scripts/review_apps/review-apps.sh which could cause problems.
  3. Move the individual deployments steps in the review-deploy job definition directly to better separate the potential cause of failure.

Does this MR meet the acceptance criteria?

Conformity

Closes https://gitlab.com/gitlab-org/gitlab-ce/issues/62161, https://gitlab.com/gitlab-org/gitlab-ce/issues/63639.

Edited by Rémy Coutable

Merge request reports