Review / Staging job tries to delete a deployment which never exists and fails.
Hello all,
i tried to setup auto-devops for our projects with a production, staging and review environment. Currently we are using version 2.11.1 of the Chart (because we need the feature of creating annotated service accounts)
When adding the chart via merge request we get the following error message in each project we tried:
Error from server (NotFound): deployments.apps "review-hvoss-mast-d32yec" not found
After contacting gitlab support and investigating the issue further, my guess is that auto-devops tries to delete / remove (or do anything else) a deployment which never existed in first place (Because each review environment is new on first creation).
When creating a deployment manually with the same name the review job runs successful, but the deployment / image does not change.
Since i guess this is a bug in auto-devops i created this issue.
All logs are send within https://support.gitlab.com/hc/requests/221853
To provide further information about the current environment:
.gitlab/auto-deploy-values.yaml:
ingress:
tls:
enabled: false
annotations:
kubernetes.io/ingress.class: "nginx-internal"
livenessProbe:
path: "/actuator/health/liveness"
readinessProbe:
path: "/actuator/health/readiness"
serviceAccount:
name: product-284-staging-service-account
createNew: true
annotations:
eks.amazonaws.com/role-arn: "arn:aws:iam::<ID>:role/product-role"
chart/Chart.yaml:
apiVersion: v1
description: GitLab's Auto-deploy Helm Chart
name: auto-deploy-app
version: 2.11.1
icon: https://gitlab.com/gitlab-com/gitlab-artwork/raw/master/logo/logo-square.png
section of .gitlab-ci.yaml:
.auto-deploy:
image: "registry.gitlab.com/gitlab-org/cluster-integration/auto-deploy-image:v2.11.1"
Full job output:
Running with gitlab-runner 13.11.0 (7f7a4bb0)
on runner-gitlab-runner-76ff9b56b7-mx78q BFxd7pQs
Resolving secrets
00:00
Preparing the "kubernetes" executor
00:00
Using Kubernetes namespace: gitlab-managed-apps
Using Kubernetes executor with image registry.gitlab.com/gitlab-org/cluster-integration/auto-deploy-image:v2.11.1 ...
Preparing environment
00:04
WARNING: Pulling GitLab Runner helper image from Docker Hub. Helper image is migrating to registry.gitlab.com, for more information see https://docs.gitlab.com/runner/configuration/advanced-configuration.html#migrate-helper-image-to-registrygitlabcom
Waiting for pod gitlab-managed-apps/runner-bfxd7pqs-project-284-concurrent-0h8s8t to be running, status is Pending
Running on runner-bfxd7pqs-project-284-concurrent-0h8s8t via runner-gitlab-runner-76ff9b56b7-mx78q...
Getting source from Git repository
00:00
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/datastreamloader/mapper/.git/
Created fresh repository.
Checking out 9a5c3a45 as hvoss-master-patch-08784...
Skipping Git submodules setup
Executing "step_script" stage of the job script
00:06
$ auto-deploy check_kube_domain
$ auto-deploy download_chart
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /builds/datastreamloader/mapper.tmp/KUBECONFIG
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /builds/datastreamloader/mapper.tmp/KUBECONFIG
"stable" has been added to your repositories
Download skipped. Using the chart at local path 'chart/'...
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /builds/datastreamloader/mapper.tmp/KUBECONFIG
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /builds/datastreamloader/mapper.tmp/KUBECONFIG
$ auto-deploy ensure_namespace
NAME STATUS AGE
mapper-284-review-hvoss-mast-jmcbcy Active 12s
$ auto-deploy initialize_tiller
Deprecated: Helm 3 does not have Tiller.
$ auto-deploy create_secret
Create secret...
secret/gitlab-registry-datastreamloader-mapper replaced
$ auto-deploy deploy
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /builds/datastreamloader/mapper.tmp/KUBECONFIG
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /builds/datastreamloader/mapper.tmp/KUBECONFIG
Validating chart version...
Fetching the previously deployed chart version...
Fetching the deploying chart version... v2.11.1
secret/review-hvoss-mast-jmcbcy-secret replaced
Using helm values file '.gitlab/auto-deploy-values.yaml'
Deploying new stable release...
WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /builds/datastreamloader/mapper.tmp/KUBECONFIG
WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /builds/datastreamloader/mapper.tmp/KUBECONFIG
Release "review-hvoss-mast-jmcbcy" does not exist. Installing it now.
NAME: review-hvoss-mast-jmcbcy
LAST DEPLOYED: Thu Jul 8 07:15:21 2021
NAMESPACE: mapper-284-review-hvoss-mast-jmcbcy
STATUS: deployed
REVISION: 1
TEST SUITE: None
Error from server (NotFound): deployments.apps "review-hvoss-mast-jmcbcy" not found
Uploading artifacts for failed job
00:00
Uploading artifacts...
WARNING: environment_url.txt: no matching files
WARNING: tiller.log: no matching files
ERROR: No files to upload
Cleaning up file based variables
00:01
ERROR: Job failed: command terminated with exit code 1
Edit:
After testing the staging job in multiple projects:
Error from server (NotFound): deployments.apps "staging" not found
Same issue.