Setting `KUBERNETES_TERMINATIONGRACEPERIODSECONDS` in the helm chart sets `deletionGracePeriodSeconds` on the runner pods

Summary

I am experiencing the same issues described in this ticket: #3247 (closed)

I tried the recommendation in this comment: #3247 (comment 309641116)

So i added this to my gitlab runners helm chart values.yaml:

envVars:
   - name: KUBERNETES_TERMINATIONGRACEPERIODSECONDS
     value: 600

Steps to reproduce

envVars:
   - name: KUBERNETES_TERMINATIONGRACEPERIODSECONDS
     value: 600

This affects all jobs, regardless of definition.

Actual behavior

New runner pods have terminationGracePeriodSeconds: 600 AND deletionGracePeriodSeconds: 600 set. This is NOT expected, and causes our cluster to scale wildly as all pods now take 10 mins to delete.

Expected behavior

Newly created runner pods should have terminationGracePeriodSeconds defined as 600

Relevant logs and/or screenshots

Note:

• Creation Time
• Deletion Time
• Real Time (bottom right)

Pod is scheduled for deletion, even though job succeeded

Environment description

Gitlab runner helm chart (kubernetes executor) and Google Kubernetes Engine cluster with Autoscaling.

values.yaml contents

## The GitLab Server URL (with protocol) that want to register the runner against
## ref: https://docs.gitlab.com/runner/commands/README.html#gitlab-runner-register
##
gitlabUrl: https://gitlab.com/

## Configure the maximum number of concurrent jobs
## ref: https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section
##
concurrent: 50

## Defines in seconds how often to check GitLab for a new builds
## ref: https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section
##
checkInterval: 5


## Configure GitLab Runner's logging level. Available values are: debug, info, warn, error, fatal, panic
## ref: https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-global-section
##
logLevel: info

## For RBAC support:
rbac:
  create: true
  ## Define specific rbac permissions.
  resources: ["pods", "pods/exec", "secrets"]
  verbs: ["get", "list", "watch", "create", "patch", "delete"]
  ## Run the gitlab-bastion container with the ability to deploy/manage containers of jobs
  ## cluster-wide or only within namespace
  clusterWideAccess: false

resources:
  limits:
  requests:
    memory: 128Mi
    cpu: 100m
    memory: 256Mi
    cpu: 200m

## Specify a imagePullPolicy
## 'Always' if imageTag is 'latest', else set to 'IfNotPresent'
## ref: http://kubernetes.io/docs/user-guide/images/#pre-pulling-images
##
# imagePullPolicy: Always

## Configure environment variables that will be present when the registration command runs
## This provides further control over the registration process and the config.toml file
## ref: `gitlab-runner register --help`
## ref: https://docs.gitlab.com/runner/configuration/advanced-configuration.html
##
envVars:
   - name: KUBERNETES_TERMINATIONGRACEPERIODSECONDS
     value: 600


## Configuration for the Pods that the runner launches for each new job
##
runners:

  ## Run all containers with the privileged flag enabled
  ## This will allow the docker:stable-dind image to run if you need to run Docker
  ## commands. Please read the docs before turning this on:
  ## ref: https://docs.gitlab.com/runner/executors/kubernetes.html#using-docker-dind
  ##
  privileged: true

  ## The amount of time, in seconds, that needs to pass before the runner will
  ## timeout attempting to connect to the container it has just created.
  ## ref: https://docs.gitlab.com/runner/executors/kubernetes.html
  pollTimeout: 600

  ## Namespace to run Kubernetes jobs in (defaults to 'default')
  ##
  namespace: gitlab-runners

  ## Gitlab runner tags to identify jobs to run
  runUntagged: true



  ## Build Container specific configuration
  ##
  builds:
    cpuRequests: 1
    cpuLimit: 2
    cpuRequestsOverwriteMaxAllowed: 8
    cpuLimitOverwriteMaxAllowed: 16
    memoryRequests: 2Gi
    memoryLimit: 4Gi
    memoryRequestsOverwriteMaxAllowed: 8Gi
    memoryLimitOverwriteMaxAllowed: 16Gi

  ## Service Container specific configuration
  ##
  services:
    cpuRequests: 250m
    memoryRequests: 512Mi
    # cpuLimit: 400m
    # memoryLimit: 256Mi

  ## Helper Container specific configuration
  ##
  helpers:
    cpuRequests: 20m
    memoryRequests: 128Mi
    cpuLimit: 100m
    memoryLimit: 256Mi```
</details>

### Used GitLab Runner version

kubernetes executor

`gitlab-runner:alpine-v13.0.1`

Running with gitlab-ci-multi-runner 1.4.2 (bcc1794f) Using Docker executor with image golang:1.8 ...

-->

## Possible fixes

<!--
(If you can, link to the line of code that might be responsible for the problem)
--->