## Status Update (2023-03-22) You can now apply this configuration using the custom podspec feature. The feature is currently behind a feature flag `FF_USE_ADVANCED_POD_SPEC_CONFIGURATION` ### Config Example ``` [[runners]] environment = ["FF_USE_ADVANCED_POD_SPEC_CONFIGURATION=true"] [runners.kubernetes] image = "alpine" [[runners.kubernetes.pod_spec]] name = "tester" patch_type = "strategic" patch = ''' initContainers: - name: "init-permissions" resources: limits: cpu: "..." requests: cpu: "..." ''' ``` We are also capturing any feedback wrt this feature in the issue https://gitlab.com/gitlab-org/gitlab-runner/-/issues/29659+ [Documentation](https://docs.gitlab.com/runner/executors/kubernetes.html#overwrite-generated-pod-specifications-alpha) ## Status update (2023-02-25) In 15.10, (2023-03-22), we are working on releasing (behind a feature flag) a new [feature](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/3114) that will enable users to pass a [PodSpec](https://v1-18.docs.kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#podspec-v1-core) in either `JSON` or `YAML`. This will allow the setting of resource limits for init containers on the runner worker pods. Note - this won't enable setting those limits for the runner manager pod. ## Summary <!-- Briefly summarize the bug --> As per this [ticket](https://gitlab.zendesk.com/agent/tickets/245712)(internal), the customer has set up a security policy to check that all containers have resources limits set. But on setting the limits via the `values.yaml` file as per below, the init container does not pick up these settings and instead they encounter the error below ``` ERROR: Job failed (system failure): prepare environment: setting up build pod: admission webhook "validation.gatekeeper.sh" denied the request: [azurepolicy-container-limits-58cd4ab40ea8e76caf7d] container <init-permissions> has no resource limits. Check https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading for more information ``` ## Actual behaviour On investigating further I checked the generated container while a pipeline was running and indeed the resource limits are not set for the `init-permissions` container <!-- What actually happens --> ## Expected behaviour It should be possible to configure resource limits for the `init-permissions` container <!-- What you should see instead --> ## Relevant logs and/or screenshots <!-- Paste the job logs inside of the code blocks (```) below so it would be easier to read. --> <details> <summary> kubectl describe pod runner-ogbuzdkp-project-2-concurrent-0tsc2c </summary> I described the generated container while a pipeline was running and indeed this is the case ```yaml Name: runner-ogbuzdkp-project-2-concurrent-0tsc2c Namespace: gitlab ... Init Containers: init-permissions: Container ID: containerd://491f8f1ae11ffa356d6e4ec52985a452220985f48f8f70d5e0d76ef8da85ed04 Image: registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper:x86_64-e0218c92 Image ID: registry.gitlab.com/gitlab-org/gitlab-runner/gitlab-runner-helper@sha256:7c6b865d5154255db981e1947a3cddf96f64207ef0536658ef10975c8e6162cf Port: <none> Host Port: <none> Command: sh -c touch /logs-2-17/output.log && (chmod 777 /logs-2-17/output.log || exit 0) State: Terminated Reason: Completed Exit Code: 0 Started: Fri, 22 Oct 2021 12:54:22 +0200 Finished: Fri, 22 Oct 2021 12:54:22 +0200 Ready: True Restart Count: 0 Environment: <none> Mounts: /builds from repo (rw) /logs-2-17 from logs (rw) /scripts-2-17 from scripts (rw) /var/run/secrets/kubernetes.io/serviceaccount from default-token-sskf9 (ro) Containers: build: Container ID: containerd://0f128f78b3313a4eaacae97a8a3fb9b7f93cd372daf892451554cc54dad8e379 Image: ubuntu:18.04 Image ID: docker.io/library/ubuntu@sha256:0fedbd5bd9fb72089c7bbca476949e10593cebed9b1fb9edf5b79dbbacddd7d6 Port: <none> Host Port: <none> Command: sh -c if [ -x /usr/local/bin/bash ]; then exec /usr/local/bin/bash elif [ -x /usr/bin/bash ]; then exec /usr/bin/bash elif [ -x /bin/bash ]; then exec /bin/bash elif [ -x /usr/local/bin/sh ]; then exec /usr/local/bin/sh elif [ -x /usr/bin/sh ]; then exec /usr/bin/sh elif [ -x /bin/sh ]; then exec /bin/sh elif [ -x /busybox/sh ]; then exec /busybox/sh else echo shell not found exit 1 fi State: Running Started: Fri, 22 Oct 2021 12:54:23 +0200 Ready: True Restart Count: 0 Limits: cpu: 250m memory: 256Mi Requests: cpu: 100m memory: 128Mi ``` </details> ## Environment description <!-- Are you using shared Runners on GitLab.com? Or is it a custom installation? Which executors are used? Please also provide the versions of related tools like `docker info` if you are using the Docker executor. --> <!-- Please add the contents of `config.toml` inside of the code blocks (```) below, remember to remove any secret tokens! --> <details> <summary> config.toml contents </summary> ```toml runners: locked: false config: | [[runners]] [runners.kubernetes] image = "ubuntu:18.04" cpu_request = "100m" cpu_limit = "250m" helper_cpu_request = "100m" helper_cpu_limit = "250m" service_cpu_request = "100m" service_cpu_limit = "250m" memory_request = "128Mi" memory_limit = "256Mi" helper_memory_request = "128Mi" helper_memory_limit = "256Mi" service_memory_request = "128Mi" service_memory_limit = "256Mi" ``` </details> ### Used GitLab Runner version <!-- Please run and paste the output of `gitlab-runner --version`. If you are using a Runner where you don't have access to, please paste at least the first lines the from build log, like: --> kubectl exec -it gitlab-gitlab-runner-6448ccffb8-x7dk5 -- gitlab-runner --version ``` Defaulted container "gitlab-gitlab-runner" out of: gitlab-gitlab-runner, configure (init) Version: 14.3.2 Git revision: e0218c92 Git branch: 14-3-stable GO version: go1.13.8 Built: 2021-09-30T16:10:52+0000 OS/Arch: linux/amd64 ``` ## Possible fixes Propagate the resource settings while [building the container](https://gitlab.com/gitlab-org/gitlab-runner/-/blob/main/executors/kubernetes/kubernetes.go#L445) <!-- (If you can, link to the line of code that might be responsible for the problem) --->