Cert manager templates failing, trying to define projected volume for custom-ca-certificates
Summary
When selecting to install the cert manager, instead of using the self-signed option, deployments are being defined with a broken "custom-ca-certificates" volume. When creating the templates using the self-signed option, a deployment, e.g. gitlab-gitlab-shell is defined with a volume:
- name: custom-ca-certificates
projected:
defaultMode: 0400
sources:
- secret:
name: gitlab-wildcard-tls-ca
---
when building the same templates but using the certmanager definitions instead of self-signed, the same volume is still being defined, but the definition is now broken, sources is missing, causing kubectl to error.
- name: custom-ca-certificates
projected:
defaultMode: 0400
sources:
---
i.e. the path to the secret is missing, though I suspect because we're now trying to use cert-manager, it shouldn't be defined at all.
this causes the config to be invalid and prevents deployment.
e.g.
error validating "k8s-ssl-cert-manager.yaml": error validating data: ValidationError(Deployment.spec): missing required field "selector" in io.k8s.api.apps.v1beta2.DeploymentSpec; if you choose to ignore these errors, turn validation off with --validate=false
Steps to reproduce
helm template --name gitlab --namespace gitlab-ssl ./gitlab
--set certmanager.install=true
--set global.ingress.configureCertmanager=true
--set certmanager-issuer.email="email@example.com"
--set shared-secrets.enabled=false \
k8s-ssl-cert_manager.yaml
kubectl apply -f k8s-ssl-cert_manager.yaml
(compare to a file generated using:
helm template --name gitlab --namespace gitlab-ssl ./gitlab
--set certmanager.install=false
--set global.ingress.configureCertmanager=false
--set certmanager-issuer.email="email@example.com"
--set shared-secrets.enabled=true \
k8s-ssl-self_signed.yaml
which I believe is the "opposite" config to be used for self-signed generation,, and which works, except it prevents gitlab-runner from working. )
Configuration used
Full configs files attached.
helm template --name gitlab --namespace gitlab-ssl ./gitlab \
--set certmanager.install=true \
--set global.ingress.configureCertmanager=true \
--set certmanager-issuer.email="email@example.com" \
--set shared-secrets.enabled=false \
> k8s-ssl-cert_manager.yaml
Current behaviour
kubectl apply -f k8s-ssl-cert_manager.yaml
.
.
.
error: error validating "k8s-ssl-cert_manager.yaml": error validating data: ValidationError(Deployment.spec.template.spec.volumes[4].projected): missing required field "sources" in io.k8s.api.core.v1.ProjectedVolumeSource;
Expected behavior
I should be able to apply a config generated using a "cert-manager, true" definition and therefore also be able to use gitlab-runner
Versions
- Chart: commit f37bbde9 (HEAD -> master, origin/master, origin/HEAD) tag 1.1.3
- Platform:
- Cloud:AWS
- Self-hosted: Kube via Kops on AWS
- Kubernetes: (
kubectl version
)- Client:v1.10.0
- Server:v1.10.2
- Helm: (
helm version
)- Client: 2.9.1
- Server: 2.9.1
Relevant logs
configmap "gitlab-certmanager-issuer-certmanager" created
configmap "gitlab-gitlab-runner" created
configmap "gitlab-gitaly" created
configmap "gitlab-gitlab-shell" created
configmap "gitlab-nginx-ingress-tcp" created
configmap "gitlab-migrations" created
configmap "gitlab-sidekiq-all-in-1" created
configmap "gitlab-sidekiq" created
configmap "gitlab-task-runner" created
configmap "gitlab-unicorn" created
configmap "gitlab-workhorse-config" created
configmap "gitlab-unicorn-tests" created
configmap "gitlab-minio-config-cm" created
configmap "gitlab-nginx-ingress-controller" created
configmap "gitlab-redis" created
configmap "gitlab-registry" created
persistentvolumeclaim "gitlab-minio" created
persistentvolumeclaim "gitlab-redis" created
serviceaccount "gitlab-certmanager-issuer" created
role.rbac.authorization.k8s.io "gitlab-certmanager-issuer" created
rolebinding.rbac.authorization.k8s.io "gitlab-certmanager-issuer" created
serviceaccount "gitlab-certmanager" created
serviceaccount "gitlab-gitlab-runner" created
serviceaccount "gitlab-nginx-ingress" created
customresourcedefinition.apiextensions.k8s.io "certificates.certmanager.k8s.io" configured
customresourcedefinition.apiextensions.k8s.io "clusterissuers.certmanager.k8s.io" configured
customresourcedefinition.apiextensions.k8s.io "issuers.certmanager.k8s.io" configured
clusterrole.rbac.authorization.k8s.io "gitlab-certmanager" configured
clusterrolebinding.rbac.authorization.k8s.io "gitlab-certmanager" configured
role.rbac.authorization.k8s.io "gitlab-nginx-ingress" created
rolebinding.rbac.authorization.k8s.io "gitlab-nginx-ingress" created
service "gitlab-gitaly" created
service "gitlab-gitlab-shell" created
service "gitlab-unicorn" created
service "gitlab-minio-svc" created
service "gitlab-nginx-ingress-controller-metrics" created
service "gitlab-nginx-ingress-controller" created
service "gitlab-nginx-ingress-controller-stats" created
service "gitlab-nginx-ingress-default-backend" created
service "gitlab-redis" created
service "gitlab-registry" created
pod "gitlab-unicorn-test-runner-qcjuj" created
deployment.apps "gitlab-certmanager" created
deployment.extensions "gitlab-gitlab-runner" created
error: error validating "k8s-ssl-cert_manager.yaml": error validating data: ValidationError(Deployment.spec.template.spec.volumes[4].projected): missing required field "sources" in io.k8s.api.core.v1.ProjectedVolumeSource; if you choose to ignore these errors, turn validation off with --validate=false
compared to output when using the self-signed:
kc apply -f k8s-ssl-self_signed.yaml -n gitlab-ssl
configmap "gitlab-gitlab-runner" created
configmap "gitlab-gitaly" created
configmap "gitlab-gitlab-shell" created
configmap "gitlab-nginx-ingress-tcp" created
configmap "gitlab-migrations" created
configmap "gitlab-sidekiq-all-in-1" created
configmap "gitlab-sidekiq" created
configmap "gitlab-task-runner" created
configmap "gitlab-unicorn" created
configmap "gitlab-workhorse-config" created
configmap "gitlab-unicorn-tests" created
configmap "gitlab-minio-config-cm" created
configmap "gitlab-nginx-ingress-controller" created
configmap "gitlab-redis" created
configmap "gitlab-registry" created
configmap "gitlab-shared-secrets" created
persistentvolumeclaim "gitlab-minio" created
persistentvolumeclaim "gitlab-redis" created
serviceaccount "gitlab-gitlab-runner" created
serviceaccount "gitlab-nginx-ingress" created
serviceaccount "gitlab-shared-secrets" created
role.rbac.authorization.k8s.io "gitlab-shared-secrets" created
rolebinding.rbac.authorization.k8s.io "gitlab-shared-secrets" created
role.rbac.authorization.k8s.io "gitlab-nginx-ingress" created
rolebinding.rbac.authorization.k8s.io "gitlab-nginx-ingress" created
service "gitlab-gitaly" created
service "gitlab-gitlab-shell" created
service "gitlab-unicorn" created
service "gitlab-minio-svc" created
service "gitlab-nginx-ingress-controller-metrics" created
service "gitlab-nginx-ingress-controller" created
service "gitlab-nginx-ingress-controller-stats" created
service "gitlab-nginx-ingress-default-backend" created
service "gitlab-redis" created
service "gitlab-registry" created
pod "gitlab-unicorn-test-runner-dyobr" created
deployment.extensions "gitlab-gitlab-runner" created
deployment.apps "gitlab-gitlab-shell" created
deployment.apps "gitlab-sidekiq-all-in-1" created
deployment.apps "gitlab-task-runner" created
deployment.apps "gitlab-unicorn" created
deployment.apps "gitlab-minio" created
deployment.apps "gitlab-nginx-ingress-controller" created
deployment.apps "gitlab-nginx-ingress-default-backend" created
deployment.apps "gitlab-redis" created
deployment.apps "gitlab-registry" created
statefulset.apps "gitlab-gitaly" created
job.batch "gitlab-migrations.0" created
job.batch "gitlab-minio-create-buckets.0" created
job.batch "gitlab-shared-secrets.0-vxa" created
job.batch "gitlab-shared-secrets.0-yvx-selfsign" created
ingress.extensions "gitlab-unicorn" created
ingress.extensions "gitlab-minio" created
ingress.extensions "gitlab-registry" created
role.rbac.authorization.k8s.io "gitlab-gitlab-runner" created
rolebinding.rbac.authorization.k8s.io "gitlab-gitlab-runner" created
horizontalpodautoscaler.autoscaling "gitlab-gitlab-shell" created
horizontalpodautoscaler.autoscaling "gitlab-sidekiq-all-in-1" created
horizontalpodautoscaler.autoscaling "gitlab-unicorn" created
horizontalpodautoscaler.autoscaling "gitlab-registry" created
poddisruptionbudget.policy "gitlab-gitaly" created
poddisruptionbudget.policy "gitlab-gitlab-shell" created
poddisruptionbudget.policy "gitlab-sidekiq" created
poddisruptionbudget.policy "gitlab-unicorn" created
poddisruptionbudget.policy "gitlab-minio-v1" created
poddisruptionbudget.policy "gitlab-nginx-ingress-controller" created
poddisruptionbudget.policy "gitlab-nginx-ingress-default-backend" created
poddisruptionbudget.policy "gitlab-redis-v1" created
poddisruptionbudget.policy "gitlab-registry-v1" created
In config files, compare: k8s-ssl-self_signed.yaml, lines 2204:2206 with k8s-ssl-cert_manager.yaml lines 2306:2307