Create a script/binary to make grabbing of secrets/configurations more elegant
With the introduction of helmfile
for https://gitlab.com/gitlab-com/gl-infra/k8s-workloads/gitlab-com, the ability to grab secrets and non-secret configurations is compressed into a very complex single line of helm templating. Example:
{{ exec "bash" (list "-c" (print "echo " $gkms_omnibus_secrets_base64 "| base64 -d | jq -r '.\"gitlab-server\".\"google-creds\".json_base64'")) | b64dec | trimSuffix "\n" | indent 16 }}
And the above requires some setup, namely the variable gkms_omnibus_secrets_base64
, in the above example:
{{- $gkms_omnibus_secrets_base64 := (exec "bash" (list "-c" (print "gsutil cat gs://gitlab-" $gkms_source "-secrets/gitlab-omnibus-secrets/" $gkms_source ".enc | gcloud --project " .Environment.Values.google_project " kms decrypt --locat ion global --keyring=gitlab-secrets --key " $gkms_source " --ciphertext-file=- --plaintext-file=-")) | b64enc | quote) }}
In the above examples we are asking helm to execute:
bash "-c" "gsutil cat gs://gitlab-pre-secrets/gitlab-omnibus-secrets/pre.enc | gcloud --project gitlab-pre kms decrypt --location global --keyring=gitlab-secrets --key pre --ciphertext-file=- --plaintext-file=-"
To populate the variable gkms_omnibus_secrets_base64
, followed by
bash "-c" echo "$gkms_omnibus_secrets_base64"| base64 -d | jq -r '."gitlab-server"."google-creds".json_base64'
To populate our secret value.
It would be nicer if we can make this less fragile. The amount of shelling out required is high. Using a script to handle the above would be much easier and would allow us to test outside of the use of helmfile
when needed. If the above were to fail, helmfile
does not print any sort of easy to grasp error message. An example of this:
in ./helmfile.yaml: in .helmfiles[0]: in releases/gitlab-secrets.yaml: failed to read gitlab-secrets.yaml: reading document at index 1: yaml: invalid leading UTF-8 octet
Running with LOG_LEVEL=debug
barely helps as this requires the person troubleshooting to hunt for the error mangled yaml.
Utilize this issue to determine/build a script that helmfile
can execute that performs the setup and grabbing of the appropriate requested value. It would be best to write this in either golang or ruby and the script should have appropriate tests.
When built we can include this in our base image for helmfile
to easily consume.