Proposal: Refactor Credentials to be origin agnostic using initContainer and emptyDir
This comes from a thought that has been bouncing around in my head since I re-worked the Redis chart before the holidays, so I will attempt to provide a clear explanation.
TL;DR
One of the pain points that we are consistently made aware of is how secrets aren't secret. At least not in the way that they are when protected via Vault and the like.
Using an initContainer to write the necessary values to an emptyDir's Pod-local tmpfs instead of directly mapping
secrets should allow interchanging of the sources. E.g. Vault, Google KMS, or secrets should all be possible and interchangeable, with the appropriate changes to which initContainer is used.
Explanation
When working to provide proper password security in the Redis chart, I had a problem where I could not directly pass the gitlab-redis secret's password into the Redis container without exposing it as an environment variable. Using environment variables is counter to the methods we're looking to use, let the ease with which one can examine the environment of a container. To resolve this I ended up using an initContainer that received the secrets mount and a blank emptyDir which it then wrote the configuration file into. The Redis container then receives that populated emptyDir and persistent disk mount. This keeps the Redis container itself from having to know about the secrets, ConfigMap, or where the secure values came from in any way.
The initContainer I have created in the Redis chart uses busybox's sh command to execute a configure shell script from the ConfigMap that appends the password to the redis.conf entry, and renders that into the emptyDir mounted at /redis. The Redis container itself, mounts that emptyDir, now containing redis.conf complete with password, at /etc/redis and points the redis-server process to that configuration file.