Support AWS secrets manager as alternative to Vault
Release notes
You could connect from GitLab CI/CD to cloud providers using environment variables. This approach works fine for many use cases but does not scale well if you need advanced permissions management or would prefer a signed, short-lived, contextualized connection to your cloud provider. GitLab shipped initial support for JWT token-based connection in version 12.10. The current development builds on the already well known CI_JOB_JWT token and introduced a CI_JOB_JWT_V2 variable that can be used to connect to AWS, GCP, Vault and likely many other cloud services.
The new variable is automatically injected into your pipeline, but is not backwards compatible with the current CI_JOB_JWT approach. As a result, together with the V2 release, we introduced a V1 version. Until GitLab 15.0 the CI_JOB_JWT will work without any changes. In GitLab 15.0 it will be changed to point to the V1 version, and the V1 version will remain in the product until GitLab 15.6 when both V1 and V2 will be removed and only the CI_JOB_JWT token will remain with the new value.
Problem to solve
We plan to implement Vault as a secrets store bundled with GitLab, but some customers will prefer to use an AWS-provided service. It will be possible to use our provided Vault with GKE (and any generic Kubernetes cluster), but they also provide their own first-party capability.
Intended users
Many developer and operations users will interact with this feature, but the primary integrator will be security operations teams.
Further details
This will provide more flexibility to teams, ensuring that GitLab is valuable even when not using our bundled secrets solution.
Proposal
We should allow for configuration to select a different secrets provider apart from the default provided Vault one. This should be implemented in a way that
Permissions and Security
Implementing this feature will require a comprehensive security evaluation by @gitlab-com/gl-security/appsec. The goal here is to improve security available both to GitLab itself, for CI/CD pipelines, and for users who want to store secrets in general associated with projects under development in GitLab.