Engineering Discovery for Vault Integration/Migration
Goal
- Discover the best backend architecture for Vault Integration/Migration in GitLab, that allows us to work on issues iteratively.
- The proposal should stick with MVC, not a moonshot, but enough extensible to cover the future vision of Vault integration/migration
Steps
- Vault Integration
- Vault Migration
Step: Vault Integration
Use cases
- Users can manage secrets of a their deployed application with the Vault.
- Users can inject external environment variables into pipeline jobs from Vault instead of using GitLab CI/CD environment variables
- https://gitlab.com/gitlab-org/gitlab-ce/issues/40720#note_150002234
Depths
- Project-level integration
- Group-level integration
- Instance-level integration
Installation methods
- Installing Vault via GKE integration (Issue: https://gitlab.com/gitlab-org/gitlab-ee/issues/9982)
- Manual
Technical Approaches
- GitLab-Rails Vault integration
- GitLab-Runner Vault integration
Approach: GitLab-Rails Vault integration
- Vault URL
- Vault Auth params (Token/App Role/etc)
- Provide API to update the Auth Params to automate token renewal process
Functionality | Path |
---|---|
Project-level CI/CD Secret Variables | gitlab/external/#{namespace}/#{project_name}/environment_variables |
Group-level CI/CD Secret Variables | gitlab/external/#{namespace}/environment_variables |
GitLab-Runner Vault integration
Approach:Users can specify external Vault instance into the GitLab-Runner's config.toml file.
Step: Vault Migration
Usecases
- Users can choose the secret store of Secret Variables between Database or Vault
- Discontinue using attr_encrypted
- Allow the
db_key_base
secret to be rotated - Job tokens
- GitLab CI/CD environment variables (Project/Group-level)
- Examine which secrets we can move to Vault from gitlab.rb
Installation support
- Bundle Vault with GitLab omnibus (Issue: omnibus-gitlab#4317 (closed))
Paths
Functionality | Path |
---|---|
Project-level CI/CD Secret Variables | gitlab/internal/#{namespace}/#{project_name}/environment_variables |
Group-level CI/CD Secret Variables | gitlab/internal/#{namespace}/environment_variables |
Job Token | gitlab/internal/job_tokens/#{job_id} |
user's passwords | gitlab/internal/users/#{user_id} |
Additional Thought
- Encryption as a Service: Transit Secrets Engine. Storing encrypted values in DB.
Questions
- How do we effectively use Dynamic Secrets? (e.g. tokens will expire in a day)
- Does GitLab need an authentication method?
Future Future vision
References
Edited by Shinya Maeda