Secrets manager is enabled but unusable when OpenBao not responding
Summary
The Secrets manager feature can be enabled in a project when OpenBao is offline and responding
Environment:
- GitLab Chart v9.6.1
- GitLab v18.6.1-ee
Steps to reproduce
- Stop OpenBao server.
- Go to the general settings of a project.
- Enable
Secrets manager.
See also steps to reproduce in the Rails console: #582364 (comment 2923469674)
What is the current bug behavior?
Feature appears as enabled BUT the feature is not provisioned in OpenBao. This results in a broken state that users can't fix.
What is the expected correct behavior?
- Settings page shows an error.
- Feature remains disabled in the UI.
Workaround
Delete project_secrets_manager record for the project, and enable again.
Relevant logs and/or screenshots
Possible fixes
Change SecretsManagerClient#make_request method to handle the case where the server is not responding.
Also consider 502 Gateway errors, etc.
These particular server errors should be logged.
Duo suggested this:
# In SecretsManagerClient#make_request
def make_request
# Add proper error handling for:
# - Connection timeouts
# - 502 Gateway errors
# - Server unavailability
# - Network failures
rescue Net::TimeoutError, Errno::ECONNREFUSED => e
# Log the error and raise a specific exception
# that the provision service can handle
end
Links
- https://gitlab.com/gitlab-org/gitlab/-/blob/dd8251acd165208d61067f399f25c4c52a6807cc/ee/lib/secrets_management/secrets_manager_client.rb#L351
- https://gitlab.com/gitlab-org/gitlab/-/blob/dd8251acd165208d61067f399f25c4c52a6807cc/ee/app/services/secrets_management/project_secrets_managers/provision_service.rb#L33
Edited by Fabien Catteau
