Skip to content

Update RPC to determine when a secrets check is needed

Problem to solve

Right now, secret push protection blocks automated workflows unnecessarily.

Sample scenario

A user pushes commits into a new branch and creates a merge request, there's absolutely no issue with running secret push protection on those commits before they're committed to the repository. However, when the merge request is being merged, it gets added to a merge train and:

  • When a merge train is created, UserMergeBranch() RPC1. is called from the monolith. The RPC in turn invokes access checks on the Rails side, which result in secrets push check being called.
  • In that case, secret push protection runs on the merge commit. This doesn't make sense because the code is already committed to the repository – so if there were any secrets, they have already been leaked by now.
  • There are also instances where secret push protection will block the merge train because the merge commit included secrets that had been skipped and now they're being detected again and the merge is blocked.

Secret push protection should not run for these automated workflows. It was built for protecting secrets from being committed to a repository, when the code already has a secret in, there's no point in running secret push protection.

Today, we're unable to identify when secret push protection is called from an actual user push vs. some gitaly RPC running access checks to confirm a merge can take place, i.e. UserMergeBranch() RPC. We had to use the protocol field to get unblocked earlier, but it only helps us to confirm whether the push took place over http/s or ssh vs. web. Without knowing which RPC called /internal/allowed, we cannot be certain secret push protection is only invoked for intended workflows.

1. A GitLab RPC (Remote Procedure Call) is a high-level access mechanism to Git repositories. Gitaly, a component of GitLab, provides these RPCs to read and write Git data. GitLab uses these RPCs to coordinate Git repository storage and retrieval across various GitLab components.

Overview

Gitaly performs /allowed request to Gitlab Rails to verify whether an RPCs can be performed to mutate a Git repository.

Currently, it always sends git-receive-pack and action params which is used for access check and in the places like:

The action is checked to be either of these values: git-receive-pack/git-upload-pack/git-upload-archive

Proposal

RPC has more granular instructions to determine when a secrets check is needed.

Pass enable_secrets_check to RPCs where secret check is needed.

Gitaly::UserCommitFilesRequestHeader.new(
  repository: @gitaly_repo,
  user: Gitlab::Git::User.from_gitlab(user).to_gitaly,
  branch_name: encode_binary(branch_name),
  commit_message: encode_binary(commit_message),
  commit_author_name: encode_binary(author_name),
  commit_author_email: encode_binary(author_email),
  start_branch_name: encode_binary(start_branch_name),
  start_repository: start_repository&.gitaly_repository,
  force: force,
  start_sha: encode_binary(start_sha),
  sign: sign,
  expected_old_oid: target_sha,
  timestamp: Google::Protobuf::Timestamp.new(seconds: Time.now.utc.to_i),
  gitaly_context: { enable_secrets_check: true } # <--- define context
)

Apply secrets check when protocol is http or ssh, or when Gitaly context includes enable_secrets_check key.

def http_or_ssh_protocol?
  %w[http ssh].include?(changes_access.protocol)
end

def use_diff_scan?
  http_or_ssh_protocol? || secret_check_enabled?
end

def secret_check_enabled?
  changes_access.gitaly_context[:enable_secrets_check] == true
end

source

Edited by Amar Patel