Skip to content

Use batched new blobs check [RUN ALL RSPEC] [RUN AS-IF-FOSS]

What does this MR do?

The Git access checks support checking the size of pushed objects. If we have a quarantine environment (which is always the case when access checks are executed via git receive-pack's hooks), then we simply derive the size from the quarantined objects. Otherwise, we use get_new_blobs, which does a git rev-list $REVISION --not --all.

Using git-rev-list(1) with --not --all can be a very expensive operation depending on the repository's shape. Especially when it's a biggish monorepo with lots of references, this query can easily take tens of seconds. Given that we call the RPC once per change, this thus roughly scales O(len(changes) * len(existing_refs)).

To improve this situation, Gitaly has implemented a new list_blobs() RPC which takes a set of revisions. Like this, we can batch all calls into a single one and thus avoid some of the overhead if there are multiple changes at once. The new code which does that is currently implemented behind a feature flag.

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

Does this MR contain changes to processing or storing of credentials or tokens, authorization and authentication methods or other items described in the security review guidelines? If not, then delete this Security section.

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by Patrick Steinhardt

Merge request reports