secret detection should dynamically set git depth in MR pipelines
Summary
When secret detection runs in merge request pipeline, we currently fetch all ref history with maximum depth of 2147483647. However, for large repositories this can take a long time. We need to optimize the git fetch for jobs in MR pipelines so that it sets git depth dynamically to reliably cover new commits added to branch source and reduce the git fetch time.
Steps to reproduce
- Create a project
- Configure secret detection to run on MR pipelines in that project by using
latesttemplate:
include:
- template: Jobs/Secret-Detection.latest.gitlab-ci.yml
- Create an MR
- It won't be immediately visible what depth the analyzer uses. To see that, you can trigger an error by removing the remote causing the
git fetchcommand to trigger an error:
secret_detection:
before_script:
- git remote remove origin
Example Project
This job ran in an MR pipeline. You can see there that the analyzer tries to fetch all branch history:
running git command: /usr/bin/git fetch --depth=2147483647 origin kategrechishkina-main-patch-a49d
While the branch (and MR) only has 3 unique commits.
What is the current bug behavior?
All branch history is fetched
What is the expected correct behavior?
Only the newly-added commits are fetched
Output of checks
This bug happens on GitLab.com
Possible fixes
Edited by Kate Grechishkina