GET_SOURCES_ATTEMPTS retries too verbose

Summary

When GET_SOURCES_ATTEMPTS is configured for a CI job, multiple attempts are made to fetch sources. After some failure, runner clears the presumably corrupted sources. This removal is verbose: a line is printed to job log for each file removed. In large, monorepo-style repositories, this quickly overwhelms job log size limit.

Steps to reproduce

  1. Set variables: { GET_SOURCES_ATTEMPTS: 3 } for a job
  2. Run a job once successfully so it populates the workspace on the runner
  3. Cause the job to fail to fetch the sources. For example, make it fetch a submodule that the pipeline owner can't access.

Actual behavior

Lines like rm 'path/to/file' flood the head of job log, one for each file that was present in the workspace. In large enough repositories, this spam can consume log size limit.

Expected behavior

It is possible to silence individual file reports. I would even find it reasonable to disable them by default or completely, because I'm not sure when they can provide valuable insight.

Relevant logs and/or screenshots

job log

Here, the fetch was failing due to a permissions issue. The permissions issue is irrelevant, but it provides a nice example for what the logs look like.

In case the failure was temporary, the job could have succeeded with the fetch eventually, but its logs would not be captured.

Running with gitlab-runner 18.11.2 (68229485)
[...]
Getting source from Git repository
Fetching changes with git depth set to 5...
Initialized empty Git repository in /builds/my-project/.git/
Created fresh repository.
Checking out dc9bc7ad as detached HEAD (ref is refs/merge-requests/4589/merge)...
Updating/initializing submodules with git depth set to 3...
Submodule 'my-submodule' (http://example.org/my-submodule.git) registered for path 'my-submodule'
Synchronizing submodule url for 'my-submodule'
Cloning into '/builds/my-project/my-submodule'...
remote: Insufficient permissions to pull from the repository of project #3721.
fatal: unable to access 'http://example.org/my-submodule.git/': The requested URL returned error: 403
fatal: clone of 'http://example.org/my-submodule.git' into submodule path '/builds/my-project/my-submodule' failed
Failed to clone 'my-submodule'. Retry scheduled
Cloning into '/builds/my-project/my-submodule'...
remote: Insufficient permissions to pull from the repository of project #3721.
fatal: unable to access 'http://example.org/my-submodule.git/': The requested URL returned error: 403
fatal: clone of 'http://example.org/my-submodule.git' into submodule path '/builds/my-project/my-submodule' failed
Failed to clone 'my-submodule' a second time, aborting
Updating submodules failed. Retrying...
Synchronizing submodule url for 'my-submodule'
Cloning into '/builds/my-project/my-submodule'...
remote: Insufficient permissions to pull from the repository of project #3721.
fatal: unable to access 'http://example.org/my-submodule.git/': The requested URL returned error: 403
fatal: clone of 'http://example.org/my-submodule.git' into submodule path '/builds/my-project/my-submodule' failed
Failed to clone 'my-submodule'. Retry scheduled
Cloning into '/builds/my-project/my-submodule'...
remote: Insufficient permissions to pull from the repository of project #3721.
fatal: unable to access 'http://example.org/my-submodule.git/': The requested URL returned error: 403
fatal: clone of 'http://example.org/my-submodule.git' into submodule path '/builds/my-project/my-submodule' failed
Failed to clone 'my-submodule' a second time, abortingRetrying in 5s
Deleting all tracked and untracked files due to source fetch failure
Deleting tracked and untracked files...
rm 'a/a/a/a/0001.txt'
rm 'a/a/a/a/0002.txt'
rm 'a/a/a/a/0003.txt'
[and so on for every file in the repository]
rm 'd/z/e/e/7882.txt'
rm 'd/z/e/e/7
Job's log exceeded limit of 4194304 bytes.
Job execution will continue but no more output will be collected.

Environment description

This is a self-hosted GitLab EE installation.

Used GitLab Runner version

Running with gitlab-runner 18.11.2

Possible fixes

From browsing the codebase, we found this: shells/abstract.go:572 uses git rm, which is verbose by default:

--quiet

git rm normally outputs one line (in the form of an rm command) for each file removed. This option suppresses that output.

- git rm documentation

I don't think verbose output here is at all desirable, as long as errors are still visible. I struggle to come up with a scenario where these lines would provide some valuable insights. The fix could then be as easy as hardcoding the --quiet flag. However, I can see how getting rid of the output completely may be undesirable.