Improve support for `git clean`
What does this MR do?
Adds support for GIT_CLEAN_FLAGS
to allow customize the behavior of git clean
.
This makes GIT_CLEAN_FLAGS
:
- when not defined to use
-ffdx
, - when
none
to disablegit clean
.
This is configurable via following statements in .gitlab-ci.yml
:
variables:
missing GIT_CLEAN_FLAGS # use default, the `-ffdx`
GIT_CLEAN_FLAGS: "" # use default, the `-ffdx`
GIT_CLEAN_FLAGS: -ffdx -e cache/
GIT_CLEAN_FLAGS: none # disable git clean
This MR also improves performance of checkout in very big (a lot of files) repositories:
- We do not use
git reset --hard
as this is not needed, because we already dogit checkout -f
, - Moves
git clean
aftergit checkout -f
as this is desired behavior, - We remove one I/O intensive operation (
git reset --hard
), reducing I/O by 33%:git checkout -f
andgit clean
only which has the same I/O footprint.
Closes #4109 (closed)
Why was this MR needed?
Make git clean
behavior flexible and performant.
Does this MR meet the acceptance criteria?
-
Documentation created/updated -
Added tests for this feature/bug -
In case of conflicts with master
- branch was rebased
What are the relevant issue numbers?
Related to https://forum.gitlab.com/t/disable-git-clean-between-jobs/7494
Merge request reports
Activity
mentioned in merge request !1228 (closed)
changed milestone to %11.10
added bugperformance + 1 deleted label
- Resolved by Kamil Trzciński
mentioned in issue #2900
- Resolved by Kamil Trzciński
@ayufan I found some that may be related but I opened this issue explicitly to track this exact functionality: #4109 (closed)
- Resolved by Steve Xuereb
@ayufan would it make sense to move
git reset --hard
inside of a different MR? This would make it easier to revert is something goes wrong. It is a small change but I think it's better track different behaviors changes in different MRs.
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
- Resolved by Steve Xuereb
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
Few suggestions/question back to you @ayufan
assigned to @ayufan
Discussion to open an MR for docs in gitlab-ce so we don't forget. I think best place to add this kind of documentation would be in https://docs.gitlab.com/ee/ci/yaml/#variables
added 12 commits
-
d9c1ddff...fe2fe8a6 - 10 commits from branch
master
- e6f8474a - Optimise checkout strategy
- 0ce1119a - Improve support for git clean
-
d9c1ddff...fe2fe8a6 - 10 commits from branch
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
assigned to @ayufan
assigned to @nolith
added [Deprecated] Category:Runner devopsverify labels
- Resolved by Kamil Trzciński
@ayufan a couple of minor nitpicks.
assigned to @ayufan
assigned to @nolith
enabled an automatic merge when the pipeline for f4c1feb2 succeeds
mentioned in commit ef40d5f9
@ayufan @brendan I think that with this MR being merged we can also close this issue: #1453 (closed)
mentioned in issue #1453 (closed)
maybe !772 (closed) could be closed as well
mentioned in merge request !772 (closed)
mentioned in merge request gitlab-com/www-gitlab-com!21184 (merged)
mentioned in issue gitlab-ci-runner#27 (moved)
@ayufan Is it possible to extend this feature to the submodule strategy? I've just tried in 11.10rc2 and it only works for the main branch. The fetch strategy of submodules keeps doing a forced git clean (even ignoring .gitignore).
Thank you.
@yperchy It's being handled in !1297 (closed) :)
@tmaczukin Thanks for answering :)
mentioned in issue #3473 (closed)
mentioned in issue #4175 (closed)
mentioned in epic &1238 (closed)
mentioned in merge request !925 (closed)