Skip to content

WIP: Fix cleaning of removed sub-submodules

Cliff Woolley requested to merge cliffwoolley/gitlab-runner:cliffwoolley-recursive-submodule-clean into master

What does this MR do?

Fixes cleaning of removed sub-submodules when using the fetch strategy.

Why was this MR needed?

In https://gitlab.com/gitlab-org/gitlab-runner/-/blob/master/shells/abstract.go#L414-416 the sequence is effectively as follows for that function:

...
git submodule sync --recursive
git submodule foreach --recursive git clean -ffxd
git submodule foreach --recursive git reset --hard
git submodule update --init --recursive
...

What this does is:

  1. Update all the submodules' URLs, recursively
  2. Recursively remove any untracked files in the submodules
  3. Recursively put back any removed, tracked files in the submodules
  4. Recursively check out the correct commit in all submodules

The problem is step 2 happening before step 4. Let's say a submodule itself has a submodule, AND that in the new commit hash, that submodule has been removed. When we do step 2, no problem, we're on the earlier commit, and that sub-submodule is tracked and retained. When we do step 4, suddenly that directory that used to be sub-submodule is now some unknown directory that's left behind. If we had done the clean after the update, this would have been correctly removed.

What's the best way to test this MR?

What are the relevant issue numbers?

Edited by Cliff Woolley

Merge request reports