Gitlab Runner v16.9.0: submodule update fails when using GIT_STRATEGY: fetch if submodule moved between commits

Summary

When a submodule moves, there are cases where the "fetch" GIT_STRATEGY will fail to update submodules.

Steps to reproduce

  1. Set GIT_STRATEGY: fetch in the pipeline definition
  2. Add a submodule to a certain path in the git project
  3. Push those changes
  4. Add a commit in which the submodule is moved in a folder inside the current location of the submodule, so that the new submodule's location is "inside" the current submodule.
  5. Push that commit and see the pipeline failing because of a failed git fetch.

Actual behavior

First of all, the pipeline's GIT_STRATEGY: fetch is a requirement since the "clone" strategy is too costly. However, it doesn't seem to work when sequentially triggering pipelines on commits which have different git project structure. If a submodule moved, the runner simply won't be able to correctly update the submodules, probably because it is using git in a way that wasn't anticipating this scenario. Here's the case:

For clarity,

  • the older path for the submodule will be refered as path/to
  • the new path for the submodule will be refered as path/to/submodule

On the older commit

If the pipeline is run on the older commit (where the submodule was not nested) after the newer one, you get:

Updating/initializing submodules recursively with git depth set to 20...
Synchronizing submodule url for 'path/to'
fatal: not a git repository: path/to/../../.git/modules/path/to

I am guessing that the git project currently has an existing folder at .git/modules/path/submodule since the more recent version's submodule is located there. Hence, on the older version's pipeline, it tries to reuse that .git folder but fails because the path/to folder already exists and is not a git repositry (the expected repository is in fact in path/to/submodule!).

On the newer commit

If the pipeline is run on the newer commit (where the submodule was moved 1 folder deeper) after the old one, you get:

error: submodule git dir '/builds/my_repo/.git/modules/path/to/submodule' is inside git dir '/builds/my_repo/.git/modules/path/to'
fatal: refusing to create/use '/builds/my_repo/.git/modules/path/to/submodule' in another submodule's git dir
Failed to clone 'path/to/submodule'. Retry scheduled

This is a different effect to essentially the same issue. The runner tries to update the submodule in the path/to/submodule folder, but path/to already exists and is a valid repository, so git won't clone the submodule at that place.

Expected behavior

It should fetch/clone the submodule correctly.

Relevant logs and/or screenshots

job log 
See "Actual behavior"

Environment description

Docker executor, self-hosted Gitlab instance in our IT infrastructure.

config.toml contents 
I don't have access

Used GitLab Runner version

Running with gitlab-runner 16.9.0

Using our custom image.

Possible fixes

None that I know of (except using "clone" strategy.

Edited by dev123