Git shallow clone strategy omits (remote) branches
<!--IssueSummary start-->
<details>
<summary>
Everyone can contribute. [Help move this issue forward](https://handbook.gitlab.com/handbook/marketing/developer-relations/contributor-success/community-contributors-workflows/#contributor-links) while earning points, leveling up and collecting rewards.
</summary>
- [Close this issue](https://contributors.gitlab.com/manage-issue?action=close&projectId=278964&issueIid=30067)
</details>
<!--IssueSummary end-->
## Summary
When doing shallow clones (DEPTH=1+), not all remote branches are retrieved, as a matter of fact, only 'reachable' or relevant branches are fetched. The default in gitlab is 50, but even this does not guarantee all branches are available.
## Steps to reproduce
In our project, we want to create architecture specific branches from our main release branch. What the CI will do, checkout the current branch (gitlab-runner does this of course by default), and go over a list of available existing architecture branches to update these. To do so, it uses `git branch --list --remote "origin/release/v0/*" | sed -n -r 's|.*origin/release/v0/||p'` to obtain the list of 'v0' named branches. E.g. release/v0/arm32v7, release/v0/arm64v8.
However, if git fetch is set with a low depth parameter. We can create a scenario where not all remotes have been retrieved. This is related to the fact that the branches could not be reached in the current 'depth'.
The work around for now appears to be either add a git fetch to the CI task (ugly) or (according to the documentation) set the depth to 0.
<details>
<summary> .gitlab-ci.yml </summary>
```yml
.git:
<<: *common_parameters
image: registry.hub.docker.com/gitscm/git:latest
before_script:
- git config --local user.name "${GITLAB_USER_NAME}"
- git config --local user.email "${GITLAB_USER_EMAIL}"
- git config --local credential.helper "cache --timeout=2147483647"
- printf "url=${CI_PROJECT_URL}\nusername=esbs-bot\npassword=${CI_PERSONAL_TOKEN}\n\n" | git credential approve
- git remote set-url --push origin "https://esbs-bot@${CI_REPOSITORY_URL#*@}"
after_script:
- git credential-cache exit
create_release_branches:
extends: .git
stage: release
only:
refs:
- /^v\d+\.\d+$/
except:
refs:
- branches # Workaround for gitlab-org/gitlab-ce#27818
script:
- git_tag="$(git describe --exact-match --match "v[0-9]*")"
- git_tag_msg="$(git tag --format="%(contents:subject)%0a%0a%(contents:body)%0a%0a(Auto-created release candidate)" --list "${git_tag}")"
- |
git fetch --prune # Added as workaround
for release_branch in "release" $(git branch --list --remote "origin/stable/v0/*" | sed -n -r 's|.*origin/stable/v0/||p'); do
echo "Creating release branch '${release_branch}' from '${CI_COMMIT_REF_NAME}' (${CI_COMMIT_SHORT_SHA})"
git checkout "${CI_COMMIT_SHA}" -b "stable/${CI_COMMIT_TAG:?}/${release_branch}"
if [ "${release_branch}" != "release" ]; then
sed -i 's|FROM registry.hub.docker.com/library/alpine|FROM registry.hub.docker.com/'"${release_branch}"'/alpine|g' "Dockerfile"
git add "Dockerfile"
git commit --message="${release_branch}: Auto-created release branch 'stable/${CI_COMMIT_TAG:?}/${release_branch}'." --no-edit
release_prefix="${release_branch}/"
fi
git tag --annotate --message="${git_tag_msg:-See tag '${CI_COMMIT_TAG}'}" "${release_prefix:-}${CI_COMMIT_TAG}.0-rc1"
git push --follow-tags origin "HEAD"
done
```
</details>
## Actual behavior
All remotes are available on the runner, albeit at a shallow depth of the main branch
## Expected behavior
Even shallow fetch, should always retrieve all branches, not just the current branch.
## Proposal
As stated below in https://gitlab.com/gitlab-org/gitlab/issues/30067#note_215829463 not fetching all the branches is expected since we are only going far back X commits.
However, `git clone --depth` supports an additional argument [--no-single-branch](https://www.git-scm.com/docs/git-clone)
> --depth <depth>
>
> Create a shallow clone with a history truncated to the specified number of commits. Implies
> --single-branch unless --no-single-branch is given to fetch the histories near the tips of all
> branches. If you want to clone submodules shallowly, also pass --shallow-submodules.
which should resolve this issue, it clones all branches (at depth=<depth>).
The proposal is thus by default, enable this flag, so we get expected behavior in that all expected branches are available.
If processing is to heavy or this is too big a bandwith burden, this could be added as GIT_SINGLE_BRANCH="yes|no".
issue