Skip to content
GitLab
Next
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • gitlab-runner gitlab-runner
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 2,506
    • Issues 2,506
    • List
    • Boards
    • Service Desk
    • Milestones
    • Iterations
    • Requirements
  • Merge requests 122
    • Merge requests 122
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
    • Test Cases
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Container Registry
  • Monitor
    • Monitor
    • Metrics
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Code review
    • Insights
    • Issue
    • Repository
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • GitLab.orgGitLab.org
  • gitlab-runnergitlab-runner
  • Merge requests
  • !3192

Add support for extra submodule update flags

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Nejc Habjan requested to merge siemens/gitlab-runner:feat/submodule-update-extra-flags into main Nov 06, 2021
  • Overview 21
  • Commits 1
  • Pipelines 11
  • Changes 4

What does this MR do?

Adds a GIT_SUBMODULE_UPDATE_FLAGS for additional flags that can be appended to the git submodule update --init [--recursive] invocation when submodules are fetched, much like the existing GIT_FETCH_EXTRA_FLAGS.

Why was this MR needed?

Some people want to use the --remote flag (see #3011 (closed), although most of the discussion there misses how submodules really work). This goes a step further and enables adding any flag allowed with the update subcommand:
https://git-scm.com/docs/git-submodule#Documentation/git-submodule.txt-update--init--remote-N--no-fetch--no-recommend-shallow-f--force--checkout--rebase--merge--referenceltrepositorygt--depthltdepthgt--recursive--jobsltngt--no-single-branch--ltpathgt82308203

For example --jobs for parallel fetches, --[no-]single-branch etc.

One thing I'd like to point out is that this would slightly overlap with the existing GIT_SUBMODULE_STRATEGY (for --recursive), GIT_DEPTH (when applied to submodules), and GIT_SUBMODULE_PATHS, because all of these could potentially be manually supplied to GIT_SUBMODULE_UPDATE_EXTRA_FLAGS as well. Because of this, I initially considered just having a GIT_SUBMODULE_UPDATE_STRATEGY, but I like the flexibility of this more, and it avoids future feature requests ("can I increase parallel jobs for submodules .. etc).

Are you okay with this overlap? I guess in some ways this could even replace some of the custom code for building the submodule update command, although this would be a breaking change. If needed I can try to provide some examples of how this might overlap.


If you decide this is the right approach I'll add a companion docs MR in the main repo (see https://gitlab.com/search?search=GIT_SUBMODULE_STRATEGY&group_id=9970&project_id=278964&scope=&search_code=true&snippets=false&repository_ref=master)

What's the best way to test this MR?

Add a runner built from this branch, register it and add the variable:

variables:
  GIT_SUBMODULE_STRATEGY: normal
  GIT_SUBMODULE_UPDATE_FLAGS: --remote --force --jobs 4

Enable CI_DEBUG_TRACE=true in CI variables and observe that these are passed to the update command.
I suppose the point here is not to also test git behavior but of ypu want to be sure, also enable GIT_TRACE=1 and observe that git runs 4 jobs (run-command.c:1628 run_processes_parallel: preparing to run up to 4 tasks).

Example: https://gitlab.com/nejch1/submodule-update-flags/-/jobs/1755771962#L606

++ echo 'Updating/initializing submodules with git depth set to 50...'
Updating/initializing submodules with git depth set to 50...
++ git submodule sync
++ git submodule foreach 'git clean -ffxd'
++ git submodule foreach 'git reset --hard'
++ git submodule update --init --depth 50 --remote --force --jobs 4
++ git submodule foreach 'git clean -ffxd'

What are the relevant issue numbers?

Closes #3011 (closed).
Closes #27277 (closed) (if people are happy supplying it as a flag - @steveazz would this be enough as you opened that issue?)

🛠 with ❤ at Siemens

Edited Jan 19, 2022 by Nejc Habjan
Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: feat/submodule-update-extra-flags