Gitlab CI with git submodule update --remote gives error

Overview

I can not run git submodule update --remote in Gitlab CI by following this documentation https://docs.gitlab.com/ee/ci/git_submodules.html.

I need the --remote flag so it can get the latest changes in the branch that is specified in the .gitmodules otherwise I need to update each project that is using the submodules to use the latest submodule commit. I want all projects to get the latest changes automatically without manual operations, --remote help a lot in this case, however, I get an error even if I am a maintainer in both projects.

I have this gitlab-ci.yml:

deploy_dev:
  stage: deploy_dev
  image: bitnami/git:2.36.0-debian-10-r0
  environment: dev
  variables: 
    GIT_STRATEGY: clone
    GIT_SUBMODULE_STRATEGY: recursive
    GIT_SUBMODULE_UPDATE_FLAGS: --init --remote --merge --recursive
  ...

I get this error when in the submodule there is a new commit but the project that has the submodule is pointing to an old commit:
fatal: Needed a single revision. Unable to find current origin/<branch> revision in submodule path <submodule>

I tried this workaround but gives auth issue: https://stackoverflow.com/a/57770651

Gitlab version Self Managed: 14.9

Additional symptoms

If a branch is specified for a submodule together with GIT_SUBMODULE_UPDATE_FLAGS: --remote then the repository fails to check out.

Proposal for resolution

• {placeholder}