Skip to content

FF_GIT_URLS_WITHOUT_TOKENS: 1 not working

Issues with FF_GIT_URLS_WITHOUT_TOKENS enabled

  1. runner is not able to pull in submodules, if they need auth
  2. On windows, we see error logs when the new cred helper gets called with e.g. "erase"
    • this generally does not create a failure by itself, but results in error messages that are not relevant to the user
  3. On windows, when GCM (git-credential-manager) is at play, and caches creds from a previous run, pulling the repo fails

Findings

These all seem to be related to the changes made to remove the CI_JOB_TOKEN from any git config / git call.

  1. The new cred helper is only configured for the mother repo, but is not used in the submodule repo. Therefore, if any submodule needs auth (and there isn't any other (caching) auth helper at play), those repos can't be pulled in.
  2. Under certain circumstances, git might call the new cred helper with other arguments than "get" (e.g. "erase"). The way the cred helper on windows is currently set up would then try to call something with the name "erase"; "erase" is a (default) alias to "Remove-Item"; the whole call will fail.
  3. On windows, when GCM is at play, and state is shared across multiple builds (e.g. shell executor), GCM will cache the credential. Next time a job runs, the credential (effectively the CI_JOB_TOKEN) from the previous run will be used, however this credential is not valid anymore.

Potential fixes

  1. Ensure the cred helper will not only be used with the mother repo, but with all submodule repos too
  2. Refactor the cred helper on windows, so that
    • it supports "erase" & "store", but makes them noop
    • it allows for other arguments and makes them noop, optionally with logging
  3. Find a way to either not cache relevant creds at all, or a way to prune those creds at the beginnig of a build
    • ideally without touching the users' git config (neither --global nor --system)

Some more context

FF_GIT_URLS_WITHOUT_TOKENS was introduced with 17.5.0. It was turned on by default, because this should have helped minimizing the the possibility of token leaks for customer. After reports from users re. the above metioned issues, we switched the default of FF_GIT_URLS_WITHOUT_TOKENS to 0 and rolled that out with 17.5.1. Customers confirmed that his resolved their failures.

We should however see to set the default for FF_GIT_URLS_WITHOUT_TOKENS to 1 again.

Open questions

Re. GCM caching

Wingit brings GCM by default, and configures it globally (ie. git config --system). We can technically check on GCM and prune creds. But what about other credential helpers?

Expected outcome

FF_GIT_URLS_WITHOUT_TOKENS should default to 1

  • thus not storing the CI_JOB_TOKEN anywhere
  • submodules with auth should be able to be pulled successfully
  • other cred handlers, potentially caching ones, should not interfere with "our" cred helper

Related issues

Edited by Hannes Hörl