fix(cli): Redact masked variable values from include-resolution output

Paweł Farysrequested to merge
gitlab-community/gitlab-org/ci-cd/runner-tools/glci:glci-138 into main

What this MR does and why?

GitLab expands $VAR / ${VAR} in include: entries before resolving them, and glci does the same. Every warning and error then quoted the expanded entry, so a variable holding a secret reached stderr in clear text — across lint, doctor, show, jobs, variables, merged and run, plus the daemon log and the traces of child and cross-project trigger jobs.

Redaction happens at the output boundary rather than at each call site, because it has to. httpGet wraps the error from client.Do, and Go's *url.Error carries the full URL — net/http strips a userinfo password but never a query parameter. A remote include printed the secret twice, once from glci's own fetching %s wrapper and once from inside the wrapped error, so no change to glci's format strings could have prevented the second.

What gets masked

A value is redacted when the variable is marked, and never because its value looks secret. This mirrors Gitlab::Ci::Config::External::Context#mask_variables_from, which masks a variable if and only if variable[:masked] is set.

Variables fetched from the API already carry GitLab's masked flag and are honoured unconditionally. The local layers (--env, --env-file, preset env:, .glci.env) have no such flag — that is the gap #138 (closed) identifies — so this adds one:

glci merged --mask PRIVATE_TOKEN
[pipelines.deploy]
env = { PRIVATE_TOKEN = "glpat-xxxxxxxxxxxx" }
masked = ["PRIVATE_TOKEN"]

--mask names a variable rather than a value, so it applies whichever layer supplied it, and it is the local equivalent of ticking Mask variable in the UI. A project .glciconfig.toml may only add to a global preset's masked list, never remove from it, so a cloned repository cannot unhide a variable the user marked — the same restriction [token] already carries.

Why nothing is masked automatically

An earlier revision redacted any unmarked value that matched Ci::Maskable::REGEX. That is dropped, because being more protective than GitLab is its own failure: a value hidden locally and then printed by a real pipeline teaches the user their secret was protected when it never was. Local output now agrees with what the server does.

For the same reason glci refuses to mask a value GitLab would reject, and says so instead of hiding it:

glci: warning: DEPLOY_PHRASE left visible: GitLab would not accept this value for masking, so hiding it
locally would misreport what a real pipeline does. It must be a single line with no spaces, at least 8
characters, and use only letters, digits and _ : @ - + . ~ = /

Both of GitLab's rules are applied: REGEX for an expanded variable, and MASK_AND_RAW_REGEX for one marked raw:.

Coverage

Each masked value is matched in every form it can take on the way to a message — raw, percent-encoded, and %q-quoted — because an include path is escaped before it reaches the API URL and several fetchers quote it. --unmask turns redaction off, and when anything is redacted glci says so once on stderr and names the flag, so a masked path is never mistaken for a failed one.

The value redactor moves from pkg/daemon to pkg/variables so the CLI, the daemon log and a trigger job's trace share one implementation and one redact-before-sanitize ordering.

This retracts the caveat documented in !154 (merged).

Steps to reproduce

# .glci.env
PRIVATE_TOKEN=glpat-FAKEFAKEFAKE1234
# .gitlab-ci.yml
include:
  - remote: https://gitlab.example.invalid/ci.yml?private_token=$PRIVATE_TOKEN
job:
  script: [true]

Before, on main, the token is printed twice — once by glci and once by the wrapped *url.Error:

$ glci merged 2>&1 >/dev/null
parsing CI config: resolving includes: fetching https://gitlab.example.invalid/ci.yml?private_token=glpat-FAKEFAKEFAKE1234: Get "https://gitlab.example.invalid/ci.yml?private_token=glpat-FAKEFAKEFAKE1234": dial tcp: ...

After, marking the variable hides both:

$ glci merged --mask PRIVATE_TOKEN 2>&1 >/dev/null
glci: values in include: paths were redacted; re-run with --unmask to see them
parsing CI config: resolving includes: fetching https://gitlab.example.invalid/ci.yml?private_token=[MASKED]: Get "https://gitlab.example.invalid/ci.yml?private_token=[MASKED]": dial tcp: ...

glci merged --unmask restores the original output. Without --mask the value is still printed, which is what GitLab does with an unmasked variable.

Two more shapes worth checking: swapping the include for project: "$GROUP/ci" covers the skipping project include warning and the percent-encoded API URL, and glci run covers the daemon path, where the same error previously reached both the terminal and ~/.glci/daemon.log.

Relevant issues and other links

Merge request reports