Prevent credential leakage by reducing masked variables restrictions
Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.
Problem to solve
As mentioned in #213159 (closed), protecting job logs only when CI_DEBUG_TRACE is set to true will most likely be insufficient.
Although the CI_DEBUG_TRACE flag has proven to be problematic if not used with caution, it's not the only way to leak secrets. The more abstract problem seems to be that we let Maintainers make mistakes.
For example, handling the CI_DEBUG_TRACE as described in #213159 (closed) case would not help in cases where credential leakage occurs by other means, for example by using echo or env to display environment variables via a pre-checkout hook.
Allowing Maintainers to modify artifacts whose output ultimately lands in a job's log, which is a normal thing for Maintainers to do, forces us to deal with all infinite ways in which sensitive variables could possibly be leaked: CI_DEBUG_TRACE, echo, env and more. This is, I argue, impractical.
Intended users
User experience goal
Help prevent unintended credential leakage.
Proposal
Instead of approaching this problem in a case-by-case basis we should improve GitLab's CI/CD variable masking capabilities, I argue this is actually fixing it, so that there are no restrictions w/r/t format, length or content. How to best do this is to be defined, but it is clear, that there are plenty scenarios where CI/CD variables should without doubt be masked even if they are too complex in format (service accounts, usually in JSON format).
Regardless how we choose to improve variable masking, current restrictions on variable masking seem unreasonable, as many masking-worthy variables fall precisely within the formats that cannot be masked according to our documentation:
https://docs.gitlab.com/ee/ci/variables/#masked-variable-requirements
Alternatively, we could implement access controls when accessing job logs, as described in https://gitlab.com/gitlab-org/gitlab/-/issues/250745, so that only maintainers and above can access them by default.
This would help Maintainers avoid making mistakes that could lead to credential leakage.
If we sufficiently improve variable masking, no amount of echo, env, etc will circumvent variable masking, as even complex variables that are unprotected due to their complexity and length today will be masked.
Additional comments
we need to ensure that this operation is memory and time safe on the Runner side, and that it does not obfuscate a build log if masking parameters are not defined in a sane way. Multiple strategies can be taken here, but the most simple one is presumably relaxing a limit without removing it completely.