Mask values in logs that match secrets detection

Release notes

Problem to solve

We have a scenario where the sensitive values are generated in the pipeline so they are not known prior to the pipeline run.

The value is consumed by a 3rd party tool - terraform. The value can be visible as part of a terraform plan. Terraform has support to flag and suppress sensitive values however if we do that then we can’t see other stuff in the plan that we want to see. We want to tell terraform NOT to treat the value as sensitive but then also ensure it’s not visible in the Gitlab logs.

Additionally, it’s made us think that we could have other places where sensitive values are being exposed in gitlab logs so it’d actually be a really useful and really cool feature if there was a way to set some sensitive value regexes that can be applied at project level, group level or globally. Then we could do things like set a regex to identify AWS credentials and have confidence that they won’t get exposed anywhere in Gitlab logs.

Proposal

Some iterations:

  • Use Secret detection to find/present default secrets matched in job logs through the .json report - GitLab Free feature.
  • Use Secret detection to find/mask default secrets matched in job logs in the job log console and raw output - GitLab Premium maybe?
  • Use Secret detection to find/present default secrets matched in job logs through Ultimate features - GitLab Ultimate feature.
  • Use the custom ruleset from Secret Detection to alert into the existing secrets detection dashboard/reports/etc. - GitLab Ultimate feature
  • Use the custom ruleset to find/mask data in logs as they stream to the job log console and raw output - GitLab Ultimate feature
  • Log detection/masking as an audit event - GitLab Premium

Intended users

Metrics

This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.

Edited by James Heimbuck