Centralize token patterns for various features that depend on them

We have token patterns in many places:

  1. ✅ Secret scanning: https://gitlab.com/gitlab-org/security-products/analyzers/secrets/-/blob/1b2e92916178a0717908894232beab230241648f/gitleaks.toml
  2. Default masking in job traces (gitlab-runner!3639 (merged)): https://gitlab.com/gitlab-org/gitlab-runner/-/blob/af1064e547c0a5d40796c3777a73d22b6d53e468/helpers/trace/internal/tokensanitizer/token_masker.go#L68
  3. ✅ Secret push protection (Ruby gem) (https://gitlab.com/gitlab-org/gitlab/-/blob/master/gems/gitlab-secret_detection/lib/gitleaks.toml)
  4. ✅ Secret detection service (Ruby service) Automate ruleset (gitlab-org/security-products/secret-detection/secret-detection-service!36 - merged) • rossfuhrman • 17.6
  5. User warnings in text fields (#368434 (closed)): https://gitlab.com/gitlab-org/gitlab/-/blob/eff00776ef24dc0754573050dd2e63bc48fe252e/app/assets/javascripts/lib/utils/secret_detection.js#L17
  6. Masking in Language Server Protocol server implementation: https://gitlab.com/gitlab-org/editor-extensions/gitlab-lsp/-/blob/d91362c69053d685591b611985534af70cc6988c/src/common/secret_redaction/gitleaks_rules.ts
  7. DAST rules for detecting secrets leaked in web pages (related issue: https://gitlab.com/gitlab-org/gitlab/-/issues/488395+s)

The latter two would benefit from a more comprehensive list that can be found in the secret scanning feature and would help prevent token leaks for us and our users. Different parts of the product running in completely different infrastructures are involved here but there would be a great benefit in unifying our pattern source so it's worth thinking about. A runtime solution to this might not be feasible, but perhaps something at build time that fetches the patterns from a predefined GitLab-maintained source could be a solution.

This issue focuses on places in the product where those patterns appear, but we also have custom secret detection tooling which replicates those patterns again and would also benefit from a central pattern definition.

Edited by rossfuhrman