Allow custom token prefix for all token types

With feat: add token prefix (!20968 - merged), Siemens added a token prefix for PATs that was configurable at instance-level to make it easier to identify instance-specific leaks, for example for use with tools like gitleaks and GitHub's secret scanning programs (https://docs.github.com/en/developers/overview/secret-scanning-partner-program).

It would be great if we were able to prefix all tokens for self-managed instances to distinguish them from GitLab SaaS and other self-hosted instances.

Example:
https://gitlab.com/gitlab-org/gitlab/-/blob/7bcc0888949a15d1ecbc188cd00832b822e092b0/app/models/ci/trigger.rb#L9

Proposal

Provide a way to customize token prefixes for all tokens, so that specific instance token leaks can be identified.

For example, adding SIE- as a prefix for all kinds of tokens would distinguish the pattern from GitLab.com's pattern of gl :

Token Type	Status Instance Prefix	MR	Team	Respective EMs, PMs and AppSec SC
Personal access token			groupauthentication	Already involved in this issue
Impersonation token			groupauthentication	Already involved in this issue
Project access token			groupauthentication	Already involved in this issue
Group access token			groupauthentication	Already involved in this issue
Feed token		!179852 (merged)	groupauthentication	Already involved in this issue
OAuth Application Secret			groupauthentication	Already involved in this issue
Deploy token			groupenvironments	@nmezzopera @nagyv-gitlab @ameyadarshan Please see the discussion below for context
Runner authentication token			grouprunner	@nicolewilliams @DarrenEastman @cmaxim Please see the discussion below for context
CI/CD Job token			grouppipeline security & groupauthentication	@shampton @jocelynjane @greg Please see the discussion below for context
Trigger token			grouppipeline execution	@carolinesimpson @rutshah Please see the discussion below for context
Incoming mail token			groupauthentication	Already involved in this issue
GitLab agent for Kubernetes token			groupenvironments	@nmezzopera @nagyv-gitlab @ameyadarshan Please see the discussion below for context
GitLab session cookies			groupauthentication	Already involved in this issue
SCIM Tokens			groupauthentication	Already involved in this issue
Feature Flags Client token			groupenvironments	@nmezzopera @nagyv-gitlab @ameyadarshan Please see the discussion below for context

Intended users

• Ingrid (Infrastructure Operator)
• Sam (Security Analyst)
• probably more :)

/cc @bufferoverflow
/cc @dcouture from #371396 (closed)

This issue was automatically tagged with the label ~"group::authentication and authorization" by TanukiStan, a machine learning classification model, with a probability of 0.22.

If this label is incorrect, please tag this issue with the correct group label as well as automation:ml wrong to help TanukiStan learn from its mistakes.

If you are unsure about the correct group, please do not leave the issue without a group label. Please refer to GitLab's shared responsibility functionality guidelines for more information on how to triage this kind of issues.

Authors who do not have permission to update labels can leave the issue to be triaged by group leaders initially assigned by TanukiStan

This message was generated automatically. You're welcome to improve it.

added automation:ml groupauthentication and authorization [DEPRECATED] labels

Thanks for the issue @nejc ! I would actually advocate for the opposite and remove the configuration possibility for the Personal Access Token. Scanning tools look for glpat- and anyone who changes that is potentially more exposed.

Is it something important for Siemens to be able to use custom prefixes?

Yes, this is very important as we use a custom prefix to be able to detect leaked credentials that belong to our GitLab instance and I'm pretty sure many security aware customers have set a specific prefix to do the same.

@dcouture to give a bit of context, if we are able to determine the token is from our instance, we can more easily proactively auto-revoke it if the user does not respond to security alerts instead of attempting this on any glpat- token that may come from any instance in the wild.

We have a central scanning service that extends gitleaks config, and we provide our pattern when entering secrets scanning programs.

Currently I'm not sure if the new tokens are being picked up by security tools as they're a bit more specific, at least for gitleaks I've had to add them myself I see the point though in general and we understand that there is a trade-off between having tokens discoverable and being able to act on it.

The prefix feature for PATs also predates the default glpat- prefix added later, by around a year (13.7 -> 14.5), so IMO it would be awkward to remove it.

Oh that's unfortunate about Gitleaks. I know it was added to our secret detection tools (gitlab-org/security-products/analyzers/secrets!174 (merged)) but the process to bring it back to the upstream gitleaks project should be improved. Thanks for the context!

@dcouture we've discussed this a bit in our team and thought of another approach that might work better for both SaaS and self-hosted, i.e. discoverable and easy to act on for everyone.

How about instead of configurable prefixes for every type of token, have a single instance-level token prefix prepended. I know, a prefix for a prefix . This would avoid having a bunch of configuration options, and security tools would detect them all, but additionally tokens from specific instances could still be identified.

This would then be applied to all tokens that can be prefixed.

For example, adding SIE- as an instance prefix, you'd get this for all kinds of tokens (this is just a proposal, some of them don't exist atm) -> See table in description #388379

..and so on. This would add more padding to token length, but I see this already happened with the pipeline trigger token and seems like it's not an issue. WDYT?

It would still cause false negatives in tools that do automatic revocation / validation of tokens but I think it's better yes! And if it's better for your internal tooling I'm not opposed to it from the security point of view. I'll let the product folks chime in with their opinion!

@dcouture The idea would be that GitLab documents that anything ending in the expected format decided by GitLab e.g. glpat-XXXX would be detected as a GitLab token. A simple regex could ignore the prefix (which would have a certain max-size and allowed chars), does that cover your concern?

@dlouzan It would be detected, but some tools attempt to automatically validate the token by doing an API request (TruffleHog does that and I wonder if our automatic revocation feature supports prefixes yet ) and they'd disregard those tokens if they don't take the unexpected prefix.

@dcouture The new feature is focusing on GitLab.com SaaS only (Automatically revoke GitLab.com PATs discovered... (#371658 - closed)) with glpat- pattern.

Anyway, large GitLab instances would like to join Automatic response to leaked secrets: Partner I... (&4944) similar to https://docs.github.com/en/developers/overview/secret-scanning-partner-program and they will have a custom prefix for PATs.

@nejc Great Idea, that also reduces the config options! Could you maybe extend the table above with what exists and what not? I was also thinking about keeping the PAT as because this is out already in the wild since a while, e.g., instead of SIE-glpat- just SIE-.

I agree in this case it seems like the instance prefix might even be preferable for non-SaaS, (at least for auto revocation), so it doesn't attempt to auto-revoke all kinds of glpats, but only those on the current instance (since it presumably will be too hard to also detect a base URL for the self-revoke API from just the token leak itself).

@bufferoverflow I've updated the description above with a table, #388379, I'll keep it updated based on tokens done in Prefix all authentication tokens for easier det... (&8923).

@dcouture the question is if we agree on this approach with the GitLab product staff, whether we should add the token-type prefixes first for all tokens, or go ahead with the instance prefix already regardless of the existing prefixes?

I'll loop in @hsutor who's the product manager for ~"group::authentication and authorization"

Thanks!

@hsutor from our perspective (self-hosted) we would first focus on contributing an instance prefix for all tokens (regardless of whether they have a specific hardcoded token type prefix already), and then later add hardcoded prefixes for specific types still missing (Prefix all authentication tokens for easier det... (&8923)). WDYT?

fyi @connorgilbert @amarpatel while I don't think the proposal negatively affects the auto revocation on SaaS, it's worth noting the prefix change if we offer the service on self-managed in future.

@nejc Just to clarify, is the proposal to add a prefix for all token types in the table above, or just the first 3 that have the ?

@hsutor the idea is to add this to all token types. So the relevant column is the 3rd one, i.e. "Self-hosted prefix for all tokens".

I realize this is in addition to adding hardcoded prefixes for each token type which is a separate effort still in progress (hence the proposed table of how this would look with both prefixes if an instance prefix is added). I've edited out some of the old issue description to avoid confusion hopefully.

@adil.farrukh yes this is actually also part of our motivation, to make self-managed auto-revocation easier (whether this is native in GitLab or as an integration with another service via the API such as GitLab & GitHub secrets scanning partnerships when tokens leak elsewhere).

Ok @nejc, thanks for the clarification and I am on board with this feature.

I want to tag the owners of the other token types so they are aware:

Token	Owner
pipeline trigger token	@jheimbuck_gl
runner registration token	@DarrenEastman
runner authentication token	@DarrenEastman
CI job token	@jheimbuck_gl
deploy token	@cbalane
runner registration token	@DarrenEastman

thanks @hsutor. not sure if it will matter to Secrets Detection but tagging in @connorgilbert for awareness as well.

@hsutor CI_JOB_Tokens are under grouppipeline execution our group is responsible for the CI_JOB_JWT token

On the Secret Detection impact—today we don't consult the instance-specific PAT prefix, but would like to: Support secret detection injection of gitlab in... (#345606).

This would mean we'd inject the instance-specific token prefixes into the detection rules instead of hardcoding glpat-, etc.

@hsutor @adil.farrukh I wanted to let you know that I've started working on this feature. That MR starts with support for feed tokens, as they are easy to test and to show the concept. I'll add support for other token types in follow-up MRs, once this approach has been approved.

As discussed, this MR proposes to add an instance wide prefix, that is set to gl by default. The new prefix format is: #{instance_prefix}#{token_type_prefix}. By default, instance_prefix is the first part of the current prefix (gl). We can now customize the instance prefix to create a new prefix, while keeping the rest of the prefix unchanged. For a feed token with the instance_prefix of my-company-name, we'd get: my-company-name-ft-.

@nwittstruck Thank you and sounds good! I went back looking for our discussion on persisting history around the prefixes for revocation, and based on Admin Token API does not work with historic cus... (#504989) seems like we'd hold off on that until greater demand. So we are good to go ahead with this work, and get the team's reviews.

@adil.farrukh Could you please assign this issue to me? I'd like to update the table to keep track of the merge requests

I've added (hopefully) everyone responsible for the respective tokens types to the implementation status, as I've started working on this and the last ping was 2 years ago.

I think the table should be mostly accurate. If I've pinged someone by mistake, please let me know, I'll fix the table.

There is no action necessary from the people I've tagged right now, this is just for awareness.

You can find more about the current state of the implementation in this thread or in the documentation.

@nwittstruck FWIW, grouppipeline security only owns CI job token through the upcoming allowlist deprecation in 18.0. Anything forward looking beyond the deprecation (i.e. future of CI job token) will be owned by groupauthentication.

Setting label(s) devopsmanage sectiondev based on ~"group::authentication and authorization".

added devopsmanage sectiondev labels

Hey @nejc, thank you for creating this issue!

To get the right eyes on your issue more quickly, we encourage you to follow the community issue workflow.

PLEASE NOTE: You will need to use the @gitlab-bot label command to apply labels to this issue.

To set expectations, GitLab product managers or team members can't make any promise if they will proceed with this. However, we believe everyone can contribute, and welcome you to work on this proposed change, feature or bug fix. There is a bias for action, so you don't need to wait. Try and spin up that merge request yourself.

If you need help doing so, we're always open to mentor you to drive this change. This message was generated automatically. You're welcome to improve it.

added automation:self-triage-encouraged label

mentioned in issue gitlab-org/quality/triage-reports#10837 (closed)

mentioned in issue gitlab-org/quality/triage-reports#10915 (closed)

added featureenhancement label

added typefeature label

mentioned in merge request siemens/gitlab!65 (closed)

changed title from Make hardcoded token prefixes configurable to Add single instance token prefix (was: Make hardcoded token prefixes configurable)

changed the description

added WGTokenMgmt label

changed the description

changed title from Add single instance token prefix (was: Make hardcoded token prefixes configurable) to Add single instance token prefix

changed the description

changed milestone to %Backlog

added to epic &5793

changed the description

marked this issue as related to #345606

@bufferoverflow I wanted to bring out one of your comments from above:

The new feature is focusing on GitLab.com SaaS only (Automatically revoke GitLab.com PATs discovered... (#371658 - closed)) with glpat- pattern.

I wanted to clarify that automatic revocation of leaked PATs works in both GitLab.com and self-managed, but today requires the glpat- prefix. The automatic revocation of PATs works differently from external secrets liked AWS; for PATs it's handled directly within the GitLab application, so it's not limited to GitLab.com.

We're interested in injecting the instance-specific prefixes (see comment above, #388379 (comment 1322979107), referring to #345606) but don't currently have an ETA for that based on our other work in flight.

Thanks for the update @connorgilbert !

mentioned in issue #398219

marked this issue as related to #398219

added devopsgovern sectionsec labels and removed devopsmanage sectiondev labels

mentioned in issue #426137 (closed)

added groupauthentication label and removed groupauthentication and authorization [DEPRECATED] label

It would be a great addition to these efforts being able to differentiate in between Personal Access Token, Group Access Token and Project Access Token.

My observation is that junior CI developers are integrating their own users personal access token into CI variables. Thus it would be helpful having a different pattern, that could be used to check for personal access tokens in CI variables. E.g. glgat- and glpat-.

added 1 deleted label

changed title from Add single instance token prefix to Allow custom token prefix for all token types

changed the description

FYI: I've updated the title and description to propose having a prefix available for all tokens that is entirely distinct from the default prefix (glpat- etc).

This is based on our finding that simply adding another prefix instead of changing it completely causes too many duplicates and false positives in security scanners like gitleaks and GitLab secret detection etc. (e.g. SIE-glpat-123 and glpat-123 both match, which wouldn't happen with something more distinct like SIE-pat-123).

mentioned in issue #479133

mentioned in commit gitlab-community/gitlab@1257c8e7

mentioned in merge request !179852 (merged)

mentioned in commit gitlab-community/gitlab@17dc7809

mentioned in commit gitlab-community/gitlab@a1c6eef5

mentioned in commit gitlab-community/gitlab@e2c6a1dc

mentioned in commit gitlab-community/gitlab@c401db7c

changed milestone to %17.9

added Category:System Access label

added Community contribution label

added workflowin dev label

mentioned in issue gitlab-org/developer-relations/contributor-success/team-task#490

mentioned in commit gitlab-community/gitlab@2ea5ea2e