Skip to content

Comprehensive Plan for Token Consolidation

Background

This issue contains a tentative plan for token consolidation in GitLab. This plan will be further refined and broken down when we create a technical plan and blueprint in %17.9.

Why consolidate?

We propose to consolidate the existing implementation of Tokens in GitLab for the following reasons:

Security

Today, tokens, depending on their type, contain a lot of privilege - often the same as that of the user that created them, or overly permissive with the limited, broad scopes GitLab has available to choose from. Examples are included in this Epic and this CI_Job_Token specific Epic.

In addition to their broad, non-granular scopes, they can also be long-lived. We enforced a token lifetime limit of 1 year as a breaking change , but many customers weren't aware of this and were caught off guard. The change was made optional for self-managed, but still exists on GitLab.com today. Token management tooling and notifications have been improved as part of the learnings. Even a token that can live for 1 year is too long, as a leaked token can provide extended access to sensitive data for a long period of time if not detected/revoked.

Many risks with tokens were identified as outcomes of the Token Working Group.

Maintenance

GitLab has a large number of token types. See also the token overview. The implementation of each is different (or at least, not re-used consistently), meaning every token must be maintained individually, and when behavior is changed in one token type, it is not automatically added to other token types, creating disprate experiences.

Usability

It is not clear to customers which tokens to use, when, what permissions to give them, and how to manage the tokens, especially at scale. With fewer token types, we hope to solve some of these issues.

Scalability

Scope

The scope of token consolidation is any token type listed in the Token Overview. We will evaluate which token types are to be consolidated based on the following criteria:

  1. Is the token long-lived? If yes, it should be consolidated into a token type that has a lifetime limit (can be optional)

  2. Does the token have, or have the potential to have fine grained scopes? If no, it is a candidate for consolidation. We need to consider nuance here, some tokens only have very limited scopes to begin with and may not need to have more fine-grained scopes.

  3. Is the management experience inconsistent with other token types? If yes, we may want to consolidate with a token type that has a consistent management interface and CRUD support.

  4. Is the token widely used? The less widely used a token is, and the more aspects of points 1-3 it has, the more likely it is to be consolidated.

Proposed Plan

1. Scope Definition

1a. Define criteria for consolidation candidates. An attempt at this has been made in this section, but this should be formalized and documented.

1b. Define token types to be included for consolidation effort. This list will be candidates to be evaluated, it does not mean that every token type in scope will be consolidated.

2. Existing Token Inventory

2a. Inventory existing token types (e.g., Personal Access Tokens, Project Access Tokens, Group Access Tokens, CI/CD tokens, etc.), ownership, and intended capabilities. This has been started here.

Document the tokens capabilities as evaluated against the scope criteria (eg, what are the scopes like? how long can they live? how widely adopted are they?)

Collaborate with each owning team to both raise awareness of the effort and to understand the current usage and implementation.

3. Create design blueprint

3a. Create design blueprint and peer review with Grzegorz/Kamil (tech leadership). Token consolidation blueprint: Review token con... (#504771 - closed)

3b. Identify any POCs, blockers or parallel tasks (e.g tiering status) for consolidation

3c. Peer review of breakdown plan

4. Instrument adoption and usage

4a. Instrument adoption and usage for all token types being considered for consolidation. We will have a better idea of what the candidates are after the initial 3 steps in the plan are complete.

5. Migration path for consolidated token types

5a. Investigate and document migration path for consolidated token types. Determine 1 token type to begin with as a PoC for migration

6. Build out foundational capabilities for consolidated token experience

6a. Before we can expect people to use a centralized token experience and fewer token types, we need to make sure we have the correct foundational pieces in place. This step is to define + build those pieces.

7. Documentation, communication and outreach

7a. Communicate through deprecations and removals process, if necessary. If we run these as migrations, no breaking changes should be expected, but there will be edge cases of token usage that we don't know about - and we should be prepared to mitigate the disruption of these.

7b. Encourage use of new tokens, disincentivize use of tokens that will be consolidated (may be able to disable legacy tokens for net new customers)

8. Migration

8a. Migrate users over from old token types in a phased manner e.g Deploy tokens to Service accounts with PATs

8b. Document the long tail work around maintaining the legacy tokens until all users have moved off or have a transparent mechanism to adopt new foundation provided.

9. Monitoring and Feedback

9a. Set up monitoring systems to track the adoption of the new token system, eventually fully remove support for legacy token types.

Edited by Hannah Sutor