Generate JWT for authentication and provide it to CI jobs
What does this MR do?
Generate JWT that can be used for authentication with 3rd parties that support JWT authentication (e.g. Vault - https://www.vaultproject.io/docs/auth/jwt/#jwt-authentication) and provide it to jobs as CI_JOB_JWT
predefined CI variable.
See #207125 (closed).
Screenshots
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry -
Documentation (if required) -
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers -
Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team
Merge request reports
Activity
changed milestone to %12.10
added workflowin dev label and removed workflowready for development label
mentioned in issue #207125 (closed)
1 Warning This merge request is quite big (more than 514 lines changed), please consider splitting it into multiple merge requests. Reviewer roulette
Changes that require review have been detected! A merge request is normally reviewed by both a reviewer and a maintainer in its primary category (e.g. frontend or backend), and by a maintainer in all other categories.
To spread load more evenly across eligible reviewers, Danger has randomly picked a candidate for each review slot. Feel free to override this selection if you think someone else would be better-suited, or the chosen person is unavailable.
To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines.
Once you've decided who will review this merge request, mention them as you normally would! Danger does not (yet?) automatically notify them for you.
Category Reviewer Maintainer backend Matthias Käppler ( @mkaeppler
)Mayra Cabrera ( @mayra-cabrera
)Generated by
DangerEdited by 🤖 GitLab Bot 🤖added 26 commits
-
3e598e86...f56ec37a - 24 commits from branch
master
- 76a4e0e5 - Implement generating signed JWT to be used in CI
- 501088eb - Add CI_JWT_AUTH as predefined CI variable
-
3e598e86...f56ec37a - 24 commits from branch
@tmaczukin This is the approach to authenticate to Vault from CI jobs that we were discussing recently, would you like to do the backend review? I'll move to writing some documentation on how to use it meanwhile.
assigned to @tmaczukin
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
@krasio I've left few first comments. However please note that I'm not a reviewer/maintainer of GitLab project, so I might be not up-to-date with our current development rules for Rails. It would be good to ask also someone else for the review
unassigned @tmaczukin
- Resolved by Tomasz Maczukin
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Shinya Maeda
- Resolved by Shinya Maeda
I'm excited for this merge request, thanks for your work, @krasio!
Would it be possible to add the user login and the user email to the token as well? We'd like to use this information to attribute artifacts created by CI jobs to users.
Edited by Markus Kaiser
added 552 commits
-
501088eb...c152e2e4 - 550 commits from branch
master
- fd272591 - Implement generating signed JWT to be used in CI
- 11ddca39 - Add CI_JWT_AUTH as predefined CI variable
-
501088eb...c152e2e4 - 550 commits from branch
added 1 commit
- 161b8f12 - Implement generating signed JWT to be used in CI
added 437 commits
-
161b8f12...1e49cb90 - 435 commits from branch
master
- 8616bcde - Implement generating signed JWT to be used in CI
- e819a46b - Document usage of CI_JOB_JWT
-
161b8f12...1e49cb90 - 435 commits from branch
added 2 commits
added 2 commits
added 2 commits
- Resolved by Krasimir Angelov
@sselhorn Can you please take a look at my initial attempt to document this:
doc/ci/variables/predefined_variables.md
doc/ci/examples/authenticating-with-hashicorp-vault/index.md
assigned to @sselhorn
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
unassigned @sselhorn
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
- Resolved by Krasimir Angelov
mentioned in issue sselhorn/test-project#1 (closed)
mentioned in issue #212252
mentioned in merge request gitlab-com/www-gitlab-com!44313 (merged)
added 875 commits
-
76ea4a5a...3915e656 - 873 commits from branch
master
- 581fc573 - Implement generating signed JWT to be used in CI
- 70db56e2 - Document usage of CI_JOB_JWT
-
76ea4a5a...3915e656 - 873 commits from branch
assigned to @sselhorn
- Resolved by Shinya Maeda
@gitlab-com/gl-security/appsec Can you please have a look?
This is not about authenticating with GitLab but to allow CI jobs to authenticate with other systems (like Vault) so there is no security risk for us but I would love some feedback from security perspective.
One standing issue we have is about not being able to mask the JWT and its expiry time - !28063 (comment 313058861).
- Resolved by Shinya Maeda
- Resolved by Krasimir Angelov
- Resolved by Shinya Maeda
- Resolved by Shinya Maeda
- Resolved by Shinya Maeda
@krasio Thanks. I left a couple of suggestions, but it looks already nice
Let's see if it looks good to app-sec folks as well.
added Technical Writing documentation + 1 deleted label
- Resolved by Krasimir Angelov
unassigned @sselhorn
added 172 commits
-
70db56e2...c778a90a - 170 commits from branch
master
- a3c88120 - Implement generating signed JWT to be used in CI
- 3228247e - Document usage of CI_JOB_JWT
-
70db56e2...c778a90a - 170 commits from branch
marked the checklist item Changelog entry as completed
marked the checklist item Documentation (if required) as completed
marked the checklist item Label as security and @ mention
@gitlab-com/gl-security/appsec
as completed@ayufan I am still waiting on @dosuken123 to approve but want to check will you be able to do maintainer review for this MR? Code-wise it's pretty small change and I think you already have most of the context around it. Let me know if you are not able to make it and I'll find another maintainer. We really hope to get this into %12.10.
assigned to @ayufan
mentioned in issue #213972
- Resolved by Krasimir Angelov
- Resolved by Shinya Maeda
mentioned in issue #213974 (closed)
- Resolved by Shinya Maeda
@krasio Thanks. I'm really impressed by the idea of JWT auth flow and really looking forward to seeing this happens in the near future. I left a bunch of comments for improving the code quality but feel free to follow-up in a separate MR if you see this MR is urgent on our mission.
I've approved this MR
- Resolved by Shinya Maeda
- Resolved by Krasimir Angelov
- Resolved by Kamil Trzciński
- Resolved by Kamil Trzciński
- Resolved by Shinya Maeda
- Resolved by Shinya Maeda
- Resolved by Kamil Trzciński
- Resolved by Shinya Maeda
- Resolved by Krasimir Angelov
- Resolved by Shinya Maeda
- Resolved by Shinya Maeda
assigned to @dosuken123 and unassigned @ayufan
added 270 commits
-
3228247e...b93dbdf7 - 268 commits from branch
master
- dcc46a65 - Implement generating signed JWT to be used in CI
- 02a3a08a - Document usage of CI_JOB_JWT
-
3228247e...b93dbdf7 - 268 commits from branch
assigned to @ayufan
- Resolved by Shinya Maeda
- Resolved by Shinya Maeda
- Resolved by Shinya Maeda
I would really like you to keep ref_protected_by as this holds the original glob used for defining the protection. Otherwise a token consuming resource provider would need to implement the same globing algorithms as GitLab does.
Anfang der weitergeleiteten E-Mail
mentioned in issue #214294
added 265 commits
-
02a3a08a...9c1a10f1 - 263 commits from branch
master
- 95d806e5 - Implement generating signed JWT to be used in CI
- f59e0ca7 - Document usage of CI_JOB_JWT
-
02a3a08a...9c1a10f1 - 263 commits from branch
added 2 commits
added workflowin review label and removed workflowin dev label
mentioned in merge request !25331 (closed)
- Resolved by Shinya Maeda
added 2 commits
I notice there is already an endpoint (undocummented?) to generate JWT used by the Docker registry :
curl -v -u gitlab-ci-token:$CI_JOB_TOKEN "https://gitlab.example.com/jwt/auth?account=gitlab-ci-token&client_id=any&offline_token=true&service=container_registry"
Unfortunatly the JWT cannot be validate because the certificate is not available, and there is no JWKS endpoint here.
Maybe good to be able to validate this JWT with the new JWKS endpoint (
https://gitlab.example.com/oauth/discovery/keys
).But these differents URL paths will be confusing. Or add alias/change the JWKS endpoint to /jwt/keys ?
Thanks !
mentioned in issue #28321 (closed)
- Resolved by Shinya Maeda
@krasio I answered. I'm pretty much fine with resolutions. I would likely ask to refine
subject
meaning to clearly indicate that subject isa build
, as this is an actual object that is generator of theJWT
. Can this be done in current arch?
unassigned @ayufan
- Resolved by Kamil Trzciński
@dosuken123 I looked at resolutions more from reviewer perspective. Would you be additional reviewer/maintainer of this?
- Resolved by Shinya Maeda
added 320 commits
-
0426f1ef...2fdb6b9c - 316 commits from branch
master
- 82180652 - Implement generating signed JWT to be used in CI
- 1a1463d9 - Document usage of CI_JOB_JWT
- 3f2bbb31 - Use build.id as subject for CI_JOB_JWT
- a9afd25c - Change default expire time for CI_JOB_JWT to 5 min
Toggle commit list-
0426f1ef...2fdb6b9c - 316 commits from branch
added 152 commits
-
a9afd25c...3b1a068f - 145 commits from branch
master
- e4d07947 - Implement generating signed JWT to be used in CI
- bf6e23e3 - Document usage of CI_JOB_JWT
- 3b194e33 - Use build.id as subject for CI_JOB_JWT
- be9e0379 - Change default expire time for CI_JOB_JWT to 5 min
- 8dcdd442 - Add new /-/jwks route used to validate CI_JOB_JWT
- 0dd976fb - Add warning to always restrict scope of Vault role
- 562e5c40 - Decouple Gitlab::Ci::JWT from JSONWebToken
Toggle commit list-
a9afd25c...3b1a068f - 145 commits from branch
mentioned in issue #214607 (closed)
assigned to @ayufan
@ayufan @krasio I've approved this MR as-is. My only concern is !28063 (comment 320994080), which I'm not entirely sure the security implication, so would you mind verifying it with
@gitlab-com/gl-security/appsec
once more in the follow-up issue? In case it turned out there is a security vulnerability, we can disable the token generation by disabling the feature flag, so at least we have a safe guard to quickly mitigate the problem.I looked at resolutions more from reviewer perspective. Would you be additional reviewer/maintainer of this?
I'm not sure what this means, but you ask me to merge this as a maintainer, I can merge.
- Resolved by Kamil Trzciński
unassigned @ayufan
assigned to @ayufan
@krasio I think it is good enough for us to merge it and iterate on it. I added a follow-up issue from unresolved comments.
mentioned in issue #214807 (closed)
marked the checklist item Code review guidelines as completed
marked the checklist item Merge request performance guidelines as completed
marked the checklist item Style guides as completed
marked the checklist item Separation of EE specific content as completed
marked the checklist item Database guides as completed
mentioned in commit fbf2cde3
@krasio can you advise is this in GitLab Core? I think that makes sense, but the issue once indicated GitLab Premium which was the original intent as teams are more likely to need a Vault and Directors are likely more interested in asking about our Secrets Management method.
This is acceptable in Core, I just need to make sure the issue reflects that.
Edited by Jackie Porter
added workflowstaging label and removed workflowin review label
added workflowcanary label and removed workflowstaging label
added workflowproduction label and removed workflowcanary label
mentioned in merge request !30702 (merged)
mentioned in issue #218692 (closed)
mentioned in merge request !34249 (merged)
mentioned in merge request !38762 (closed)
added security label
added docsimprovement label and removed 1 deleted label
mentioned in merge request gitlab-com/www-gitlab-com!61902 (merged)
mentioned in merge request !43950 (merged)
mentioned in issue #287824 (closed)
mentioned in merge request !72555 (merged)
mentioned in issue #451149 (closed)
mentioned in issue #428129