We have a project where we have multiple gitlab-ci.yml file that are used as include files for other projects. It's quite useful!
But we'd like to lint those files, through CI, of course. This is typically done through the CI lint API endpoint. That, unfortunately, seems to require an access token with Maintainer access and api scope. That's a broad access!
Furthermore, access tokens seem to expire after a year maximum, which means this pipeline breaks every year now.
Could we not use CI_JOB_TOKEN for this? The allowed endpoints explicitly include pipeline triggers, so I'm not sure why linting isn't allowed...
Required approvals
Any changes to the list of CI_JOB_TOKEN authentication scope should require AppSec approval/review.
This issue was automatically tagged with the label grouppipeline execution by TanukiStan, a machine learning classification model, with a probability of 0.93.
If this label is incorrect, please tag this issue with the correct group label as well as automation:ml wrong to help TanukiStan learn from its mistakes.
Authors who do not have permission to update labels can use the @gitlab-bot label ~"group::<correct group name>" command, or leave the issue to be triaged by group leaders initially assigned by TanukiStan.
This message was generated automatically.
You're welcome to improve it.
@jocelynjane I think it's a reasonable ask. I don't see any problem with adding access to the CI Lint endpoints. @marknuzzo Would there be any reason to not add CI Lint API endpoints to the list of endpoints that a CI_JOB_TOKEN could use?
The creator of this issue has two problems:
The CI Lint API endpoints require a maintainer role
A normal token with a maintainer role and full API access is too much scope for one token
The CI_JOB_TOKEN would be able to narrow the scope of what is accessible with the token, but it would still require the person who triggered the pipeline to a role that can use the CI Lint API endpoints (maintainer).
@marknuzzo I'm also wondering what the reasoning is for these endpoints requiring a maintainer role? They seem to be non-destructive endpoints.
I'm also wondering what the reasoning is for these endpoints requiring a maintainer role? They seem to be non-destructive endpoints.
Would there be any reason to not add CI Lint API endpoints to the list of endpoints that a CI_JOB_TOKEN could use?
Thanks for reaching out here @shampton - I agree that the CI Lint API endpoint seems to be non-destructive to me.
@furkanayhan @lauraX @avielle - would you happen to know why an access token would need maintainer access to use that endpoint and is that necessary if job token access is granted? If not, Scott, I think some additional investigation would need to be done here to determine why.
@marknuzzo the last time I worked with this endpoint was for a security issue where we were closing out some attack vectors, and if I recall correctly - some auth access was part of the solution. It might be worth to revisit this with someone from AppSec given this feature request.
@dcouture Since you were involved in those issues, do you now why the CI Lint API requires a Maintainer role? It seems OK to allow, say, a Developer user triggering a job, thus, a CI_JOB_TOKEN with that permission to access the CI Lint API. Do you see any issues with this?
POST /projects/:id/ci/lint requires the create_pipeline access.
GET /projects/:id/ci/lint requires the read_code access.
(of course, when using dry_run, the create pipeline logic is used and the protected branch policy can intervene)
For example, as a developer, I can do this for a project;
➜ curl --request GET --header "PRIVATE-TOKEN: glpat-XYZ"--url "https://gitlab.com/api/v4/projects/16477100/ci/lint?sha=dag&dry_run=true&ref=dag"{"valid":true,"errors":[],"warnings":[],"merged_yaml":"...","includes":[]}➜ curl --request GET --header "PRIVATE-TOKEN: glpat-XYZ"--url "https://gitlab.com/api/v4/projects/16477100/ci/lint?sha=master&dry_run=true&ref=master"{"valid":false,"errors":["You do not have sufficient permission to run a pipeline on 'master'. Please select a different branch or contact your administrator for assistance."],"warnings":[],"merged_yaml":null,"includes":null}
got what it's worth, i just found out about #389060 (closed) which is a proposal to allow JOB-TOKEN to push to the repository, a much far-ranging permission than linting.
It's possible to reveal CI variables from the CI lint API so that's why the create_pipeline access is required to make sure they already have access to the data that's being made available from the API.
On protected branches that's maintainer and the other ones it's developer.
@marknuzzo Since there is a risk of leaking CI/CD variables that are marked as protected, it makes sense to require Maintainer permission for CI_JOB_TOKEN to access the CI Lint API if linting is being done on a protected branch or a dry_run is being created.
The proposal mentioned in this issue can be addressed if we ask that CI Lint API be used to do only static checks to see if a project’s .gitlab-ci.yml configuration is valid on a non-protected branch. This only requires Developer role, based on the comments above. Doing a dry run may not be possible in this case as the .gitlab-ci.yml may refer to CI/CD variables that are marked as "protected" and would only be available if the call is being made by a Maintainer.
Thanks @rshambhuni for reaching out with sharing these summarized thoughts here. Adding @shampton to the discussion here as Scott is the EM for grouppipeline security. Scott - please add any thoughts you have.
@jocelynjane based on the above, I think we're okay with adding the CI Lint endpoints to the scope of the CI_JOB_TOKEN. Since the job token bases its access on the user that triggered the pipeline, we don't have to worry about the CI_JOB_TOKEN having more access than it should have in regards to the CI Lint endpoints. But to clarify the role requirements, I'll try and summarize:
Scenario
Access Requirements
GET /projects/:id/ci/lint
Able to read code - (Anybody for public projects, Authenticated for Internal, Guest+ private)
POST /projects/:id/ci/lint
Able to create a pipeline - (Developer+ non-protected branch, Maintainer+ if protected)
Using dry_run=true parameter
Able to create a pipeline - (Developer+ non-protected branch, Maintainer+ if protected)
So when we add these endpoints to the scope of the CI_JOB_TOKEN, the user will just need to make sure they meet these requirements for whatever they are wanting to do.
another thing I would add here is that it's pretty confusing for end users when private tokens expired. I have just had another job start failing because of this, and if it wasn't for this issue fresh in my mind, i would have been utterly confused as to why it failed. now i know i need to renew the token, but i can't help but feel "there must be a better way"...
@anarcat this issue is about CI_JOB_TOKEN. I believe your comment is related to PAT/GAT. @hsutor do you have a feedback issue for the annual expiration?
yeah, i know @jocelynjane - i filed this issue. :p i just wanted to clarify the reasoning behind the token in the first place: i would have less (but still other) need for this if non-expiring tickets would be available...
@anarcat You may be interested in some of our reasoning and ways to mitigate disruption, here is a post I wrote. The feedback is mostly on this issue: #369122 (closed)
Also, our token expiry isn't enforced until May, so I'm not sure why you'd be seeing that expiration happen now.
it could be a coincidence: we were doing an audit and possibly removed bots account created from access tokens; maybe I mentally mapped that to the expiry since, when I recreated the tokens, I remembered there was now a time limit.
This is on self-hosted, gitlab.torproject.org, as well, so maybe we're impacted differently than gitlab.com.
As for the rotation API: that's nice, but it won't do the "hard" part of updating the token in the CI variables (for example)...
I also share the observations from that issue that even suggesting to use Service Accounts as a workaround feels a bit like bait and switch (or, as was stated there, "gross"): we had a feature, it stops working, and we're offered to pay for a replacement...