CI Pipelines fail for API commits using Project Access Token

mentioned in issue gitlab-org/quality/triage-reports#377 (closed)

mentioned in issue gitlab-org/quality/triage-reports#378 (closed)

mentioned in issue gitlab-org/quality/triage-reports#379 (closed)

added devopsmanage groupauthentication and authorization [DEPRECATED] priority3 typebug labels

We face a similar issue. We're using dependabot to create merge requests. The pipelines of those merge requests fail:

Running with gitlab-runner 13.3.1 (738bbe5a)
  on asdf on gce runner Yo_asV9p
Preparing the "shell" executor
Using Shell executor...
Preparing environment
Running on ufg-gitlab-runner-asdf-20200921a...
Getting source from Git repository
Fetching changes...
Reinitialized existing Git repository in /mnt/disks/local/builds/Yo_asV9p/0/sbg/sbg-fe/.git/
remote: HTTP Basic: Access denied
fatal: Authentication failed for 'https://gitlab-ci-token:[MASKED]@gitlab.ufirstgroup.com/sbg/sbg-fe.git/'

mentioned in issue #257766 (closed)

Hi @eblom and @mruoss ,

please have a look at #219551 (closed). I experienced the same problem with 13.3.x but after upgrading to 13.4.x, the pulling of the code in the job was successful. However, I discovered two further problems that may or may not affect you:

#259663 (closed) It seems that build jobs are still failing when your build job is based on Docker images from Gitlab's container registry. If you are using images from hub.docker.com, you should be fine.
#259665 (closed) If your build job is assigned to a runner / executor, where a previous build job has already fetched an older version of the repository, then the fetching of updates for that repository fails. However, if your job is doing a completely fresh fetch / clone, you should again be fine.

@mruoss I'm working on exactly the same scenario, of using project access tokens together with Dependabot.

Hi @klaasdellschaft,

we haven't updated to 13.4 as of yet, but reading your description we would immediately encounter #259663 (closed), since most of our jobs are based on docker images hosted in our own container registry. So we will probably continue avoiding using project access token in favour of a personal access token stored in a group variable for now

mentioned in issue #261974 (closed)

marked #259663 (closed) as a duplicate of this issue

marked this issue as related to #259663 (closed)

mentioned in issue #259663 (closed)

mentioned in issue gitlab-org/quality/triage-reports#468 (closed)

I've made a simple patch to resolve our simple use case.

How simple is our case ?

It's a single project and single job in a single pipeline, the job invokes a simple bash to deploy the code.

No container registry thing , no packages registry thing.

Show me your patch.

the patch is against version 13.4.3-ee (fd96f779), the original file can be found online here, and at /opt/gitlab/embedded/service/gitlab-rails/lib/gitlab/auth.rb on your local disk.

--- auth-original.rb	2020-10-14 06:00:00.000000000 +0000
+++ auth-patched.rb	2020-10-14 06:00:00.000000000 +0000
@@ -47,7 +47,7 @@
         # is enabled.
         result =
           service_request_check(login, password, project) ||
-          build_access_token_check(login, password) ||
+          build_access_token_check(login, password, project) ||
           lfs_token_check(login, password, project) ||
           oauth_access_token_check(login, password) ||
           personal_access_token_check(password, project) ||
@@ -276,16 +276,17 @@
         end
       end

-      def build_access_token_check(login, password)
+      def build_access_token_check(login, password, project)
         return unless login == CI_JOB_USER
         return unless password

         build = find_build_by_token(password)
         return unless build
         return unless build.project.builds_enabled?
+        return if project && build.user.project_bot? && !project.bots.include?(build.user)

         if build.user
-          return unless build.user.can?(:log_in)
+          return unless build.user.project_bot? || build.user.can?(:log_in)

           # If user is assigned to build, use restricted credentials of user
           Gitlab::Auth::Result.new(build.user, build.project, :build, build_authentication_abilities)

Why you SHOULD NOT use it (aka. KNOWN ISSUES).

In my opinion, this is a super super complex issue.

As far as I know, according to some of the documentation(I didn't read all the docs):

in pipelines, the git submodule may use the access token to update
multi-project pipelines may use the access token to trigger pipelines in another project

But, the project access token is bind to a bot user, which I CANNOT add to any other projects as a regular user. So this will cause those cross-project accessing failed.

And the patch above, surely doesn't have the ability to resolve those issues.

Plus, I've never wrote any Ruby code before, I'm not confident about it. But it indeed resolved the problem we were facing.

How does the patch work ?

The original problem is caused by return unless build.user.can?(:log_in).
Since the bot user can not login, so the API always returns 401.

In the patch, I'm checking the given token is bind to a bot user, and the bot is bind to the corresponding project which is running this particular job.
If all those conditions are true, then no more 401 will be responded.

mentioned in issue gitlab-org/quality/triage-reports#526 (closed)

mentioned in issue gitlab-org/quality/triage-reports#604 (closed)

mentioned in issue gitlab-org/quality/triage-reports#662 (closed)

mentioned in issue gitlab-org/quality/triage-reports#670 (closed)

mentioned in issue gitlab-org/quality/triage-reports#703 (closed)

Same here, trying to use project token to integrate Crowdin and GitLab. So, after bots that have been counted as payed seats, we now have bots that can only be used for what? Seriously guys, maybe less MVP and less features, but building features that work from start and are useful?

Yeah agreed! Assuming this was going to be possible (per descriptions on different tickets), we spent a lot of time building stuff using personal access tokens. Now that project access token is released we wanted to use it, but can't. Now we are stuck with either using my personal token, or waiting (what seems like forever) to see some progress on this ticket and #259663 (closed), but this issue is not even on backlog; so I guess we'll just wait.

@mushakov Dirk is facing HTTP basic :Access denied error while using project access token for gitlab-runner.He is a Gitlab EE ultimate customer & here's the internal link for the ticket(https://gitlab.zendesk.com/agent/tickets/179015). Any workaround or fix updates on the issue will be helpful.