Backport of the audit event fix from 'Count CI job tokens sent as basic auth or bearer against the job user' to 19.3

Hardik Galarequested to merge
hgala/631338-skip-job-token-audit-19-3 into 19-3-stable-ee

What does this MR do and why?

This MR backports the audit event fix from Count CI job tokens sent as basic auth or beare... (!259720 - merged) to 19.3, for LabKit rack rate limiter emits user_authenticat... (#631338 - closed).

The rate limiters (LabKit and Rack::Attack) identify the requester through RequestAuthenticator before the endpoint authenticates the request. For a JOB-TOKEN request, each lookup streamed a user_authenticated_using_job_token audit event, and the endpoint streamed it again. On GitLab Dedicated, an instance that streams audit events with no event type filter got several streaming jobs per CI job-token request, and the jobs filled Redis.

The fix makes audit_job_token_authentication a no-op in the EE RequestAuthenticator. The endpoint still streams the event once per request.

Only part of the original MR is backported:

  • Included: the 5-line no-op in ee/lib/ee/gitlab/auth/request_authenticator.rb, and its spec for a JOB-TOKEN header.
  • Not included: the change to how job tokens sent as basic auth or bearer are counted, and its specs. That is a behavior change for the tier-aware rate limits, not part of this bug.

Why a backport

The fix is in 19.5 and later, and in 19.4.4 and later through the 19.4 backport Backport of the audit event fix from 'Count CI ... (!264557 - merged). It is not in 19.3.

The LabKit cohort flags are on by default from 19.3.0. So every 19.3 instance that streams audit events with no event type filter is exposed, also on self-managed. The only other workaround is an event type filter on each streaming destination, which the instance administrator must add.

This MR missed 19.3.7, which was already packaged. It targets 19.3.8 in the next patch release. The incident review and the decision to backport are in https://gitlab.com/gitlab-com/feature-change-locks/-/work_items/128+.

Turning off the six rate_limiter_use_labkit_rack_cohort_* flags is not a full workaround, because Rack::Attack resolves the requester through the same RequestAuthenticator path. This patch covers both paths.

How to verify

  • The new spec, ee/spec/lib/ee/gitlab/auth/request_authenticator_spec.rb, checks that a JOB-TOKEN lookup in RequestAuthenticator resolves the user without an audit call. Without the fix, the EE find_user_from_job_token calls audit_job_token_authentication, so the spec should fail on 19-3-stable-ee.
  • ee/spec/requests/api/ci/runner_spec.rb still expects the event once from the endpoint. It is unchanged, and the same spec passes on master with the fix.
  • The commit is a cherry-pick of the 19.4 backport commit. It applied to 19-3-stable-ee with no conflicts.
  • The specs and RuboCop run in CI. They were not run locally.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

  • This MR is backporting a bug fix, documentation update, or spec fix, previously merged in the default branch.
  • The MR that fixed the bug on the default branch has been deployed to GitLab.com (not applicable for documentation or spec changes).
  • The MR title is descriptive (e.g. "Backport of 'title of default branch MR'"). This is important, since the title will be copied to the patch blog post.
  • Required labels have been applied to this merge request
  • This MR has been approved by a maintainer (only one approval is required).
  • Ensure the E2E Omnibus GitLab EE pipeline (triggered by thee2e:test-on-omnibus-ee job) has succeeded. If it has failed (or shows warnings), investigate the failures. If you determine the failures are unrelated, you may proceed. If you need assistance investigating, request help in the #s_developer_experience Slack channel to confirm the failures are unrelated to the merge request.

Note to the merge request author and maintainer

If you have questions about the patch release process, please:

Merge request reports