[Backport 18.6] Exclude Git LFS paths from Git HTTP throttling

What does this MR do and why?

Problem

https://gitlab.com/gitlab-org/gitlab/-/work_items/581142 introduced changes to exclude Git HTTP and Git LFS from authenticated web rate limiting. Unfortunately, the Git HTTP throttling logic had a bug that became visible after this fix.

Details

Before the fix https://gitlab.com/gitlab-org/gitlab/-/work_items/581142

Git HTTP and Git LFS traffic was throttled by the Authenticated Web rate limiter. It was incorrect and caused a double throttling issue when several rate limiters were enabled (Authenticated Web rate limiter and Git HTTP rate limiter were counting Git HTTP traffic twice).

The fix resolved this issue and made Git HTTP traffic to be throttled only by Git HTTP rate limiter.

But this change uncovered a bug with Git HTTP rate limiter. It was throttling Git LFS traffic as well.

As a result, the Git HTTP rate limiter started throttling Git LFS requests.

Solution

Git LFS requests should not be subject to the same rate limiting as regular Git HTTP operations.

This change excludes LFS paths (info/lfs/objects/batch) from both authenticated and unauthenticated Git HTTP throttling to prevent legitimate LFS operations from being incorrectly rate limited.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• This MR is backporting a bug fix, documentation update, or spec fix, previously merged in the default branch.
• The MR that fixed the bug on the default branch has been deployed to GitLab.com (not applicable for documentation or spec changes).
• The MR title is descriptive (e.g. "Backport of 'title of default branch MR'"). This is important, since the title will be copied to the patch blog post.
• Required labels have been applied to this merge request
  • severity label and bug subtype labels (if applicable)
  • If this MR fixes a bug that affects customers, the customer label has been applied.
• This MR has been approved by a maintainer (only one approval is required).
• Ensure the e2e:test-on-omnibus-ee job has succeeded, or if it has failed, investigate the failures. If you determine the failures are unrelated, you may proceed. If you need assistance investigating, reach out to a Software Engineer in Test in #s_developer_experience.

Note to the merge request author and maintainer

If you have questions about the patch release process, please:

• Refer to the patch release runbook for engineers and maintainers for guidance.
• Ask questions on the #releases Slack channel (internal only).
• Once the backport has been merged, the commit changes will be automatically deployed to a release environment that can be used for manual validation. See after merging runbook for details.