Exclude Jira Connect server requests from unauthenticated IP throttle

Ashwin Srequested to merge
29996-exclude-jira-connect-server-requests-from-unauthenticated-throttle into master

What does this MR do and why?

The Jira development tool requests that Atlassian's servers send to the GitLab for Jira Cloud app no longer count against the unauthenticated per-IP rate limit (60 an hour) on GitLab.com.

The app authenticates with Atlassian JWTs. Only JiraConnect::ApplicationController checks them. The rack rate limiter never resolves them, so every request counts as anonymous against the client IP. Every Jira Cloud customer shares Atlassian's egress IPs, so they all share one 60-an-hour budget. When it runs out, Jira cannot list or link GitLab workspaces and repositories for anyone.

The excluded routes are the app's Jira development tool actions, which Atlassian's servers call: /-/jira_connect/workspaces/search, /-/jira_connect/repositories/search and /-/jira_connect/repositories/associate, plus their /o/:organization_path/ copies. The app descriptor registers these as searchConnectedWorkspaces, searchRepositories and associateRepository.

One day of observe-mode data (2xx responses only) shows about 2,495 successful requests a day on these routes would be blocked. Most are WorkspacesController#search (2,481 of 4,074). The full table is in the issue.

Known tradeoffs

  • Lifecycle events (/-/jira_connect/events/installed and /uninstalled) stay on the per-IP rule. EventsController verifies their asymmetric JWT by fetching Atlassian's public key over the network on every request, without a cache, before it checks the signature. Excluding them would let any anonymous client make Puma workers send outbound requests. Their volume is low (41 successful requests a day), and they were only flagged because they shared Atlassian's IPs with workspaces/search, which this MR removes from that budget. Caching the key by kid would be a separate follow-up for the integration's owners.
  • The browser-facing app pages (subscriptions, OAuth callbacks, branches, installations, app descriptor) stay on the per-IP rule. Their volume is small, and they load in users' browsers, so they count against users' own IPs. Specs pin that they still count.
  • The excluded routes stay under the general unauthenticated_web throttle. A request spec checks this.
  • Follow-up on the issue: resolve the symmetric JWT at the rack layer to the JiraConnectInstallation and count it as its own requester type. That would bring the development tool routes back under a per-installation limit. Open question: which plan tier an installation gets, since one installation can be linked to several namespaces.

References

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to https://gitlab.com/gitlab-com/gl-infra/production-engineering/-/work_items/29996

Edited by Ashwin S

Merge request reports