Share etag cache keys between scoped and unscoped URLs

Rutger Wesselsrequested to merge
etag-cache-key-ignores-organization-path into master

What does this MR do and why?

GitLab uses ETag caching for polling endpoints. Polling means the browser asks the same URL again and again, to check for updates. Examples are the realtime_changes endpoints for issues and epics.

An ETag is a random stamp. GitLab stores it in Redis. The Redis key is the URL path of the request.

When the browser sends its old stamp, and Redis has the same stamp, the server answers "304 Not Modified". It sends no data. Rails code does not run.

When something changes, for example an epic title, GitLab writes a new stamp for that key. It builds the key with a URL helper, like realtime_changes_group_epic_path.

GitLab now has organization-scoped URLs. They look like /o/:organization_path/.... They show the same page as the URL without /o/.... URL helpers can return either form.

This caused a bug:

  1. The browser polls using one URL form.
  2. The code that writes a new stamp uses the other URL form.
  3. These two forms make two different Redis keys.
  4. The stamp the browser checks never changes.
  5. The browser gets "304 Not Modified" forever.
  6. The page never gets new data, and there is no error message.

In development and test, there is a second problem. Gitlab::EtagCaching::Store checks that each key matches a known URL pattern. The epic pattern only allows paths that start with /groups/. A scoped key does not match. This raises InvalidKeyError during GraphQL mutations, shown as a 500 error.

This MR adds a new method, Gitlab::Routing::OrganizationsHelper.unscoped_path. It removes the /o/:organization_path part from the start of a path. Gitlab::EtagCaching::Router::Rails and the store now use it. Polling and writing use the same Redis key again. GraphQL ETag keys, like /api/graphql:..., do not change.

This MR must merge before !254952 (merged). That MR makes URL helpers return organization-scoped URLs in more places, and its pipeline shows this bug.

References

How to set up and validate locally

  1. In a Rails console, with a non-default organization my-org and a group my-group with epic iid 1: 
    store = Gitlab::EtagCaching::Store.new
etag = store.touch('/o/my-org/groups/my-group/-/epics/1/realtime_changes')
store.get('/groups/my-group/-/epics/1/realtime_changes') == etag # => true; before this MR the touch raised InvalidKeyError
  2. Run the specs: 
    bin/rspec spec/lib/gitlab/routing/organizations_helper_spec.rb spec/lib/gitlab/etag_caching ee/spec/lib/ee/gitlab/etag_caching

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist.

🤖 Generated with Claude Code

Edited by Rutger Wessels

Merge request reports