Instrument API pagination calls to track metrics

We are implementing keyset pagination in #32186 (closed). At some point, we would like to deprecate offset based pagination for high offsets and offer keyset based pagination instead.

We started to discuss deprecation strategy in gitlab-org/database-team/team-tasks#9 (comment 229585557).

We don't have enough data yet to understand the impact better. @joshlambert raised the question below:

Do we have data to help make a decision here?

  • How urgent is it that we block offset pagination
  • What % of our requests would we need to block, based on offset size? If it's a very small number that accounts for a lot of the (ab)use, I'd feel more comfortable with a shorter deprecation window.
  • Do we have a sense of how common larger offsets are, to try to set a reasonable threshold here?

In this issue, we'll instrument the offset based pagination to gather metrics and data to help answer these questions.

Additionally, we should get data on which sorting-options are used for the individual API endpoints. We may end up supporting id-based sorting only for keyset pagination, which would render other sorting-options unavailable for high offsets (where offset based pagination is discouraged/disabled).

Assignee Loading
Time tracking Loading