Instrument API pagination calls to track metrics
We are implementing keyset pagination in #32186 (closed). At some point, we would like to deprecate offset based pagination for high offsets and offer keyset based pagination instead.
We started to discuss deprecation strategy in gitlab-org/database-team/team-tasks#9 (comment 229585557).
We don't have enough data yet to understand the impact better. @joshlambert raised the question below:
Do we have data to help make a decision here?
- How urgent is it that we block offset pagination
- What % of our requests would we need to block, based on offset size? If it's a very small number that accounts for a lot of the (ab)use, I'd feel more comfortable with a shorter deprecation window.
- Do we have a sense of how common larger offsets are, to try to set a reasonable threshold here?
In this issue, we'll instrument the offset based pagination to gather metrics and data to help answer these questions.
Additionally, we should get data on which sorting-options are used for the individual API endpoints. We may end up supporting id
-based sorting only for keyset pagination, which would render other sorting-options unavailable for high offsets (where offset based pagination is discouraged/disabled).