Add keyset pagination to group audit events api
What does this MR do and why?
-
Adds optional support for keyset pagination for the
[GET] /api/v4/groups/:id/audit_events
endpoint. -
This is a non-breaking change as the default behviour does not change an existing API consumers need not make any changes.
-
Offset pagination for groups with large numbers of group-level audit events often causes timeouts when querying above record 50,000 (ish) due to poor database performance. (See postgres.ai internal link https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/10946/commands/39346)
-
A similar query using offset pagination is many orders of magnitude faster: https://console.postgres.ai/gitlab/gitlab-production-tunnel-pg12/sessions/10946/commands/39350 (22ms)
How to set up and validate locally
- Create a group (or use one that already exists)
- Ensure that it contains a lot of group-level audit events.
- Make an API query such as this (as a group owner or instance administrator)
GET /api/v4/groups/35/audit_events?per_page=2&pagination=keyset&order_by=created_at
- In the response headers, check the
Link
header which should be in this format:
<http://127.0.0.1:3000/api/v4/groups/35/audit_events?cursor=eyJpZCI6IjIwMTYzOSIsIl9rZCI6Im4ifQ%3D%3D&id=35&order_by=created_at&page=1&pagination=keyset&per_page=2>; rel="next"
- Make the same request again, adding
cursor={CURSOR_FROM_HEADER_ABOVE}
to the URL params and check that the results are the next ones in the set.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Relates to #333968 (closed)
Merge request reports
Activity
assigned to @mwoolf
Suggested Reviewers (beta)
The individuals below may be good candidates to participate in the review based on various factors.
You can use slash commands in comments to quickly assign
/assign_reviewer @user1
.Suggested Reviewers @smcgivern
,@eread
,@tigerwnz
,@tkuah
,@alexbuijs
If you do not believe these suggestions are useful, please apply the label Bad Suggested Reviewer. You can also provide feedback for this feature on this issue:
https://gitlab.com/gitlab-org/gitlab/-/issues/357923
.Automatically generated by Suggested Reviewers Bot - an experimental ML-based recommendation engine created by ~"group::applied ml".
Edited by GitLab Reviewer-Recommender Bot- A deleted user
added backend label
1 Warning e51a90ff: Commits that change 30 or more lines across at least 3 files should describe these changes in the commit body. For more information, take a look at our Commit message guidelines. 1 Message This merge request adds or changes documentation files. A review from the Technical Writing team before you merge is recommended. Reviews can happen after you merge. Documentation review
The following files require a review from a technical writer:
doc/api/audit_events.md
doc/api/index.md
The review does not need to block merging this merge request. See the:
-
Metadata for the
*.md
files that you've changed. The first few lines of each*.md
file identify the stage and group most closely associated with your docs change. - The Technical Writer assigned for that stage and group.
- Documentation workflows for information on when to assign a merge request for review.
Reviewer roulette
Changes that require review have been detected!
Please refer to the table below for assigning reviewers and maintainers suggested by Danger in the specified category:
Category Reviewer Maintainer backend Catalin Irimie ( @cat
) (UTC+1, same timezone as@mwoolf
)James Fargher ( @proglottis
) (UTC+12, 11 hours ahead of@mwoolf
)To spread load more evenly across eligible reviewers, Danger has picked a candidate for each review slot, based on their timezone. Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.
To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines. Please consider assigning a reviewer or maintainer who is a domain expert in the area of the merge request.
Once you've decided who will review this merge request, assign them as a reviewer! Danger does not automatically notify them for you.
If needed, you can retry the
danger-review
job that generated this comment.Generated by
Danger- Resolved by 🤖 GitLab Bot 🤖
@mwoolf - please add typebug typefeature, typemaintenance or a subtype label to this merge request.- typebug: Defects in shipped code and fixes for those defects. This includes all the bug types (availability, performance, security vulnerability, mobile, etc.)
- typefeature: Effort to deliver new features, feature changes & improvements. This includes all changes as part of new product requirements like application limits.
- typemaintenance: Up-keeping efforts & catch-up corrective improvements that are not Features nor Bugs. This includes restructuring for long-term maintainability, stability, reducing technical debt, improving the contributor experience, or upgrading dependencies.
See the handbook for more guidance on classifying.
This message was created with automation and Engineering Productivity is looking for feedback in this issue:
https://gitlab.com/gitlab-org/quality/engineering-productivity/team/-/issues/43
changed milestone to %15.2
removed workflowin dev label
removed missed:15.1 label
Allure report
allure-report-publisher
generated test report!review-qa-blocking:
test report for 8b053eacexpand test summary
+-----------------------------------------------------------------------------------------+ | suites summary | +------------------------------------+--------+--------+---------+-------+-------+--------+ | | passed | failed | skipped | flaky | total | result | +------------------------------------+--------+--------+---------+-------+-------+--------+ | Verify | 12 | 0 | 1 | 9 | 13 | ❗ | | Plan | 47 | 0 | 1 | 24 | 48 | ❗ | | Manage | 38 | 0 | 2 | 25 | 40 | ❗ | | Create | 23 | 0 | 2 | 16 | 25 | ❗ | | Protect | 2 | 0 | 0 | 2 | 2 | ❗ | | Feature flag handler sanity checks | 9 | 0 | 0 | 0 | 9 | ✅ | | Version sanity check | 0 | 0 | 1 | 0 | 1 | ➖ | | Configure | 0 | 0 | 1 | 0 | 1 | ➖ | | Package | 0 | 0 | 1 | 0 | 1 | ➖ | | Secure | 2 | 0 | 0 | 2 | 2 | ❗ | +------------------------------------+--------+--------+---------+-------+-------+--------+ | Total | 133 | 0 | 9 | 78 | 142 | ❗ | +------------------------------------+--------+--------+---------+-------+-------+--------+
- A deleted user
added documentation label
requested review from @eread
requested review from @manojmj
- Resolved by Manoj M J
- Resolved by Manoj M J
- Resolved by Max Woolf
added Technical Writing docsfeature labels
removed review request for @eread
mentioned in issue gitlab-com/www-gitlab-com#13517 (closed)
- Resolved by Jan Provaznik
Thanks @manojmj for pointing out the obvious flaw in my changes! I think I've addressed them but I'm keen for your feedback.
- Resolved by Max Woolf
- Resolved by Max Woolf
- Resolved by Max Woolf
removed review request for @manojmj
@manojmj
, thanks for approving this merge request.This is the first time the merge request is approved. To ensure full test coverage, a new pipeline has been started.
For more info, please refer to the following links:
requested review from @jprovaznik
- Resolved by Jan Provaznik
- Resolved by Jan Provaznik
removed review request for @jprovaznik
requested review from @eread
mentioned in issue #367528 (closed)
requested review from @jprovaznik
enabled an automatic merge when the pipeline for 93287c69 succeeds
added 1015 commits
-
4a76d45e...1a0eb315 - 1012 commits from branch
master
- a2baaf29 - Add group audit event keyset pagination
- e51a90ff - Backend reviewer suggestions
- 8b053eac - Apply 2 suggestion(s) to 2 file(s)
Toggle commit list-
4a76d45e...1a0eb315 - 1012 commits from branch