[Audit Events] Provide all audit events under a group via the API
What does this MR do and why?
implements the query suggested by @ahegyi here
related to: #337757 (closed)
Query Plan
Query plan: https://explain.depesz.com/s/9INS
How to set up and validate locally
- Have an EE enabled GitLab instance
- find a grouo ID for a group you are owner of
- This group will need to have some
project
andgroup
audit events. You can create some from the rails console usingFactoryBot
:# using the admin user user = User.find(1); nil # flightjs/Flight project project = Project.find(7); nil # @flightjs group group = project.group; nil # create audit events FactoryBot.create(:project_audit_event, target_project: project, user: user); nil 2.times { FactoryBot.create(:group_audit_event, target_group: group, user: user) }; nil
- Using the group id, make a request to the API
curl -v \ --header "PRIVATE-TOKEN: $TOKEN" \ -H "Content-Type: application/json" \ -H "Accept: application/json, text/plain, */*" \ --url "localhost:3000/api/v4/groups/33/audit_events" | json_pp
- you should only see
Group
audit events in the response[ { "author_id" : 1, "created_at" : "2023-06-01T07:49:01.334Z", "details" : { "author_email" : "admin@example.com", "author_name" : "Jane Doe", "change" : "project_creation_level", "entity_path" : "flightjs", "from" : "", "ip_address" : "127.0.0.1", "target_details" : "Flightjs", "target_id" : 33, "target_type" : "Group", "to" : "Developers + Maintainers" }, "entity_id" : 33, "entity_type" : "Group", "id" : 5 }, { "author_id" : 1, "created_at" : "2023-06-01T07:49:01.321Z", "details" : { "author_email" : "admin@example.com", "author_name" : "Jane Doe", "change" : "project_creation_level", "entity_path" : "flightjs", "from" : "", "ip_address" : "127.0.0.1", "target_details" : "Flightjs", "target_id" : 33, "target_type" : "Group", "to" : "Developers + Maintainers" }, "entity_id" : 33, "entity_type" : "Group", "id" : 4 } ]
- enabled the feature flag with the rails console
Feature.enable(:audit_event_group_rollup)
- make the same request, you should now see the project audit event in the response also
[ { "author_id" : 1, "created_at" : "2023-06-01T07:49:01.334Z", "details" : { "author_email" : "admin@example.com", "author_name" : "Jane Doe", "change" : "project_creation_level", "entity_path" : "flightjs", "from" : "", "ip_address" : "127.0.0.1", "target_details" : "Flightjs", "target_id" : 33, "target_type" : "Group", "to" : "Developers + Maintainers" }, "entity_id" : 33, "entity_type" : "Group", "id" : 5 }, { "author_id" : 1, "created_at" : "2023-06-01T07:49:01.321Z", "details" : { "author_email" : "admin@example.com", "author_name" : "Jane Doe", "change" : "project_creation_level", "entity_path" : "flightjs", "from" : "", "ip_address" : "127.0.0.1", "target_details" : "Flightjs", "target_id" : 33, "target_type" : "Group", "to" : "Developers + Maintainers" }, "entity_id" : 33, "entity_type" : "Group", "id" : 4 }, { "author_id" : 1, "created_at" : "2023-06-01T07:48:45.375Z", "details" : { "author_email" : "admin@example.com", "author_name" : "Jane Doe", "change" : "packages_enabled", "entity_path" : "flightjs/Flight", "from" : "true", "ip_address" : "127.0.0.1", "target_details" : "Flight", "target_id" : 7, "target_type" : "Project", "to" : "false" }, "entity_id" : 7, "entity_type" : "Project", "id" : 3 } ]
Background Context
We want to be able to return all audit events under a given groups via a single endpoint. We have an issue, API endpoint to retrieve all audit events under a given group, to track this feature.
We actually had already implemented that feature, but the performance was so poor in production that it functionally did not work. The database query timed out almost 100% of the time.
The proposal was to improve the existing query's performance by using
the InOperatorOptimization
library.
Attempts to do this did not show a good performance boost (details here)
Another Approach
We then thought about implementing this feature with a new
approach. The Idea is that we can add a new column, group_rollup
, to
the audit_events
table. This column will be the id
of the
top-level group the audit event belongs to.
We could then update the Auditor to write this value at audit event creation, allowing us to implement the endpoint with a simple query:
SELECT *
FROM "audit_events"
WHERE "audit_events"."group_rollup_id" == @group.id
This query performed much better in database lab (details in this spike issue)
Back to InOperatorOptimization
In the MR for the group_rollup_id
approach, it was discovered that
with some tweaking and adding a new index, the InOperatorOptimization
might just work
So this commit adds the application code for that approach.
The index additions are tracked in these issues:
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.