Allows filtering of streaming audit events
What does this MR do and why?
Implements API to allow streamed audit events to be filtered on a per-destination basis
Introduce APIs to:
- List which events are being streamed to the given endpoint
query {
group(fullPath: "compliance-tanuki") {
id
externalAuditEventDestinations {
nodes {
destinationUrl
verificationToken
id
eventTypeFilters // collection of strings
}
}
}
}
- Cause the endpoint to receive a specific event
mutation createEventTypeFilter {
auditEventsStreamingDestinationEventsCreate(input: {
destinationId: "gid://gitlab/AuditEvents::ExternalAuditEventDestination/10",
eventTypeFilters: ["repository_download_operation"]
}){
errors
eventTypeFilters
}
}
- Cause the endpoint to no longer receive a specific event
mutation destroyEventTypeFilter {
auditEventsStreamingDestinationEventsDestroy(input: {
destinationId: "gid://gitlab/AuditEvents::ExternalAuditEventDestination/10",
eventTypeFilters: ["repository_download_operation"]
}){
errors
eventTypeFilters
}
}
Migration logs on local
❯ rake db:migrate:up:main VERSION=20220911200106
main: == 20220911200106 CreateAuditEventsStreamingEventTypeFilters: migrating =======
main: -- create_table(:audit_events_streaming_event_type_filters, {})
main: -- quote_column_name(:audit_event_type)
main: -> 0.0000s
main: -> 0.0353s
main: == 20220911200106 CreateAuditEventsStreamingEventTypeFilters: migrated (0.0360s)
❯ rake db:migrate:down:main VERSION=20220911200106
main: == 20220911200106 CreateAuditEventsStreamingEventTypeFilters: reverting =======
main: -- drop_table(:audit_events_streaming_event_type_filters, {})
main: -> 0.0024s
main: == 20220911200106 CreateAuditEventsStreamingEventTypeFilters: reverted (0.0039s)
Query Plans
rails query tested for 1000 records.
destination.event_type_filters.by_audit_event_type(event_type_filters).delete_all
query plan
EXPLAIN for: SELECT "audit_events_streaming_event_type_filters".* FROM "audit_events_streaming_event_type_filters" WHERE "audit_events_streaming_event_type_filters"."external_audit_event_destination_id" = 10 AND "audit_events_streaming_event_type_filters"."audit_event_type" IN ('4', '2', '3')
QUERY PLAN
----------------------------------------------------------------------------------------------------------------------------------------------------------------
Index Scan using idx_event_type_filters_on_external_audit_event_destination_id on audit_events_streaming_event_type_filters (cost=0.26..2.28 rows=1 width=34)
Index Cond: (external_audit_event_destination_id = 10)
Filter: (audit_event_type = ANY ('{4,2,3}'::text[]))
How to set up and validate locally
- Enable feature flag
allow_audit_event_type_filtering
- Setup audit event streaming destination https://docs.gitlab.com/ee/administration/audit_event_streaming.html#add-a-new-event-streaming-destination
- Perform audit action. for example: download repository using zip download button.
- Check streaming is working.
- Create event type filter using
auditEventsStreamingDestinationEventsCreate
mutation use any event name other than audit action we are performing. - Perform action again, audit event should be created but not streamed.
- Create event type filter using
auditEventsStreamingDestinationEventsCreate
mutation for the action we are performing. for example for zip download useeventTypeFilters: ["repository_download_operation"]
. - Perform action again, audit event should be created but streamed.
- Remove this filter using
auditEventsStreamingDestinationEventsDestroy mutation
- Perform action again, audit event should be created but not streamed.
- Remove all filters and perform any audit action, audit event should be created and streamed.
Numbered steps to set up and validate the change are strongly suggested.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Related to #344845 (closed)
Edited by Harsimar Sandhu